One of the websites hosted on a server I currently manage has a cache directory which doesn’t take much space but tens of thousands of tiny files. Each second a dozen of files are created in the cache dir. Hence using a hard disk directory puts some serious load on the server consequence of the many fopen and fclose HDD I/O operations.
To get through the problem, the solution was obvious use a directory which stores its information in memory.
There are of course other benefits of using a memory to store data in as we all know as access to RAM is so many times faster.
GNU/Linux is equipped with a tmpfs since kernel version 2.4.x, primary usage of tmpfs file system across many G / Linux distributious is the /tmp directory.
Some general tmpfs information about tmpfs is explained in mount’s manual e.g.: man mount, a good other reading is the tmpfs kernel documentation file
An implementation of tmpfs is /dev/shm .
/dev/shm is a standard memory device used among Linuces, its actually an ordinary directory one can list with ls . /dev/shm is used as a “virtual directory” memory space. Below is an output of /dev/shm from my notebook, one can see few files stored in memory which belong to the pulse audio linux architecture:
linux:~$ ls -al /dev/shm
ls -al /dev/shm/total 7608drwxrwxrwt 2 root root 160 Oct 10 18:05 .
drwxr-xr-x 16 root root 3500 Oct 10 10:57 ..
p-w------- 1 root root 0 Oct 10 10:57 acpi_fakekey
-r-------- 1 hipo hipo 67108904 Oct 10 17:20 pulse-shm-2067018443
-r-------- 1 hipo hipo 67108904 Oct 10 10:59 pulse-shm-2840042043
-r-------- 1 hipo hipo 67108904 Oct 10 10:59 pulse-shm-3215031142
-r-------- 1 hipo hipo 67108904 Oct 10 18:05 pulse-shm-4157723670
-r-------- 1 hipo hipo 67108904 Oct 10 18:06 pulse-shm-702872358
To measure the size of /dev/shm across different Linux distriubtions one can use the usual df cmd, e.g.:
[root@centos: ~]$ df -h /dev/shm
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 0 16G 0% /dev/shm
Above I show a df -h /dev/shm output from a CentOS server equipped with 32 GB of memory, as you can see CentOS has reserved half of the size of the system memory (16GB) for the purposes of creating files in memory through /dev/shm. The memory is dynamically assigned, so if its not use the assigned memory by it can still be used for the purposes of the services running (which by the way is very nice).
Accoring to what, I’ve read in wikipedia about tmpfs, tmpfs defaults in Linux to half of the system physical memory.
However I’ve noticed Debian Linux hosts usually reserve less memory for /dev/shm, on my personal notebook Debian /dev/shm is only 1 Giga, where on a Debian running server, Debian automatically has set it to the humble 2GB. Setting less by the way as with the Debian example, is a rather good idea since not many desktop or server applications are written to get actively advantage of the virtual /dev/shm directory.
One can directly drop files in /dev/shm which will immediately be stored in memory, and after a system reboot the files will disappear.
Let’s say you zip archive file, testing.zip and you like to store the file in memory to do so, just copy the file in /dev/shm.
linux:~$ cp -rpf testing.zip /dev/shm
You don’t even need to be root to copy files in the “virtual memory directory”. This is a reason many crackers (script kiddies), are storing their cracking tools in /dev/shm 😉
A rather funny scenario, I’ve witness before is when you list /dev/shm on some Linux server and suddenly you see a tons of brute forcing tools and all kind of “hack” related stuff belonging to some system user. Sometimes even this malicious script tools belong to the root user…
Now as I’ve said a few words on how linux’s tmpfs works here is how to mount a directory which cache content will be stored in volatile memory:
linux:~# mount -t tmpfs -o size=3G,mode=0755 tmpfs /var/www/site/cache
As you can see the above command will dynamically assign a tmpfs directory taking up from the system RAM mem which could expand up to 3GB within the system memory.
Of course before mounting, its necessery to create the /var/www/site/cache and set proper permissions in the above example I use /var/www/site/cache with a default permissions of 755 which is owned by the use with which the Apache server is running, e.g.:
linux:~# mkdir -p /var/www/site/cache
linux:~# chown -R www-data:www-data /var/www/site/cache
linux:~# chmod -R 755 /var/www/site/cache
Using a tmpfs is very handy and have many advantages, however one should be very careful with the data stored inside a tmpfs dir resource, all data will be lost in case of sudden system restart as the data is stored in the memory.
One other problems one might expect with tmpfs would be if the assigned virtual disk space gets filled with data. It never happened to me but, I’ve red online some stories that in the past this led to system crashes, today as the dox I’ve checked prescribed overfilling it will start swapping make the system terribly sluggish and eventually afred depleting the reserver swap space will start killing processes.
Using tmpfs as a cache directory is very useful on servers running Apache+PHP/Perl/Python/Ruby etc. as it can be used for stroring script generated temorary data.
Using a tmpfs can signifantly decrease server i/o created disk overheads.
Some other application I can think of though, I haven’t tested it would be if tmpfs mounted directory is used to store scripting executable files, copied after restart. Executing the script reading it directly from the “virtual directory” could for sure have very good impact especially on huge websites.
One common service which takes advantage of the elegancy of tmpfs nowdays almost all modern GNU/Linux has is udevd – The Linux dynamic device management. By the way (man udev) is a very good and must read manual especially for Linux novices to get a good basic idea on how /dev/ mamagement occurs via udev.
To make permenant directory contained in memory on Linux the /etc/fstab file should be used.
In order to mount permanently a directory as a memory device of a size of 3GB with 0755 permissions in /var/www/site/cache, as shown in the earlier example, one can use the command:
linux:~# echo 'tmpfs /var/www/site/cache/ tmpfs size=3G,mode=0755 0 0' >> /etc/fstab
This will assure the directory stored in memory would be recreated on next boot.
Nowdays the use of tmpfs is constantly growing, I’ve seen it to be used as a way to substitute ordinary disk based /tmp with a tmpfs directory contained in memory in Cloud Linux OS.
The applications of tmpfs is pretty much to the imagination of the one who wants to get advantage of it. For sure using tmpfs will be seen by the Linux GUI programs.
Going to FreeBSD and the BSD world, tmpfs is also available, however it is still considered a bit experimental. To get use of tmpfs to gain some performance, one should first enable it via bsd’s /etc/rc.conf:
freebsd# echo 'tmpfs_load="YES"' >> /etc/rc.conf
Mounting a directory permanently using tmpfs persmanently it again is doable via /etc/fstab to add a new directory inside memory with tmpfs: is done with adding:
freebsd# echo 'tmpfs /var/www/site/cache tmpfs rw 0 0' >> /etc/fstab
The native equivallent of tmpfs in FreeBSD is called mdmfs.
As I said it is slower than tmpfs but rock solid.
To mount a 4gigabyte size mdmfs “ram directory” on BSD from csh:
freebsd# mdmfs -s 4g md /var/www/site/cache
Mounting a directory permanently using tmpfs persmanently it again is doable via /etc/fstab to add a new directory inside memory with tmpfs: is done with adding:
freebsd# echo 'tmpfs /var/www/site/cache tmpfs rw 0 0' >> /etc/fstab
The native equivallent of tmpfs in FreeBSD is called mdmfs.
As I said it is slower than tmpfs but rock solid.
To mount a 4gigabyte size mdmfs “ram directory” on BSD from csh:
freebsd# mdmfs -s 4g md /var/www/site/cache
Mounting a directory permanently using tmpfs persmanently it again is doable via /etc/fstab to add a new directory inside memory with tmpfs: is done with adding:
freebsd# echo 'md /var/www/site/cache mfs rw,-s4G 2 0' >> /etc/fstab
There are some reports of users who presumable use it to increase the ports / kernel compile times, but I haven’t tried it yet so I don’t know for sure
In huge corporations like Google and Yahoo tmpfs is certanly used a lot as this technology can dramatically can improve access times to information.I’m curious to know for some good ways to get use of tmpfs to improve efficiency.
If someone has some readings or has some idea please shar with me 😉
