Archive for March, 2012

Tiny PHP script to dump your browser set HTTP headers (useful in debugging)

Friday, March 30th, 2012

While browsing I stumbled upon a nice blog article

Dumping HTTP headers

The arcitle, points at few ways to DUMP the HTTP headers obtained from user browser.
As I'm not proficient with Ruby, Java and AOL Server what catched my attention is a tiny php for loop, which loops through all the HTTP_* browser set variables and prints them out. Here is the PHP script code:

<?php<br />
foreach($_SERVER as $h=>$v)<br />
if(ereg('HTTP_(.+)',$h,$hp))<br />
echo "<li>$h = $v</li>\n";<br />
header('Content-type: text/html');<br />

The script is pretty easy to use, just place it in a directory on a WebServer capable of executing php and save it under a name like:

If you don't want to bother copy pasting above code, you can also download the dump_HTTP_headers.php script here , rename the dump_HTTP_headers.php.txt to dump_HTTP_headers.php and you're ready to go.

Follow to the respective url to exec the script. I've installed the script on my webserver, so if you are curious of the output the script will be returning check your own browser HTTP set values by clicking here.
PHP will produce output like the one in the screenshot you see below, the shot is taken from my Opera browser:

Screenshot show HTTP headers.php script Opera Debian Linux

Another sample of the text output the script produce whilst invoked in my Epiphany GNOME browser is:

HTTP_USER_AGENT = Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Version/5.0 Safari/531.2+ Debian/squeeze (2.30.6-1) Epiphany/2.30.6
HTTP_ACCEPT = application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
HTTP_ACCEPT_LANGUAGE = en-us, en;q=0.90
HTTP_COOKIE = __qca=P0-2141911651-1294433424320;
__utma_a2a=8614995036.1305562814.1274005888.1319809825.1320152237.2021;wooMeta=MzMxJjMyOCY1NTcmODU1MDMmMTMwODQyNDA1MDUyNCYxMzI4MjcwNjk0ODc0JiYxMDAmJjImJiYm; 3ec0a0ded7adebfeauth=22770a75911b9fb92360ec8b9cf586c9;
__utmb=238407297.1.10.1333023754; __utmc=238407297;|utmccn=(referral)|utmcmd=referral|utmcct=/blog/

You see the script returns, plenty of useful information for debugging purposes:
HTTP_HOST – Virtual Host Webserver name
HTTP_USER_AGENT – The browser exact type useragent returnedHTTP_ACCEPT – the type of MIME applications accepted by the WebServerHTTP_ACCEPT_LANGUAGE – The language types the browser has support for
HTTP_ACCEPT_ENCODING – This PHP variable is usually set to gzip or deflate by the browser if the browser has support for webserver returned content gzipping.
If HTTP_ACCEPT_ENCODING is there, then this means remote webserver is configured to return its HTML and static files in gzipped form.
HTTP_COOKIE – Information about browser cookies, this info can be used for XSS attacks etc. 🙂
HTTP_COOKIE also contains the referrar which in the above case is:|utmccn=(referral)
The Cookie information HTTP var also contains information of the exact link referrar:

For the sake of comparison show_HTTP_headers.php script output from elinks text browser is like so:

* HTTP_USER_AGENT = Links (2.3pre1; Linux 2.6.32-5-amd64 x86_64; 143x42)
* HTTP_ACCEPT_ENCODING = gzip,deflate * HTTP_ACCEPT_CHARSET = us-ascii, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10, ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16, windows-1250, windows-1251, windows-1252, windows-1256,
windows-1257, cp437, cp737, cp850, cp852, cp866, x-cp866-u, x-mac, x-mac-ce, x-kam-cs, koi8-r, koi8-u, koi8-ru, TCVN-5712, VISCII,utf-8 * HTTP_ACCEPT_LANGUAGE = en,*;q=0.1
* HTTP_CONNECTION = keep-alive
One good reason, why it is good to give this script a run is cause it can help you reveal problems with HTTP headers impoperly set cookies, language encoding problems, security holes etc. Also the script is a good example, for starters in learning PHP programming.


Finally TeamViewer 7 release for GNU / Linux is available

Thursday, March 29th, 2012


TeamViewer 7 Debian GNU / Linux released, my desktop screenshot

After a long time of inavailability, finally TeamViewer 7 for GNU / Linux is out!

Here is the short TeamViewer 7 Linux release note :

In our TeamViewer download area you will now find TeamViewer 7 for Linux.
The final version includes a selection of new features for Linux, e. g. an integrated screenshot feature and the possibility of saving individual connection settings per computer.
You can now establish cross platform connections with other computers running version 7 and hold online meetings and presentations with up to 25 participants.

We wish you best continued success with TeamViewer!

Well, guys it was about time, even later than that… Until now I've so many times experienced troubles with Windows machines which I had to administrate remotely and they only had a TeamViewer 7 installed (a real, real pain in the ass).

The issue comes cause TeamViewer 6 is incompatible with 7 and just until very recently only TV 6 for Linux was available, so trying to access a 7 install directly from Linux was impossible.
Hence to access TeamViewer 7 Windows install, I had to use either a fresh Windows copy with TeamViewer 7 installed or a Virtual Machine like (Qemu, VirtualBox) with Windows version installed in it.
Often back then, when I had to fix remotely a Windows machine behind a firewall, I had to instruct the machine owners to replace the TeamViewer 7 with TeamViewer 6 in order to be able to easily access the remote host from my Debian Linux.

I still remember, it was quite a stuggle to find a link to a Windows .exe install file for TeamViewer 6.

Now thanksfully, TeamViewer guys are starting to make it easier for the user who would for some reason want to stick to older TV version.
I've noticed on TeamViewer's website there is already a new TeamViewer download page offering for download all the old teamviewer version – 1.x, 2.x, 3.x, 4.x, 5.x, 6.x

One example, where old teamviewer release is useful is if you want to run teamviewer on older hardware or old OS (MS Windows 98 or old Linux distro).

I've made a mirror of teamviewer 6 for Debian / Ubuntu amd 32 and 64 bit versions along with all the OS available TeamViewer version 6 and 7, for the sake of preserving it if one day link to old versions of Teamviewer disappear from their website.

Here are the TeamViewer mirrored files:

Below you see a screenshot of teamviewer 7 running on my Debian Squeeze 🙂

TeamViewer 7 Debian GNU / Linux released, my desktop screenshot

It is great teamviewer produced a Linux working application, however if you run it you realize just like the previous releases TeamViewer is not natively supposed (compiled) to run on GNU / Linux OS but uses wine (windows emulator) to launch through…
Instead of porting the application to be natively for the Linux distros once again with this new release, teamviewer developers just "hacked" it to run on top of windows emulation. Though this is working its performance, is probably a bit degraded and it depends on having install on the Linux host a bunch of useless packages which wine depends on.
Besides that even if it "works" on Linux , TeamViewer has still non-free software essense. I still use it because unfortunately, I don't know of a better remote access program capable to connect to servers behind NAT / machines located behind a tight firewalled routers.
If only (I knew of?) a TeamViewer free software / open source equivalent
I will be glad to hear if someone know a (free software / open source) TeamViewer like program able to access remote hosts with no a real (public inet) IP address?P.S.: By similar TV program I don't mean VNC, UltraVNC and this kind of other VNC derivative programs but mean close TeamViewer alternative.

Resolving “nf_conntrack: table full, dropping packet.” flood message in dmesg Linux kernel log

Wednesday, March 28th, 2012

On many busy servers, you might encounter in /var/log/syslog or dmesg kernel log messages like

nf_conntrack: table full, dropping packet

to appear repeatingly:

[1737157.057528] nf_conntrack: table full, dropping packet.
[1737157.160357] nf_conntrack: table full, dropping packet.
[1737157.260534] nf_conntrack: table full, dropping packet.
[1737157.361837] nf_conntrack: table full, dropping packet.
[1737157.462305] nf_conntrack: table full, dropping packet.
[1737157.564270] nf_conntrack: table full, dropping packet.
[1737157.666836] nf_conntrack: table full, dropping packet.
[1737157.767348] nf_conntrack: table full, dropping packet.
[1737157.868338] nf_conntrack: table full, dropping packet.
[1737157.969828] nf_conntrack: table full, dropping packet.
[1737157.969928] nf_conntrack: table full, dropping packet
[1737157.989828] nf_conntrack: table full, dropping packet
[1737162.214084] __ratelimit: 83 callbacks suppressed

There are two type of servers, I've encountered this message on:

1. Xen OpenVZ / VPS (Virtual Private Servers)
2. ISPs – Internet Providers with heavy traffic NAT network routers

I. What is the meaning of nf_conntrack: table full dropping packet error message

In short, this message is received because the nf_conntrack kernel maximum number assigned value gets reached.
The common reason for that is a heavy traffic passing by the server or very often a DoS or DDoS (Distributed Denial of Service) attack. Sometimes encountering the err is a result of a bad server planning (incorrect data about expected traffic load by a company/companeis) or simply a sys admin error…

– Checking the current maximum nf_conntrack value assigned on host:

linux:~# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_max

– Alternative way to check the current kernel values for nf_conntrack is through:

linux:~# /sbin/sysctl -a|grep -i nf_conntrack_max
error: permission denied on key 'net.ipv4.route.flush'
net.netfilter.nf_conntrack_max = 65536
error: permission denied on key 'net.ipv6.route.flush'
net.nf_conntrack_max = 65536

– Check the current sysctl nf_conntrack active connections

To check present connection tracking opened on a system:


linux:~# /sbin/sysctl net.netfilter.nf_conntrack_count
net.netfilter.nf_conntrack_count = 12742

The shown connections are assigned dynamicly on each new succesful TCP / IP NAT-ted connection. Btw, on a systems that work normally without the dmesg log being flooded with the message, the output of lsmod is:

linux:~# /sbin/lsmod | egrep 'ip_tables|conntrack'
ip_tables 9899 1 iptable_filter
x_tables 14175 1 ip_tables

On servers which are encountering nf_conntrack: table full, dropping packet error, you can see, when issuing lsmod, extra modules related to nf_conntrack are shown as loaded:

linux:~# /sbin/lsmod | egrep 'ip_tables|conntrack'
nf_conntrack_ipv4 10346 3 iptable_nat,nf_nat
nf_conntrack 60975 4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4
nf_defrag_ipv4 1073 1 nf_conntrack_ipv4
ip_tables 9899 2 iptable_nat,iptable_filter
x_tables 14175 3 ipt_MASQUERADE,iptable_nat,ip_tables


II. Remove completely nf_conntrack support if it is not really necessery

It is a good practice to limit or try to omit completely use of any iptables NAT rules to prevent yourself from ending with flooding your kernel log with the messages and respectively stop your system from dropping connections.

Another option is to completely remove any modules related to nf_conntrack, iptables_nat and nf_nat.
To remove nf_conntrack support from the Linux kernel, if for instance the system is not used for Network Address Translation use:

/sbin/rmmod iptable_nat
/sbin/rmmod ipt_MASQUERADE
/sbin/rmmod rmmod nf_nat
/sbin/rmmod rmmod nf_conntrack_ipv4
/sbin/rmmod nf_conntrack
/sbin/rmmod nf_defrag_ipv4

Once the modules are removed, be sure to not use iptables -t nat .. rules. Even attempt to list, if there are any NAT related rules with iptables -t nat -L -n will force the kernel to load the nf_conntrack modules again.

Btw nf_conntrack: table full, dropping packet. message is observable across all GNU / Linux distributions, so this is not some kind of local distribution bug or Linux kernel (distro) customization.

III. Fixing the nf_conntrack … dropping packets error

– One temporary, fix if you need to keep your iptables NAT rules is:

linux:~# sysctl -w net.netfilter.nf_conntrack_max=131072

I say temporary, because raising the nf_conntrack_max doesn't guarantee, things will get smoothly from now on.
However on many not so heavily traffic loaded servers just raising the net.netfilter.nf_conntrack_max=131072 to a high enough value will be enough to resolve the hassle.

– Increasing the size of nf_conntrack hash-table

The Hash table hashsize value, which stores lists of conntrack-entries should be increased propertionally, whenever net.netfilter.nf_conntrack_max is raised.

linux:~# echo 32768 > /sys/module/nf_conntrack/parameters/hashsize
The rule to calculate the right value to set is:
hashsize = nf_conntrack_max / 4

– To permanently store the made changes ;a) put into /etc/sysctl.conf:

linux:~# echo 'net.netfilter.nf_conntrack_count = 131072' >> /etc/sysctl.conf
linux:~# /sbin/sysct -p

b) put in /etc/rc.local (before the exit 0 line):

echo 32768 > /sys/module/nf_conntrack/parameters/hashsize

Note: Be careful with this variable, according to my experience raising it to too high value (especially on XEN patched kernels) could freeze the system.
Also raising the value to a too high number can freeze a regular Linux server running on old hardware.

– For the diagnosis of nf_conntrack stuff there is ;

/proc/sys/net/netfilter kernel memory stored directory. There you can find some values dynamically stored which gives info concerning nf_conntrack operations in "real time":

linux:~# cd /proc/sys/net/netfilter
linux:/proc/sys/net/netfilter# ls -al nf_log/

total 0
dr-xr-xr-x 0 root root 0 Mar 23 23:02 ./
dr-xr-xr-x 0 root root 0 Mar 23 23:02 ../
-rw-r--r-- 1 root root 0 Mar 23 23:02 0
-rw-r--r-- 1 root root 0 Mar 23 23:02 1
-rw-r--r-- 1 root root 0 Mar 23 23:02 10
-rw-r--r-- 1 root root 0 Mar 23 23:02 11
-rw-r--r-- 1 root root 0 Mar 23 23:02 12
-rw-r--r-- 1 root root 0 Mar 23 23:02 2
-rw-r--r-- 1 root root 0 Mar 23 23:02 3
-rw-r--r-- 1 root root 0 Mar 23 23:02 4
-rw-r--r-- 1 root root 0 Mar 23 23:02 5
-rw-r--r-- 1 root root 0 Mar 23 23:02 6
-rw-r--r-- 1 root root 0 Mar 23 23:02 7
-rw-r--r-- 1 root root 0 Mar 23 23:02 8
-rw-r--r-- 1 root root 0 Mar 23 23:02 9


IV. Decreasing other nf_conntrack NAT time-out values to prevent server against DoS attacks

Generally, the default value for nf_conntrack_* time-outs are (unnecessery) large.
Therefore, for large flows of traffic even if you increase nf_conntrack_max, still shorty you can get a nf_conntrack overflow table resulting in dropping server connections. To make this not happen, check and decrease the other nf_conntrack timeout connection tracking values:

linux:~# sysctl -a | grep conntrack | grep timeout
net.netfilter.nf_conntrack_generic_timeout = 600
net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 120
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 60
net.netfilter.nf_conntrack_tcp_timeout_established = 432000
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_last_ack = 30
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close = 10
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300
net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300
net.netfilter.nf_conntrack_udp_timeout = 30
net.netfilter.nf_conntrack_udp_timeout_stream = 180
net.netfilter.nf_conntrack_icmp_timeout = 30
net.netfilter.nf_conntrack_events_retry_timeout = 15
net.ipv4.netfilter.ip_conntrack_generic_timeout = 600
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent2 = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_recv = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 432000
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_last_ack = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_max_retrans = 300
net.ipv4.netfilter.ip_conntrack_udp_timeout = 30
net.ipv4.netfilter.ip_conntrack_udp_timeout_stream = 180
net.ipv4.netfilter.ip_conntrack_icmp_timeout = 30

All the timeouts are in seconds. net.netfilter.nf_conntrack_generic_timeout as you see is quite high – 600 secs = (10 minutes).
This kind of value means any NAT-ted connection not responding can stay hanging for 10 minutes!

The value net.netfilter.nf_conntrack_tcp_timeout_established = 432000 is quite high too (5 days!)
If this values, are not lowered the server will be an easy target for anyone who would like to flood it with excessive connections, once this happens the server will quick reach even the raised up value for net.nf_conntrack_max and the initial connection dropping will re-occur again …

With all said, to prevent the server from malicious users, situated behind the NAT plaguing you with Denial of Service attacks:

Lower net.ipv4.netfilter.ip_conntrack_generic_timeout to 60 – 120 seconds and net.ipv4.netfilter.ip_conntrack_tcp_timeout_established to stmh. like 54000

linux:~# sysctl -w net.ipv4.netfilter.ip_conntrack_generic_timeout = 120
linux:~# sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 54000

This timeout should work fine on the router without creating interruptions for regular NAT users. After changing the values and monitoring for at least few days make the changes permanent by adding them to /etc/sysctl.conf

linux:~# echo 'net.ipv4.netfilter.ip_conntrack_generic_timeout = 120' >> /etc/sysctl.conf
linux:~# echo 'net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 54000' >> /etc/sysctl.conf

Tracking I/O hard disk server bottlenecks with iostat on GNU / Linux and FreeBSD

Tuesday, March 27th, 2012

Hard disk overhead tracking on Linux and FreeBSD with iostat

I've earlier wrote an article How to find which processes are causing hard disk i/o overhead on Linux there I explained very rawly few tools which can be used to benchmark hard disk read / write operations. My prior article accent was on iotop and dstat and it just mentioned of iostat. Therefore I've wrote this short article in attempt to explain a bit more thoroughfully on how iostat can be used to track problems with excessive server I/O read/writes.

Here is the command man page description;
iostatReport Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems

I will further proceed with few words on how iostat can be installed on various Linux distros, then point at few most common scenarious of use and a short explanation on the meaning of each of the command outputs.

1. Installing iostat on Linux

iostat is a swiss army knife of finding a server hard disk bottlenecks. Though it is a must have tool in the admin outfut, most of Linux distributions will not have iostat installed by default.
To have it on your server, you will need to install sysstat package:

a) On Debian / Ubuntu and other Debian GNU / Linux derivatives to install sysstat:

debian:~# apt-get --yes install sysstat

b) On Fedora, CentOS, RHEL etc. install is with yum:

[root@centos ~]# yum -y install sysstat

c) On Slackware Linux sysstat package which contains iostat is installed by default. 

d) In FreeBSD, there is no need for installation of any external package as iostat is part of the BSD world (bundle commands).
I should mention bsd iostat and Linux's iostat commands are not the same and hence there use to track down hard disk bottlenecks differs a bit, however the general logic of use is very similar as with most tools in BSD and Linux.

2. Checking a server hard disk for i/o disk bottlenecks on G* / Linux

Once having the sysstat installed on G* / Linux systems, the iostat command will be added in /usr/bin/iostat
a) To check what is the hard disk read writes per second (in megabytes) use:

debian:~# /usr/bin/iostat -m
Linux 2.6.32-5-amd64 (debian) 03/27/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
15.34 0.36 2.76 2.66 0.00 78.88
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 63.89 0.48 8.20 6730223 115541235
sdb 64.12 0.44 8.23 6244683 116039483
md0 2118.70 0.22 8.19 3041643 115528074

In the above output the server, where I issue the command is using sda and sdb configured in software RAID 1 array visible in the output as (md0)

The output of iostat should already be easily to read, for anyone who didn't used the tool here is a few lines explanation of the columns:

The %user 15.34 meaning is that 15.34 out of 100% possible i/o load is generad by system level read/write operations.
%nice – >Show the percentage of CPU utilization that occurred while executing at the user level with nice priority.
%iowait – just like the top command idle it shows the idle time when the system didn't have an outstanding disk I/O requests.
%steal – show percentage in time spent in time wait of CPU or virtual CPUs to service another virtual processor (high numbers of disk is sure sign for i/o problem).
%idle – almost the same as meaning to %iowait
tps – HDD transactions per second
MB_read/s (column) – shows the actual Disk reads in Mbytes at the time of issuing iostat
MB_wrtn/s – displays the writes p/s at the time of iostat invocation
MB_read – shows the hard disk read operations in megabytes, since the server boot 'till moment of invocation of iostat
MB_wrtn – gives the number of Megabytes written on HDD since the last server boot filesystem mount

The reason why the Read / Write values for sda and sdb are similar in this example output is because my disks are configured in software RAID1 (mirror)

The above iostat output reveals in my specific case the server is experiencing mostly Disk writes (observable in the high MB_wrtn/s 8.19 md0 in the above sample output).

It also reveals, the I/O reads experienced on that server hard disk are mostly generated as a system (user level load) – see (%user 15.34 and md0 2118.70).

For all those not familiar with system also called user / level load, this is all kind of load which is generated by running programs on the server – (any kind of load not generated by the Linux kernel or loaded kernel modules).

b) To periodically keep an eye on HDD i/o operations with iostat, there are two ways:

– Use watch in conjunction with iostat;

[root@centos ~]# watch "/usr/bin/iostat -m"
Every 2.0s: iostat -m Tue Mar 27 11:00:30 2012
Linux 2.6.32-5-amd64 (centos) 03/27/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
15.34 0.36 2.76 2.66 0.00 78.88
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 63.89 0.48 8.20 6730255 115574152
sdb 64.12 0.44 8.23 6244718 116072400
md0 2118.94 0.22 8.20 3041710 115560990
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 55.00 0.01 25.75 0 51
sdb 52.50 0.00 24.75 0 49
md0 34661.00 0.01 135.38 0 270

Even though watch use and -d might appear like identical, they're not watch does refresh the screen, executing instruction similar to the clear command which clears screen on every 2 seconds, so the output looks like the top command refresh, while passing the -d 2 will output the iostat command output on every 2 secs in a row so all the data is visualized on the screen. Hence -d 2 in cases, where more thorough debug is necessery is better. However for a quick routine view watch + iostat is great too.

c) Outputting extra information for HDD input/output operations;

root@debian:~# iostat -x
Linux 2.6.32-5-amd64 (debian) 03/27/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
15.34 0.36 2.76 2.66 0.00 78.88
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 4.22 2047.33 12.01 51.88 977.44 16785.96 278.03 0.28 4.35 3.87 24.72
sdb 3.80 2047.61 11.97 52.15 906.93 16858.32 277.05 0.03 5.25 3.87 24.84
md0 0.00 0.00 20.72 2098.28 441.75 16784.05 8.13 0.00 0.00 0.00 0.00

This command will output extended useful Hard Disk info like;
r/s – number of read requests issued per second
w/s – number of write requests issued per second
rsec/s – numbers of sector reads per second
b>wsec/s – number of sectors wrote per second
etc. etc.

Most of ppl will never need to use this, but it is good to know it exists.

3. Tracking read / write (i/o) hard disk bottlenecks on FreeBSD

BSD's iostat is a bit different in terms of output and arguments.

a) Here is most basic use:

freebsd# /usr/sbin/iostat
tty ad0 cpu
tin tout KB/t tps MB/s us ni sy in id
1 561 45.18 44 1.95 14 0 5 0 82

b) Periodic watch of hdd i/o operations;

freebsd# iostat -c 10
tty ad0 cpu
tin tout KB/t tps MB/s us ni sy in id
1 562 45.19 44 1.95 14 0 5 0 82
0 307 51.96 113 5.73 44 0 24 0 32
0 234 58.12 98 5.56 16 0 7 0 77
0 43 0.00 0 0.00 1 0 0 0 99
0 485 0.00 0 0.00 2 0 0 0 98
0 43 0.00 0 0.00 0 0 1 0 99
0 43 0.00 0 0.00 0 0 0 0 100

As you see in the output, there is information like in the columns tty, tin, tout which is a bit hard to comprehend.
Thanksfully the tool has an option to print out only more essential i/o information:

freebsd# iostat -d -c 10
KB/t tps MB/s
45.19 44 1.95
58.12 97 5.52
54.81 108 5.78
0.00 0 0.00
0.00 0 0.00
0.00 0 0.00
20.48 25 0.50

The output info is quite self-explanatory.

Displaying a number of iostat values for hard disk reads can be also achieved by omitting -c option with:

freebsd# iostat -d 1 10

Tracking a specific hard disk partiotion with iostat is done with:

freebsd# iostat -n /dev/ad0s1a
tty cpu
tin tout us ni sy in id
1 577 14 0 5 0 81
c) Getting Hard disk read/write information with gstat

gstat is a FreeBSD tool to print statistics for GEOM disks. Its default behaviour is to refresh the screen in a similar fashion like top command, so its great for people who would like to periodically check all attached system hard disk and storage devices:

freebsd# gstat
dT: 1.002s w: 1.000s
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
0 10 0 0 0.0 10 260 2.6 15.6| ad0
0 10 0 0 0.0 10 260 2.6 11.4| ad0s1
0 10 0 0 0.0 10 260 2.8 12.5| ad0s1a
0 0 0 0 0.0 0 0 0.0 20.0| ad0s1b
0 0 0 0 0.0 0 0 0.0 0.0| ad0s1c
0 0 0 0 0.0 0 0 0.0 0.0| ad0s1d
0 0 0 0 0.0 0 0 0.0 0.0| ad0s1e
0 0 0 0 0.0 0 0 0.0 0.0| acd0

It even has colors if your tty supports colors 🙂

Another useful tool in debugging the culprit of excessive hdd I/O operations is procstat command:

Here is a sample procstat run to track (httpd) one of my processes imposing i/o hdd load:

freebsd# procstat -f 50404
50404 httpd cwd v d -------- - - - /
50404 httpd root v d -------- - - - /
50404 httpd 0 v c r------- 56 0 - -
50404 httpd 1 v c -w------ 56 0 - -
50404 httpd 2 v r -wa----- 56 75581 - /var/log/httpd-error.log
50404 httpd 3 s - rw------ 105 0 TCP ::.80 ::.0
50404 httpd 4 p - rw---n-- 56 0 - -
50404 httpd 5 p - rw------ 56 0 - -
50404 httpd 6 v r -wa----- 56 25161132 - /var/log/httpd-access.log
50404 httpd 7 v r rw------ 56 0 - /tmp/apr8QUOUW
50404 httpd 8 v r -w------ 56 0 - /var/run/accept.lock.49588
50404 httpd 9 v r -w------ 1 0 - /var/run/accept.lock.49588
50404 httpd 10 v r -w------ 1 0 - /tmp/apr8QUOUW
50404 httpd 11 ? - -------- 2 0 - -

Btw fstat is sometimes helpful in identifying the number of open files and trying to estimate which ones are putting the hdd load.
Hope this info helps someone. If you know better ways to track hdd excessive loads on Linux / BSD pls share 'em pls.

The Living of Saint Peter and Saint Fevronia – a fascinating Russian cartoon retelling the saints story

Monday, March 26th, 2012

Saint Peter and Fevronia Orthodox Christian saints protector of happy family, love and blessed marriage

The anime Living Story of Saint Peter and Saint Fevronia is a modern story remake of an ancient Church living of two bright Russian saints.
The official movie genre is orthodox christian kids animation movie.
The 14 minutes cartoon re-tolds the living story of St. Peter and st, Fevronia in a playful and entertaining way like for kids or youngsters.
Nomatter the movie primary target audience is children, the cartoon is great to see for adults people as well :).
The movie genre is orthodox christian kids animation movie.
The plot is based on a true (historic record) story of two saints venerated each year across Orthodox Churches around the world.

The original story I watched was a Bulgarian translation from Russian. But since I found it to be so valuable, I look for a translated video and got one in youtube.Take 14 minutes break and watch it, I'm sure you will like it so much, that probably give it a second time glimpse alone or with your wife, children or girlfriend.

Unfortunately, the english title is mis-translated as it says "Tale" and not "Story" and there is difference in meaning between this two words.
It is not tale as tale is made up story and this is not a made story but a story based on the two saints who lived in the end of 12 and beginning of the thirteen century.
Here is in short the real Church living:
The two saints were living in Murom Russia. Peter was prince and Fevronia a poor maid a daughter of a beekeeper who made his family living by collecting wild honey in the forest.

Prince Peter was striken by a severe sickness and in a vision it was revealed to him, that the only one that can cure him is Fevronia (a village maid living in the village of Laskovo Russia. The prince went to her and since he saw she is a pious, good and wisdom rich maid promised her, if she manage to heal him to take her as a bride to his place.
By her warm prayers to God and herbs, Fevronia succeeded in healing the prince sickness. Being fully restored st.Peter changed his mind and wanted to break his promise to marry her, cause the young made not part of the aristocratic Russian society. He didn't yet reached his home and the sickness, came back. This time with a deep repentance, he came back to Fevronia and she cured him again. Then the prince merried her and made her a princess of Murom

St. Peter and Fevronia Orthodox Church saints protector of marriage

However there love in Christ had to went through high temptations. Once the couple married, the prince proud boyars requested the prince to leave his new bride, as they didn't wanted to accept a simple girl as Fevronia will be governing them. Being in uneasy situation prince Peter prefered to leave his governing power and castle but to stay with his life. Together by boat by the near river Oka they left the kingdom. Soon after Gods wrath came Murom because of people's rebellion and the people requested the chased prince family to be restored to power.
Pushed by the peasents, the boyars bringed back the couple to power. The two saints governed their kingdom with great wisdom, love and mercy to the people.

In their old age the decided to become monks in separate monasteries. St. Peter received his new monk name David and princess Fevronia took the nun name Evphrosia (Evfrosia). Even living a sepate monk / nun life the couple continued having a deep love to each other and asked God to take them from this earthly life on the same date. God answered their prayer granting them to depart this earth on the same day in the same hour!
Like in life even in death people tried to separate them.
Fevronia was put in a coffin in the nun monastery, where Peter was prepared for a monk funeral in the man monastery and they put them in separate graves.
In the morning the graves were empty and their bodies were found buried together in one grave. People realized it is Gods will they are buried together and left them buried together. Today the incorruptable bodies of the two saints can be seen and venerated in Holy Trinity's nun monastery in the town of Murom Russia. St. Peter and St. Fevronia are considered patron of the Christian marriage, couple's love and family happiness.
It is common that many young people are, asking for the two saints prayer intercession in front of God for getting a good spouse in life and good marriage.

Also the two saints are oftenly asked for prayer for improving a marriege bindings.
The two saints feast day is like the Orthodox Antipode of the Roman Catholic feast of couples in love – St. Valentine.

As you see, St. Peter and St. Fevronia living is full of wisdom and true spirituality, and there is plenty we the modern disbelieving people can learn from it.
Let God by the two saints holy prayers have mercy on us.

How to reduce spam in PHPBB based internet forum on Debian GNU / Linux

Monday, March 26th, 2012

phpbb reduce spam bot registrations on Debian Linux tiny script

I had to install two PHPBB based internet forums, some long time ago. Since long time passed and I haven't checked what's happening with them I just noticed. They start filling up spam threads. The phpbb installations are done using the standard shipped deb packages in Debian Linux Lenny.

After checking online, I found one smart solution to . The idea is very simple most spam bots are written in a way that they don't have a properly set timezone. Therefore the quickest way to get rid of spam bots which try to auto register and put spam content inside the a forum category or post is to add a simple if condition in php to check the browser set timezone:

The file to add the php if condition is ucp_register.phpThe phpbb package install places default phpbb path on Debian is /usr/share/phpbb3/ and hence the file I had to modify is located in:


To make the TZ check one needs to modify ../www/includes/ucp/ucp_register.php and look for php array definition:

$data = array(
'username' => utf8_normalize_nfc(request_var('username', '', true)),
'new_password' => request_var('new_password', '', true),
'password_confirm' => request_var('password_confirm', '', true),
'email' => strtolower(request_var('email', '')),
'email_confirm' => strtolower(request_var('email_confirm', '')),
'confirm_code' => request_var('confirm_code', ''),
'lang' => basename(request_var('lang', $user->lang_name)),
'tz' => request_var('tz', (float) $timezone),

Right after this chunk of code add the if condition code which is like so:

if ($data['tz'] == '-12.00')
die('Die, bot! Die.');

From now onwards, any attempt for new user registration with an incorrect timezone of -12.00 will be immediately stopped while the forum spammer bot will be offered an empty page 🙂

Another good practice is to disable Birthday Listing from phpbb Admin Control panel (ACP). Go to menus:

ACP -> General -> Board Settings -> Enable Birthday listing: (No)

Enable birthday listing phpbb forum screenshot

I like disabling birthday listing, as when it is enabled and you have some spammer registrations, which even though didn't succeeded to contaminate your forum content has specified a birthday and therefore there profiles gets popping up each different day on the main page of the forum.
This will not eradicate all spammer bots, but at least will significantly decrease spammer bot registrations.

Double Feast in our Bulgarian Orthodox Church – feast of Annunciation and comemmoration of St. John Lesstvichnik

Monday, March 26th, 2012

One more year, we are into the great Lent and we're given the Grace to hear the Good tidings (Good News) handed down by Archangel Gabriel to our Virgin Lady (The Mother of God). We hear the GOOD NEWS that our Lord Jesus Christ is conceived immaculately and is to be born from the Holy Theotokos most pure body:

Here is a little part of the Gospel New Testamention reading from Luke for the day:

28 The angel went to her and said, Greetings, you who are highly favored! The Lord is with you.
29 Mary was greatly troubled at his words and wondered what kind of greeting this might be.
30 But the angel said to her, Do not be afraid, Mary; you have found favor with God.
31 You will conceive and give birth to a son, and you are to call him Jesus.
32 He will be great and will be called the Son of the Most High. The Lord God will give him the throne of his father David,
33 and he will reign over Jacob’s descendants forever; his kingdom will never end.
34 How will this be, Mary asked the angel, since I am a virgin?
35 The angel answered, The Holy Spirit will come on you, and the power of the Most High will overshadow you. So the holy one to be born will be called the Son of God.
36 Even Elizabeth your relative is going to have a child in her old age, and she who was said to be unable to conceive is in her sixth month.
37 For no word from God will ever fail.
38 I am the Lord’s servant, Mary answered. May your word to me be fulfilled. Then the angel left her.

You can also read the feast troparion and some more information on this great Christian feast here

This year, today's feast also coincides with the comemmoration of one great Church Father – Saint John (monk) Lestvichnik .

st. john Lestvichnik Orthodox Christian icon

St. John Lestvichnik is honoured as one of our great ascetic saints across all Orthodox Christian dome.
Actually Lestvichnik word (comes from ancient Bulgarian / ancient Slavonic) and is a title. st. John received the Lestvichnik title as his spiritual life was so high and up to God, like climbing from earth to Gods heavens.

The meaning of Lestvichnik is "of the Ladder". After his title name Lestvichnik was named also his greatest life work Lestvica, which translated to english means Ladder

To read more about st. John Lestvichnik living a good Egnlish translation is here

Just in case to preserve, this great saints short living (english translation), I've made a mirror of st. John Lestvichnik (short) living here

St.John Lestvichnik (of the Ladder) 12th, 13 century icon

Lestvica is a very popular ancient monks living rule (guidance text);
"a true instruction for those following after invariably, and as such would be a ladder of affirmation, which would lead those wishing it to the Heavenly gates…"

Lestvica was written by st. John Lestvichnik, after a 40 years ascetic seclusion in (speachless solitude
The monks spiritual instructions rules and advices work "The Ladder" bears the title Ladder described by st. John like so:

"I have constructed a ladder of ascent...
from the earthly to the holy...
in the form of the thirty years of age for the Lord's maturity, symbolically I have constructed a ladder of 30 steps, by which, having attained the Lord's age, we find ourselves with the righteous and secure from falling down".
The purpose of this work, is to teach – that the reaching of salvation requires difficult self-denial and demanding ascetic deeds. "The Ladder" presupposes, first;
1. a cleansing from the impurity of sin – the eradication of vices and passions in the old man;

2. the restoration in man of the image of God. – Although the book was written for monks, any christian living in the world can receive from it the hope of guidance for ascent to God as well as get a support for better spiritual life.
The Monks Theodore the Studite(Comm. 11 November and 26 January), Sergei of Radonezh (Comm. 25 September and 5 July), Joseph of Volokolamsk (Comm. 9 September and 18 October), and others – in their instructions relied on "The Ladder" as an important book for salvific guidance.

Auto restart Apache on High server load (bash shell script) – Fixing Apache server temporal overload issues

Saturday, March 24th, 2012


I've written a tiny script to check and restart, Apache if the server encounters, extremely high load avarage like for instance more than (>25). Below is an example of a server reaching a very high load avarage:;

server~:# uptime
13:46:59 up 2 days, 18:54, 1 user, load average: 58.09, 59.08, 60.05
load average: 0.09, 0.08, 0.08

Sometimes high load avarage is not a problem, as the server might have a very powerful hardware. A high load numbers is not always an indicator for a serious problems. Some 16 CPU dual core (2.18 Ghz) machine with 16GB of ram could probably work normally with a high load avarage like in the example. Anyhow as most servers are not so powerful having such a high load avarage, makes the machine hardly do its job routine.

In my specific, case one of our Debian Linux servers is periodically reaching to a very high load level numbers. When this happens the Apache webserver is often incapable to serve its incoming requests and starts lagging for clients. The only work-around is to stop the Apache server for a couple of seconds (10 or 20 seconds) and then start it again once the load avarage has dropped to less than "3".

If this temporary fix is not applied on time, the server load gets increased exponentially until all the server services (ssh, ftp … whatever) stop responding normally to requests and the server completely hangs …

Often this server overloads, are occuring at night time so I'm not logged in on the server and one such unexpected overload makes the server unreachable for hours.
To get around the sudden high periodic load avarage server increase, I've written a tiny bash script to monitor, the server load avarage and initiate an Apache server stop and start with a few seconds delay in between.

# script to check server for extremely high load and restart Apache if the condition is matched
check=`cat /proc/loadavg | sed 's/\./ /' | awk '{print $1}'`
# define max load avarage when script is triggered
# log file
# location of inidex.php to overwrite with temporary message
# location to Apache init script
site_maintenance_msg="Site Maintenance in progress - We will be back online in a minute";
if [ $check -gt "$max_load" ]; then>
#25 is load average on 5 minutes
cp -rpf $index_php_loc $index_php_loc.bak_ap
echo "$site_maintenance_msg" > $index_php_loc
sleep 15;
if [ $check -gt "$max_load" ]; then
$apache_init stop
sleep 5;
$apache_init restart
echo "$(date) : Apache Restart due to excessive load | $check |" >> $high_load_log;
cp -rpf $index_php_loc.bak_ap $index_php_loc

The idea of the script is partially based on a forum thread – Auto Restart Apache on High Load is a link to my script

The script is written in a way that it makes two "if" condition check ups, to assure 100% there is a constant high load avarage and not just a temporal 5 seconds load avarage jump. Once the first if is matched, the script first tries to reduce the server load by overwritting a the index.php, index.html script of the website with a one stating the server is ongoing a maintenance operations.
Temporary stopping the index page, often reduces the load in 10 seconds of time, so the second if case is not necessery at all. Sometimes, however this first "if" condition cannot decrease enough the load and the server load continues to stay too high, then the script second if comes to play and makes apache to be completely stopped via Apache init script do 2 secs delay and launch the apache server again.

The script also logs about, the load avarage encountered, while the server was overloaded and Apache webserver was restarted, so later I can check what time the server overload occured.
To make the script periodically run, I've scheduled the script to launch every 5 minutes as a cron job with the following cron:

# restart Apache if load is higher than 25
*/5 * * * * /usr/sbin/ >/dev/null 2>&1

I have also another system which is running FreeBSD 7_2, which is having the same overload server problems as with the Linux host.
Copying the auto restart apache on high load script on FreeBSD didn't work out of the box. So I rewrote a little chunk of the script to make it running on the FreeBSD host. Hence, if you would like to auto restart Apache or any other service on FreeBSD server get /usr/sbin/ my script and set it on cron on your BSD.

This script is just a temporary work around, however as its obvious that the frequency of the high overload will be rising with time and we will need to buy new server hardware to solve permanently the issues, anyways, until this happens the script does a great job 🙂

I'm aware there is also alternative way to auto restart Apache webserver on high server loads through using monit utility for monitoring services on a Unix system. However as I didn't wanted to bother to run extra services in the background I decided to rather use the up presented script.

Interesting info to know is Apache module mod_overload exists – which can be used for checking load average. Using this module once load avarage is over a certain number apache can stop in its preforked processes current serving request, I've never tested it myself so I don't know how usable it is. As of time of writting it is in early stage version 0.2.2
If someone, have tried it and is happy with it on a busy hosting servers, please share with me if it is stable enough?

How to digital watermark to a picture – Protect pic with copyright image or text with composite, convert and Phatch on GNU / Linux

Friday, March 23rd, 2012

Watermarking is a technique to identify a physical or non-physical object with its owner (creator). First watermarks in history originates from very ancient times.

There are two basic types of Watermark types:

I. Physical Watermarks (classical)

II. Digital Watermarks

Historically Classical Watermarks, were mostly important. As we tend to use more and more visible and we switch to use of more invisible stuff, nowdays the importance and use of digital watermarks is steadily raising.

You have most likely already seen pictures from websites which contain a copyright holder message stamp or website logo on it.
As you could imagine the picture watermark is placed in order to prevent pictures from being re-used in another internet space ,without the picture copyright holder explicit permission…

Watermarks have entered most if not all areas of our life, but we often don't recognize they're there / rarely think about them.
Few of the many "physical watermarks" we use daily are:

  • paper money watermark (to protect against anti money forgery)
  • bank debit / credit cards stamps near the card chip
  • postcards paper stamps

There are too many different kind of "physical watermarks" and since this is not the accent of this article, I will continue straight to explaiin a bit on Digital (picture) watermarks and how to watermark images with ImageMagick image editting command line suit.

Just like with physical watermarks, there are different kinds of digital watermarks. There are:

  • Picture (Images) digital watermarks
  • – Steganography

  • Video watermarks
  • Audio stream digital watermarks
  • Visual digital watermarks
  • – Visible area of text or picture over another text picture or video

  • Invisible digital watermarks
  • – digital information (files) metadata with watermark content etc.

The topic of watermarking is quite wide, so I will stop here and focus on the main idea of this article – to show how to place digital watermark on graphic image or collection of pictures.

The most straightway non-interactive way to do picture watermarking is with ImageMagick's composite command line tool. This little handy tool is capable of creating watermarks in single and multiple pictures.

If you prefer to have a simple text as a watermark, then you should use imagick's convert cmd.

1. Putting a watermark of picture in the right bottom corner

$ composite -gravity southeast -dissolve 100 \
watermark_picture.png image-to-watermark.png \

Snoopy Writting pc freak watermark picture text watermark on the right bottom corner with composite

2. Placing watermark to picture in the bottom right corner

$ composite -gravity northeast -dissolve 80 \
watermark_picture.png image-to-watermark.png \

Snoopy Writting pc freak picture text watermark on the right bottom corner with composite

3. Watermarking picture in the bottom left corner

$ composite -gravity southwest -dissolve 90 \
watermark_picture.png image-to-watermark.png \

Snoopy writting watermarking picture on the bottom left corner imagemagick (composite)

4. Watermarking picture in the top left corner

$ composite -gravity northwest -dissolve 100 \
watermark_picture.png image-to-watermark.jpg \

As you see from above example, composite even accept mixing up input / output between PNG and JPEG pictures 🙂

Output Watermarked Image picture on top left corner with pc-freak logo image Imagick composite

5. Put a watermark in the image center

$ composite -gravity center -dissolve 100 \
watermark_picture.png image-to-watermark.png \

position watermark on the picture middle (center) composite output picture

6. Sealing image with custom text / Text Watermarking a picture

a) Writting text watermark to an image centered in "footer"

$ convert image-to-watermark.png -pointsize 20 \
-draw "gravity south fill black text 0,12 \
'hip0s Watermark'" output-watermarked-image.jpg

This will place a watermark in position 0,12, meaning the text will be added in the bottom center of the watermarked image.

Watermarking a picture sealing with custom text image imagick composite pic

-pointsize 20 defines the text font size. hip0s Watermark is the actual text that will be stamped.

b) Writting image watermark with font type customization (Arial Tahoma etc.):

To list all available fonts ready to be used by convert, type:

$ convert -list font
$ convert -list font |grep -i arial
Font: Arial-Black-Regular
family: Arial Black
glyphs: /usr/share/fonts/truetype/msttcorefonts/Arial_Black.ttf
Font: Arial-Bold
family: Arial
glyphs: /usr/share/fonts/truetype/msttcorefonts/Arial_Bold.ttf
Font: Arial-Bold-Italic
family: Arial
glyphs: /usr/share/fonts/truetype/msttcorefonts/arialbi.ttf
Font: Arial-Italic
family: Arial
glyphs: /usr/share/fonts/truetype/msttcorefonts/ariali.ttf
Font: Arial-Regular
family: Arial
glyphs: /usr/share/fonts/truetype/msttcorefonts/Arial.ttf

$ convert -list type

On my system, I have 392 of fonts installed, to check the number of installed fonts ready for use by convert I used:

$ convert -list font|grep -i 'font:'|wc -l

To only check exact fonts names usable in convert:

$ convert -list font|grep -i 'font:'

To use the red marked Arial-Regular for font of the text picture timestamp issue;

$ convert watermark_picture.jpg -font Arial-Regular \
-pointsize 20 -draw "gravity south fill black text 0,12 'hip0s Watermark'" \

Watermark and with Arial-Regular font image magick convert screenshot dog type writter

c) Using external font with convert to place image text watermark

Lets say you would like to use an external font (arhangai.ttf) not listed in convert -font list usable fonts:

$ convert image-to-watermark.png -pointsize 20 \
-font /usr/share/fonts/truetype/arhangai/arhangai.ttf \
-draw "gravity south fill black text 0,12 \
'hip0s Watermark'" output-watermarked-image_7.png

Talking about fonts, if you would like to add some external, nice free-fonts (ttf) files to your current logged in user, exec:

hipo@noah:~$ cd ~/fonts
hipo@noah:/fonts$ for i in \
$(lynx -dump|grep -i .ttf|grep -i http|awk '{ print $2 }'); \
do wget -r -l2 -nd -Nc $i;

This will add 85 new nice looking fonts. Putting fonts in .fonts/ directory, are red while fonts are looked up by applications installed on respective the server or desktop GNU / Linux systems. Any font put there is ready to be used across all ImageMagick command line tools, as well as will be added across the list of possible fonts to use in GIMP and the rest of gui editors installed on the system.

According to the (watermark) texts font size passed to convert on some pictures the text written will exceed the picture dimensions and only partially some of the text intended as watermark will be visible.
If you encounter the exceed picture text problem, take few minutes and play with fonts sizes until you have a good font size to fit the approximate dimensions of the (expected minimum / maximum – horizontal and vertical) stamped picture dimensions.

For the sake of clarity, here is a list with arguments used in above, composite and convert examples.

  • composite — The ImageMagick command that combines two images.
  • -dissolve 80 — The number after the option determines the brightness of the watermark.  100 is full strength.
  • -gravity southeast — Determines the placement of watermark.
    Possible options are; north, west, south, east, northwest, northeast, southeast, southwest, center
  • watermark_picture.png — The watermark image is the first argument.
  • image-to-watermark.jpg — The second argument is the original image to be watermarked.
  • output-watermarked-image.jpg — The third argument is the new composite image to be created.
    N. B. !  If you don't specify a new file, be careful, the original file will be overwritten.

As ImageMagick is cross platform graphic editting suit – it runs on both *nix (Linux,BSD) and Windows. I have tested it on Linux, only but on FreeBSD and other BSDs it should work without any problem.
The composite and convert above examples should be easily rewritten to run on achieve watarmarking on MS Windows too.

7. Watermarking multiple pictures in a directory

To watermark multiple pictures within a directory use, a short bash loop in combination with either convert or composite could be used:

$ cd your-directory/
$ for i in *; do
convert $i -pointsize 20 -draw "gravity south fill black text 0,12 'hip0s Watermark'" output-watermarked-image.jpg

convert and composite also support wildcards like '*.JPG, *.PNG', but I'm not sure if this syntax can be used for mass picture marking?

8. Adding watermark and doing other various advanced Image Edit, Convert and Compose stuff with Phatch GUI program

Another program that is capable to put watermarks on pictures and besides that doing a number of routine graphic manipulation operations achievable with expert Image manipulation programs like GIMP / Inkscape is PHATCH = PHOTO & BATCH

Phatch is swiss army knife for doing web design or or graphics design on Linux.

Phatch is really great and easy to use program. Tt makes putting basic designer effects on pictures with no requirement for any design skills.
With Phatch you can become a designer for a day literally 😉

If you haven't used it yet, make sure you try it!
Below, are two screenshots of Phatch running on my Debian G* / Linux

Phatch Linux Debian Squeeze Screenshot

Phatch Linux Debian Squeeze Screenshot Watermark effect

Phatch is installable via apt on Debian and Ubuntu Linux. It has also a phatch-cli tools, which are a possible substitute to ImageMagick's composite / convert tools.

On deb based distros install Phatch with:

noah:~# apt-get --yes install phatch phatch-cli

In Phatch it is also possible, to create a combination of filters to be later applied to an image file or a group of image files all in a directory. The program capabilities are really outstanding, it is pure joy to work with it.

Using Phatch GUI interface is hard to comprehend in the beginning, I needed few minutes until I can get the idea how to use it. Anyhow once you know the basics, its very easy to use onwards.

Phatch currently can perform the following actions:

  • Auto Contrast – Maximize image contrast
  • Border – Crop or add border to all sides
  • Brightness – Adjust brightness from black to white
  • Canvas – Crop the image or enlarge canvas without resizing the image
  • Colorize – Colorize grayscale image
  • Common – Copies the most common pixel value
  • Contrast – Adjust from grey to black & white
  • Convert Mode – Convert the color mode of an image (grayscale, RGB, RGBA or CMYK)
  • Effect – Blur, Sharpen, Emboss, Smooth, ..
  • Equalize – Equalize the image histogram
  • Fit – Downsize and crop image with fixed ratio
  • Grayscale – Fade all colours to gray
  • Invert – Invert the colors of the image (negative)
  • Maximum – Copies the maximum pixel value
  • Median – Copies the median pixel value
  • Minimum – Copies the minimum pixel value
  • Offset – Offset by distance and wrap around
  • Posterize – Reduce the number of bits of colour channel
  • Rank – Copies the rank'th pixel value
  • Rotate – Rotate with random angle
  • Round – Round or crossed corners with variable radius and corners
  • Saturation – Adjust saturation from grayscale to high
  • Save – Save an image with variable compression in different types
  • Scale – Scale an image with different resample filters.
  • Shadow – Drop a blurred shadow under a photo with variable position, blur and color
  • Solarize – Invert all pixel values above threshold
  • Text – Write text at a given position
  • Transpose – Flip or rotate an image by 90 degrees
  • Watermark – Apply a watermark image with variable placement (offset, scaling, tiling) and opacity

Most of the function / effects Phatch in the up list works fine as I tested them to get to know the program.
The only effect that didn't worked for me is Blender effect.
Trying to apply the Blending effect I got error:

Can not apply action Blender:
'dict' object has no attribute 'rfind'

dict object has not attribiture rfind error screenshot my linux

Its really a pity blender filter don't work. I've seen on Phatch's website some pictures showing the blender effect in action and it looks really awesome.

In attempt to work around the err, I tried downloading Phatch's latest release and running it with python interpreter but it didn't work out …
I tried also to install some packages to the system that somehow seemed to be related to blenderversatile 3D modeller/renderer program but this worked neither.
I suspect Phatch blender effect is not working on Ubuntu too as I've red complains in some Ubuntu forums.
If someone succeeding making blend effect work please let me know how?

Interesting feature of Phatch is the program support for applying its predefined filters using a cli interface.
The syntax for phatch cli, should be something like:

phatch -console action_list.phatch

Where action_list.phatch is a Phatch predefined filter. Anyways I didn't manage to figure out how to use the program CLI. I'll be glad to hear if someone succeeded in using the program console, if so please share with me how?

9. Adding Watermark to pictures with GIMP

To add a watermark text or picture in GIMP, there are plenty of ways but is more time consuming by both Phatch or convert, composite..
There is a script in gimp plugin registry site – watermark.scm which adds watermarking capability to GIMP

On my system this script was installed with the deb package gimp-data-extras. To apply the plugin on a pic, I used GIMP menus:

Filters -> Eg -> Copyright Placer

GIMP Copyright Holder plugin Watermark Screenshot

If someone knows about better or quicker ways to do watermarking, please share 🙂