Posts Tagged ‘linux distributions’

How to take multiple screenshots with scrot and ImageMagick import commands in terminal on GNU / Linux and FreeBSD

Friday, January 13th, 2012

Reading Time: 3minutes

scrot and import are two commands, which can be used to take screenshot in terminal on Linux and FreeBSD:

To use scrot cmd to take screenshots on Ubuntu and Debian the scrot package has to be installed:

noah:~# apt-get install scrot
...

scrot should also be available on most other Linux distributions in the main repositories, I'll be glad to hear if someone has used it on Fedora, SUSE etc.

On FreeBSD, there is a port called scrot , to install on FreeBSD:

freebsd# cd /usr/ports/graphics/scrot
freebsd# make install clean
...

Scrot has plenty of nice arguments one can use to make a screenshot. Maybe the most handy one in my view is after a preliminary set delay before screenshot is taken.

To take screenshot with it after lets say 5 seconds delay before the screenshot:

hipo@noah:~/Desktop$ scrot -t 20 -d 5

Screenshot scrot my debian Linux gnome-termina

To put an year, month and day and year followed by screen resolution with scrot :

hipo@noah:~$ scrot '%Y-%m-%d_$wx$h.png'

Another way to take a screenshot of screen with command is by using ImageMagick'simport image manipulation package.
To take screenshot of the current screen via terminal using import , type in xterm, gnome-termina or Gnome's Run Application (ALT+F2)

hipo@noah:~$ import -window root ScreenShot.png

To make import command to save the taken screenshot in a format (minute:hour:day:month:year)i :

hipo@noah:~$ import -window root $screenshot_dir/screenshot-$(date +%M_%k_%d_%m_%Y|sed -e 's/^ *//').png

Taking a delayed screenshot is also possible via The GIMP via menus File -> Create -> Screenshot

GIMP Screenshot 15 seconds delay GIMP window screenshot

Now here is an interesting question, what if I would like to take periodic screenshots of what I do on my Desktop to take random movie scenes from a movie I watch with totem or vlc??

This task is quite easily achiavable with a little bash shell script, I wrote:

screenshot_dir='Screenshots';
seconds='60';
if [ ! -d "$screenshot_dir" ]; then
mkdir $screenshot_dir;
fi
while [ 1 ]; do
sleep $seconds;
(import -window root $screenshot_dir/screenshot-$(date +%M_%k_%d_%m_%Y|sed -e 's/^ *//').png) &
done

This script will take screenshot automatically to Screenshots/ directory every (1 min – 60 seconds)
You can also my downloads take_screenshot_every_60_secs_import.sh here

To use take_screenshot_every_60_secs_import.sh just issue the script inside xterm or gnome-terminal, after that simply use your computer as you normally would.
The script will take snapshots every minute and store all taken screenshots in Screenshots dir.

If you prefer to use scrot to take automatically the screenshots every lets say 5 minutes, you can use a script like:

screenshot_dir='Screenshots';
# 300 secs (5 mins)seconds='300';
if [ ! -d "$screenshot_dir" ]; then
mkdir $screenshot_dir;
fi
while [ 1 ]; do
sleep $seconds;
(scrot $screenshot_dir/'%Y-%m-%d_$wx$h.png') &
done

You can fetch take_screenshot_every_60_secs_scrot.sh here

The script using scrot is better in terms of efficiency, the system load scrot will put on your machine will be less.
Using some of this scripts will be handy if you need screenshots to Movies, Programs and favourite Free Software games.
Hope this is educative to someone 😉

Text Monitoring of connection server (traffic RX / TX) business in ASCII graphs with speedometer / Easy Monitor network traffic performance

Friday, May 4th, 2012

Reading Time: 7minutes

While reading some posts online related to MS-Windows TcpViewnetwork traffic analyzing tool. I've came across very nice tool for tracking connection speed for Linux (Speedometer). If I have to compare it, speedometer is somehow similar to nethogs and iftop bandwidth network measuring utilities .

What differentiates speedometer from iftop / nethogs / iptraf is it is more suitable for visualizing a network file or data transfers.
The graphs speedometer draws are way easier to understand, than iftop graphs.

Even complete newbies can understand it with no need for extraordinary knowledge in networking. This makes Speedometer, a top tool to visually see the amount of traffic flowing through server network interface (eth0) … (eth1) etc.

What speedometer shows is similar to the Midnight Commander's (mc) file transfer status bar, except the statistics are not only for a certain file transfer but can show overall statistics over server passing network traffic amount (though according to its manual it can be used to also track individual file transfers).

The simplicity for basic use makes speedometer nice tool to track for network congestion issues on Linux. Therefore it is a  must have outfit for every server admin. Below you see a screenshot of my terminal running speedometer on a remote server.

Speedometer ascii traffic track server network business screenshot in byobu screen like virtual terminal emulator

1. Installing speedometer on Debian / Ubuntu and Debian derivatives

For Debian and Ubuntu server administrators speedometer is already packaged as a deb so its installation is as simple as:

debian:~# apt-get --yes install speedometer
....

2. Installing speedometer from source for other Linux distributions CentOS, Fedora, SuSE etc.

Speedometer is written in python programming language, so in order to install and use on other OS Linux platforms, it is necessery to have installed (preferably) an up2date python programming language interpreter (python ver. 2.6 or higher)..
Besides that it is necessary to have installed the urwid -( console user interface library for Python) available for download via excess.org/urwid/

 

Hence to install speedometer on RedHat based Linux distributions one has to follow these steps:

a) Download & Install python urwid library

[root@centos ~]# cd /usr/local/src
[root@centos src]# wget -q http://excess.org/urwid/urwid-1.0.1.tar.gz
[root@centos src]# tar -zxvvf urwid-1.0.1.tar.gz
....
[root@centos src]# cd urwid-1.0.1
[root@centos urwid-1.0.1]# python setup.py install
running install
running build
running build_py
creating build
creating build/lib.linux-i686-2.4
creating build/lib.linux-i686-2.4/urwid
copying urwid/tests.py -> build/lib.linux-i686-2.4/urwid
copying urwid/command_map.py -> build/lib.linux-i686-2.4/urwid
copying urwid/graphics.py -> build/lib.linux-i686-2.4/urwid
copying urwid/vterm_test.py -> build/lib.linux-i686-2.4/urwid
copying urwid/curses_display.py -> build/lib.linux-i686-2.4/urwid
copying urwid/display_common.py -> build/lib.linux-i686-2.4/urwid
....

b) Download and install python-setuptools

python-setuptools is one other requirement of speedometer, happily on CentOS and Fedora the rpm package is already there and installable with yum:

[root@centos ~]# yum -y install python-setuptools
....

c) Download and install Speedometer

[root@centos urwid-1.0.1]# cd /usr/local/src/
[root@centos src]# wget -q http://excess.org/speedometer/speedometer-2.8.tar.gz
[root@centos src]# tar -zxvvf speedometer-2.8.tar.gz
.....
[root@centos src]# cd speedometer-2.8
[root@centos speedometer-2.8]# python setup.py install
Traceback (most recent call last):
File "setup.py", line 26, in ?
import speedometer
File "/usr/local/src/speedometer-2.8/speedometer.py", line 112
n = n * granularity + (granularity if r else 0)
^

While running the CentOS 5.6 installation of speedometer-2.8, I hit the
"n = n * granularity + (granularity if r else 0)
error.

After consultation with some people in #python (irc.freenode.net), I've figured out this error is caused due the outdated version of python interpreter installed by default on CentOS Linux 5.6. On CentOS 5.6 the python version is:

[root@centos ~]# python -V
Python 2.4.3

As I priorly said speedometer 2.8's minimum requirement for a python to be at v. 2.6. Happily there is quick way to update python 2.4 to python 2.6 on CentOS 5.6, as there is an RPM repository maintained by Chris Lea which contains RPM binary of python 2.6.

To update python 2.4 to python 2.6:

[root@centos speedometer-2.8]# rpm -Uvh http://yum.chrislea.com/centos/5/i386/chl-release-5-3.noarch.rpm[root@centos speedometer-2.8]# rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CHL[root@centos speedometer-2.8]# yum install python26

Now the newly installed python 2.6 is executable under the binary name python26, hence to install speedometer:

[root@centos speedometer-2.8]# python26 setup.py install
[root@centos speedometer-2.8]# chown root:root /usr/local/bin/speedometer
[root@centos speedometer-2.8]# chmod +x /usr/local/bin/speedometer

[root@centos speedometer-2.8]# python26 speedometer -i 1 -tx eth0

The -i will instruct speedometer to refresh the screen graphs once a second.

3. Using speedometer to keep an eye on send / received traffic network congestion

To observe, the amount of only sent traffic via a network interface eth0 with speedometer use:

debian:~# speedometer -tx eth0

To only keep an eye on received traffic through eth0 use:

debian:~# speedometer -rx eth0

To watch over both TX and RX (Transmitted and Received) network traffic:

debian:~# speedometer -tx eth0 -rx eth0

If you want to watch in separate windows TX and RX traffic while  running speedometer you can run in separate xterm windows speedometer -tx eth0 and speedometer -rx eth0, like in below screenshot:

Monitor Received and Transmitted server Network traffic in two separate xterm windows with speedometer ascii graphs

4. Using speedometer to test network maximum possible transfer speed between server (host A) and server (host B)

The speedometer manual suggests few examples one of which is:

How fast is this LAN?

host-a$ cat /dev/zero | nc -l -p 12345
host-b$ nc host-a 12345 > /dev/null
host-b$ speedometer -rx eth0

When I red this example in speedometer's manual, it wasn't completely clear to me what the author really meant, but a bit after when I thought over the example I got his point.

The idea behind this example is that a constant stream of zeros taken from /dev/zero will be streamed over via a pipe (|) to nc which will bind a port number 12345, anyone connecting from another host machine, lets say a server with host host-b to port 12345on machine host-a will start receiving the /dev/zero streamed content.

Then to finally measure the streamed traffic betweenhost-a and host-b machines a speedometer is started to visualize the received traffic on network interfaceeth0, thus measuring the amount of traffic flowing from host-a to host-b

I give a try to the exmpls, using for 2 test nodes my home Desktop PC, Linux running  arcane version of Ubuntu and my Debian Linux notebook.

First on the Ubuntu PC I issued
 

hipo@hip0-desktop:~$ cat /dev/zero | nc -l -p 12345
 

Note that I have previously had installed the netcat, as nc is not installed by default on Ubuntu and Debian. If you, don't have nc installed yet, install it with:

apt-get –yes install netcat

"cat /dev/zero | nc -l -p 12345" will not produce any output, but will display just a blank line.

Then on my notebook I ran the second command example, given in the speedometer manual:
 

hipo@noah:~$ nc 192.168.0.2 12345 > /dev/null

Here the 192.168.0.2 is actually the local network IP address of my Desktop PC. My Desktop PC is connected via a normal 100Mbit switch to my routing machine and receives its internet via  NAT. The second test machine (my laptop), gets its internet through a WI-FI connection received by a Wireless Router connected via a UTP cable to the same switch to which my Desktop PC is connected.

Finally to test / get my network maximum thoroughput I had to use:

hipo@noah:~$ speedometer -rx wlan0

Here, I  monitor my wlan0 interface, as this is my (laptop) wireless card interface over which I have connectivity to my local network and via which through the the WI-FI router I get connected to the internet.

Below is a snapshot captured showing approximately what is the max network thoroughput from:

Desktop PC -> to my Thinkpad R61 laptop

Using Speedometer to test network thorougput between two network server hosts screenshot Debian Squeeze Linux

As you can see in the shot approximately the maximum network thoroughput is in between:
2.55MB/s min and 2.59MB/S max, the speed is quite low for a 100 MBit local network, but this is normal as most laptop wireless adapters hardly transfer traffic in more than 10 to 20 MBits per sec.

If the same nework thoroughput test is conducted between two machines both connected to a same 100 M/bit switch, the traffic should be at least a 8 MB/sec.

There is something, else to take in consideration that probably makes the provided example network thoroughput measuring a bit inaccurate. The fact that the /dev/zero content is stremed over is slowing down the zeroes sent over network because of the  pipe ( | ) use slows down the stream.

5. Using speedometer to visualize maximum writting speed to a local hard drive on Linux

In the speedometer manual, I've noticed another interesting application of this nifty tool.

speedometer can be used to track and visualize the maximum writing speed a hard disk drive or hard drive partition can support on Linux OS:

A copy paster from the manual text is as follows:

How fast can I write data to my filesystem? (with at least 1GB free)
dd bs=1000000 count=1000 if=/dev/zero of=bigfile &
speedometer bigfile

However, when I tried copy/pasting the example in terminal, to test the maximum writing speed to an external USB hard drive, only dd command was started and speedometer failed to initialize and display graphs of the file creation speed.

I've found a little "hack" that makes the man example work by adding a 3 secs sleep like so:

debian:/media/Expansion Drive# dd bs=1000000 count=1000 if=/dev/zero of=bigfile & sleep 3; speedometer bigfile

Here is a screenshot of the bigfile created by dd and tracked "in real time" by speedometer:

How fast is writting data to local USB expandable hard disk Debian Linux speedometer screenshot

Actually the returned results from this external USB drive are, quite high, the possible reason for that is it is connected to my laptop over an USB protocol verion 3.

6. Using Speedometer to keep an eye on file download in progress

This application of speedometer is mostly useless especially on Linux where it is used as a Desktop.

However in some occasions if files are transferred over ssh or in non interactive FTP / Samba file transfers between Linux servers it can come handy.

To visualize the download and writing speed of lets say FTP transferred .AVI movie (during the actual file transfer) on the download host issue:

# speedometer Download-Folder/What-goes-around-comes-around.avi

7. Estimating approximate time for file transfer

There is another section in the speedometer manual pointing of the program use to calculate the time remaining for a file transfer.

The (man speedometer) provided example text is:

How long it will take for my 38MB transfer to finish?
speedometer favorite_episode.rm $((38*1024*1024))

At first glimpse it hard to understand (like the other manual example). A bit of reasoning and I comprehend what the man author meant by the obscure calculation:

$((38*1024*1024))

This is a formula used in which 38 has to be substituted with the exact file size amount of the transferred file. The author manual used a 38MB file so this is why he put $((38* … in the formula.

I give it a try – (just for the sake to see how it works) with a file with a size of 2500MB, in below two screenshot pictures I show my preparation to copy the file and the actual copying / "real time" transfer tracking with speedometer's status percentage completion bar.

xterm terminal copy file and estimate file copying operation speed on linux with speedometer preparation

Two xterm terminals one is copying a file the other one uses speedometer to estimate the time remaining to complete the file transfer from expansion USB hard drive to my laptop harddrive

 

Linux: how to Start multiple X sessions – Connect to remote Linux GNOME with no need for VNC by exporting display

Thursday, October 10th, 2013

Reading Time: 2minutes

start multiple X server Xorg sessions export graphic X display to use Linux gui from another Linux like dumb terminal
It is useful sometimes in Linux to run multiple Xservers and from there to start few Window Managers (lets say one with Window Maker and one with GNOME and FluxBox). Running second / 3rd etc. X session is nice especially when you you'd like to access remotely your Desktop (lets say from another Linux).

To start second Xsession with only terminal from which you can invoke any GUI environment use:

xinit -- :1

For third one do

xinit -- :2

 

First Xsession is working on screen :0 (e.g. xinit — :0). To access and navigate later via various X sessions depending on the Linux distribution and how it is configured to which console to start new sessions use

ALT + F5, ALT + F6, ALT + F7.

On GNU / Linux distributions where default Xorg server is running on TTY7 to switch to 2nd and 3rd Window Manager use instead:

ALT + F7, ALT + F8, ALT + F9

Alternative command to issue to launch multiple sessions with lets say GNOME (if that's default set GUI environment) use:
 

startx -- :1

and

startx -- :2

Whether you want to launch GUI environment from another Linux after connecting through SSH or telnet term client (i.e. you have old machine hardware with Linux with no graphical environment and would like to use 2nd machine with decent hardware and Xorg + GNOME running fine), way to do is via:

xhost +

and exporting DISPLAY to remote IP host.

Here is example how to launch second Xsession with GUI environment from remote Linux host.
For example we assume host which will host 2nd X session is 192.168.1.1 and host from which remote Xorg with GNOME will be accessed is 192.168.1.2.

a. Use ssh from 192.168.1.2 to 192.168.1.1

# ssh user@192.168.1.1
xorg-machine:~# xhost +

Above command allows all hosts to be able to connect to 192.168.1.1.

To enable just single host to be able to connect to Xorg server on 192.168.1.1:

xorg-machine:~# xhost +192.168.1.1

b. On xorg-machine (192.168.1.1) its necessary export display to 192.168.1.2

xorg-machine:~# export DISPLAY=192.168.1.2:0.0

To make Xorg + default GUI window manager popup then on 192.168.1.2

xorg-machine:~# startx

Client Denied By Server Configuration Linux Apache error solution

Thursday, August 1st, 2013

Reading Time: 2minutes

Client denied by server configuration fix solution Apache feather logo

If you run Apache server on Debian Linux / Ubuntu / CentOS whatever Linux OS and you try to install a new PHP application under lets say /var/www/ getting an error in Apache error.log like:
 

[Wed Jul 31 03:36:21 2013] [error] [client 192.168.10.2] client denied by server configuration: /var/www/vea/index.php, referer: http://192.168.10.9/vea/


This is due to misconfigured AllowOverrides in some of your main configuration files.

So what is causing the error?

Reason is by default in most current Linux distributions Apache is configured to have restrictive policy following the good security practice (Restrictive by default).
Apache is configured by default to not accept AllowOverrides – i.e. AllowOverride None for DocumentRoot /, because there are plenty of administrators who run Apache without having profound understanding leaving it to interpret by default mod_rewrite rules from .htaccess files etc.

To fix this issue, hence you have to add extra configuration for AllowOverride directive for directory giving the err. In this case /vea:

<Directory /var/www/vea/> 
Options -Indexes FollowSymLinks
AllowOverride AuthConfig
FileInfo Order allow,deny
Allow from all
</Directory> 

Above rules are a bit restrictive and will allow only to have .htaccess with only for protecting directory with htaccess passsword for exmpl. – (AuthUserFile, AuthGroupFile, AuthName, AuthType) .htaccess.
-Indexes – instructs /var/www/vea directory listing to be disabled, below two lines:

Order allow, deny
Allow from all

Makes the directory Allowed to be visible by all, however note that it is possible in some of other Apache configuration files to have other rules configured for /vea documentroot /var/www/ which are preventive (Default Deny) – if this is the case just walk through all Apache configs and change where necessary to Allow from all.

In some cases it is possible Web application or Website placed requires AllowOverride All directive instead. If above <Directory>

does not help and you continue to get:

[Wed Jul 31 03:36:21 2013] [error] [client xxx.xxx.xx.x] client denied by server configuration: /var/www/php-application/index.php, referer: http://xxx.xxx.xx.xx/php-application/  

Try setting Directory rules with AllowOverride All ;

<Directory /var/www/php-application/> 
Options -Indexes FollowSymLinks
AllowOverride All
FileInfo Order allow,deny
Allow from all
</Directory> 

Debian / Ubuntu server admins should check closely for AllowOverride rules in files /etc/apache2/conf.d/*

as well as in /etc/apache2/mods-available/*:

Usually there are AllowOverride rules set from files:

/etc/apache2/conf.d/apache2-doc
/etc/apache2/conf.d/localized-error-pages
/etc/apache2/conf.d/security

and also in

/etc/apache2/mods-available/alias.conf
/etc/apache2/mods-available/userdir.conf

On Debian GNU / Linux, very common reason for gettingclient denied by server configuration is AllowOverride definitions in /etc/apache2/conf.d/security, default AllowOverride there are set to None, i.e.:

<Directory />
AllowOverride None
Order Deny,Allow
Deny from all
</Directory>

If that's the case with you make sure you config rules to become:

# <Directory />
# AllowOverride None
# Order Deny,Allow
# Deny from all
# </Directory>

A very useful command to find where there is occurance of AllowOverride in Apache many configs is:

root@linux:~# cd /etc/apache2
root@linux:/etc/apache2# grep -rli AllowOverride *
apache2.conf
conf.d/localized-error-pages
conf.d/apache2-doc
conf.d/security
mods-available/userdir.conf
mods-available/alias.conf
sites-available/www.website1.com
sites-available/www.website2.com
...


 

Once you did all necessary config Restart Apache:

root@linux:~# /etc/init.d/apache2 restart
....

Tracking I/O hard disk server bottlenecks with iostat on GNU / Linux and FreeBSD

Tuesday, March 27th, 2012

Reading Time: 7minutes

Hard disk overhead tracking on Linux and FreeBSD with iostat

I've earlier wrote an article How to find which processes are causing hard disk i/o overhead on Linux there I explained very rawly few tools which can be used to benchmark hard disk read / write operations. My prior article accent was on iotop and dstat and it just mentioned of iostat. Therefore I've wrote this short article in attempt to explain a bit more thoroughfully on how iostat can be used to track problems with excessive server I/O read/writes.

Here is the command man page description;
iostatReport Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems

I will further proceed with few words on how iostat can be installed on various Linux distros, then point at few most common scenarious of use and a short explanation on the meaning of each of the command outputs.

1. Installing iostat on Linux

iostat is a swiss army knife of finding a server hard disk bottlenecks. Though it is a must have tool in the admin outfut, most of Linux distributions will not have iostat installed by default.
To have it on your server, you will need to install sysstat package:

a) On Debian / Ubuntu and other Debian GNU / Linux derivatives to install sysstat:

debian:~# apt-get --yes install sysstat

b) On Fedora, CentOS, RHEL etc. install is with yum:

[root@centos ~]# yum -y install sysstat

c) On Slackware Linux sysstat package which contains iostat is installed by default. 

d) In FreeBSD, there is no need for installation of any external package as iostat is part of the BSD world (bundle commands).
I should mention bsd iostat and Linux's iostat commands are not the same and hence there use to track down hard disk bottlenecks differs a bit, however the general logic of use is very similar as with most tools in BSD and Linux.

2. Checking a server hard disk for i/o disk bottlenecks on G* / Linux

Once having the sysstat installed on G* / Linux systems, the iostat command will be added in /usr/bin/iostat
a) To check what is the hard disk read writes per second (in megabytes) use:

debian:~# /usr/bin/iostat -m
Linux 2.6.32-5-amd64 (debian) 03/27/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
15.34 0.36 2.76 2.66 0.00 78.88
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 63.89 0.48 8.20 6730223 115541235
sdb 64.12 0.44 8.23 6244683 116039483
md0 2118.70 0.22 8.19 3041643 115528074

In the above output the server, where I issue the command is using sda and sdb configured in software RAID 1 array visible in the output as (md0)

The output of iostat should already be easily to read, for anyone who didn't used the tool here is a few lines explanation of the columns:

The %user 15.34 meaning is that 15.34 out of 100% possible i/o load is generad by system level read/write operations.
%nice – >Show the percentage of CPU utilization that occurred while executing at the user level with nice priority.
%iowait – just like the top command idle it shows the idle time when the system didn't have an outstanding disk I/O requests.
%steal – show percentage in time spent in time wait of CPU or virtual CPUs to service another virtual processor (high numbers of disk is sure sign for i/o problem).
%idle – almost the same as meaning to %iowait
tps – HDD transactions per second
MB_read/s (column) – shows the actual Disk reads in Mbytes at the time of issuing iostat
MB_wrtn/s – displays the writes p/s at the time of iostat invocation
MB_read – shows the hard disk read operations in megabytes, since the server boot 'till moment of invocation of iostat
MB_wrtn – gives the number of Megabytes written on HDD since the last server boot filesystem mount

The reason why the Read / Write values for sda and sdb are similar in this example output is because my disks are configured in software RAID1 (mirror)

The above iostat output reveals in my specific case the server is experiencing mostly Disk writes (observable in the high MB_wrtn/s 8.19md0 in the above sample output).

It also reveals, the I/O reads experienced on that server hard disk are mostly generated as a system (user level load) – see (%user 15.34 and md0 2118.70).

For all those not familiar with system also called user / level load, this is all kind of load which is generated by running programs on the server – (any kind of load not generated by the Linux kernel or loaded kernel modules).

b) To periodically keep an eye on HDD i/o operations with iostat, there are two ways:

– Use watch in conjunction with iostat;

[root@centos ~]# watch "/usr/bin/iostat -m"
Every 2.0s: iostat -m Tue Mar 27 11:00:30 2012
Linux 2.6.32-5-amd64 (centos) 03/27/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
15.34 0.36 2.76 2.66 0.00 78.88
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 63.89 0.48 8.20 6730255 115574152
sdb 64.12 0.44 8.23 6244718 116072400
md0 2118.94 0.22 8.20 3041710 115560990
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 55.00 0.01 25.75 0 51
sdb 52.50 0.00 24.75 0 49
md0 34661.00 0.01 135.38 0 270

Even though watch use and -d might appear like identical, they're not watch does refresh the screen, executing instruction similar to the clear command which clears screen on every 2 seconds, so the output looks like the top command refresh, while passing the -d 2 will output the iostat command output on every 2 secs in a row so all the data is visualized on the screen. Hence -d 2 in cases, where more thorough debug is necessery is better. However for a quick routine view watch + iostat is great too.

c) Outputting extra information for HDD input/output operations;

root@debian:~# iostat -x
Linux 2.6.32-5-amd64 (debian) 03/27/2012 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
15.34 0.36 2.76 2.66 0.00 78.88
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 4.22 2047.33 12.01 51.88 977.44 16785.96 278.03 0.28 4.35 3.87 24.72
sdb 3.80 2047.61 11.97 52.15 906.93 16858.32 277.05 0.03 5.25 3.87 24.84
md0 0.00 0.00 20.72 2098.28 441.75 16784.05 8.13 0.00 0.00 0.00 0.00

This command will output extended useful Hard Disk info like;
r/s – number of read requests issued per second
w/s – number of write requests issued per second
rsec/s – numbers of sector reads per second
b>wsec/s – number of sectors wrote per second
etc. etc.

Most of ppl will never need to use this, but it is good to know it exists.

3. Tracking read / write (i/o) hard disk bottlenecks on FreeBSD

BSD's iostat is a bit different in terms of output and arguments.

a) Here is most basic use:

freebsd# /usr/sbin/iostat
tty ad0 cpu
tin tout KB/t tps MB/s us ni sy in id
1 561 45.18 44 1.95 14 0 5 0 82

b) Periodic watch of hdd i/o operations;

freebsd# iostat -c 10
tty ad0 cpu
tin tout KB/t tps MB/s us ni sy in id
1 562 45.19 44 1.95 14 0 5 0 82
0 307 51.96 113 5.73 44 0 24 0 32
0 234 58.12 98 5.56 16 0 7 0 77
0 43 0.00 0 0.00 1 0 0 0 99
0 485 0.00 0 0.00 2 0 0 0 98
0 43 0.00 0 0.00 0 0 1 0 99
0 43 0.00 0 0.00 0 0 0 0 100
...

As you see in the output, there is information like in the columns tty, tin, tout which is a bit hard to comprehend.
Thanksfully the tool has an option to print out only more essential i/o information:

freebsd# iostat -d -c 10
ad0
KB/t tps MB/s
45.19 44 1.95
58.12 97 5.52
54.81 108 5.78
0.00 0 0.00
0.00 0 0.00
0.00 0 0.00
20.48 25 0.50

The output info is quite self-explanatory.

Displaying a number of iostat values for hard disk reads can be also achieved by omitting -c option with:

freebsd# iostat -d 1 10
...

Tracking a specific hard disk partiotion with iostat is done with:

freebsd# iostat -n /dev/ad0s1a
tty cpu
tin tout us ni sy in id
1 577 14 0 5 0 81
c) Getting Hard disk read/write information with gstat

gstat is a FreeBSD tool to print statistics for GEOM disks. Its default behaviour is to refresh the screen in a similar fashion like top command, so its great for people who would like to periodically check all attached system hard disk and storage devices:

freebsd# gstat
dT: 1.002s w: 1.000s
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
0 10 0 0 0.0 10 260 2.6 15.6| ad0
0 10 0 0 0.0 10 260 2.6 11.4| ad0s1
0 10 0 0 0.0 10 260 2.8 12.5| ad0s1a
0 0 0 0 0.0 0 0 0.0 20.0| ad0s1b
0 0 0 0 0.0 0 0 0.0 0.0| ad0s1c
0 0 0 0 0.0 0 0 0.0 0.0| ad0s1d
0 0 0 0 0.0 0 0 0.0 0.0| ad0s1e
0 0 0 0 0.0 0 0 0.0 0.0| acd0

It even has colors if your tty supports colors 🙂

Another useful tool in debugging the culprit of excessive hdd I/O operations is procstat command:

Here is a sample procstat run to track (httpd) one of my processes imposing i/o hdd load:

freebsd# procstat -f 50404
PID COMM FD T V FLAGS REF OFFSET PRO NAME
50404 httpd cwd v d -------- - - - /
50404 httpd root v d -------- - - - /
50404 httpd 0 v c r------- 56 0 - -
50404 httpd 1 v c -w------ 56 0 - -
50404 httpd 2 v r -wa----- 56 75581 - /var/log/httpd-error.log
50404 httpd 3 s - rw------ 105 0 TCP ::.80 ::.0
50404 httpd 4 p - rw---n-- 56 0 - -
50404 httpd 5 p - rw------ 56 0 - -
50404 httpd 6 v r -wa----- 56 25161132 - /var/log/httpd-access.log
50404 httpd 7 v r rw------ 56 0 - /tmp/apr8QUOUW
50404 httpd 8 v r -w------ 56 0 - /var/run/accept.lock.49588
50404 httpd 9 v r -w------ 1 0 - /var/run/accept.lock.49588
50404 httpd 10 v r -w------ 1 0 - /tmp/apr8QUOUW
50404 httpd 11 ? - -------- 2 0 - -

Btw fstat is sometimes helpful in identifying the number of open files and trying to estimate which ones are putting the hdd load.
Hope this info helps someone. If you know better ways to track hdd excessive loads on Linux / BSD pls share 'em pls.
 

Merging pictures on Linux command shell with ImageMagick merge

Friday, May 17th, 2013

Reading Time: 2minutes

combining-multiple-jpg-png-pictures-imagemagick-magician-logo

It is generally useful to combine multiple pictures into single one. A example case, where merging pictures on Linux is necessary is if you previously used ImageMagick's convert command line tool to convert PDF file (pages) to JPEG / PNG pictures. Unfortunately convertion with convert(as far as I know is only capable of generating multiple picture files instead of one single one), thus you further need montage to merge pages in separate photos to one. In my case I had my Curriculum Vitae in PDF and I needed to have same PDF in single photo for my applications for online Job Employment Belarusian portal site rabota.tut.by.

montage is one of numerous ImageMagick package script (plugins).
On all major Linux distributions (Debian / Ubuntu, Fedora, CentOS, RHEL, SuSE) montage comes installed together with imagemagickdeb / rpm package.

Whether you don't have montage on Debian / Ubuntu and deb derivatives install it via:

linux:~# apt-get install --yes imagemagick
....

On CentOS, Fedora, RHEL, SuSE to install montage:

[root@centos ~]# yum -y install imagemagick
....

To merge two JPEG Photos into single PNG format picture:
linux:~$ montage -geometry +2+2 Picture-1.jpeg Merged-picture.png

Combining more photos, lets say my 8 Pages photos output from previous PDF convert to pictures is done with:

linux:~$ montage -geometry +8+8 CV_Georgi_Georgiev_bg-0.png \ CV_Georgi_Georgiev_bg-1.png \ CV_Georgi_Georgiev_bg-2.png \ CV_Georgi_Georgiev_bg-3.png \ CV_Georgi_Georgiev_bg-4.png \ CV_Georgi_Georgiev_bg-5.png \ CV_Georgi_Georgiev_bg-6.png \ CV_Georgi_Georgiev_bg-7.png \ CV_Georgi_Georgiev_bg.png
montage has plenty of useful other options, to do various photo montages from command line. Other way to merge photos with montage is by using:

linux:~$ montage -mode concatenate -tile 1x input-pic*.jpg out.jpg

Merging photos is also possible by using directly convert.

Combining multiple photos into single JPEG or PNG with Imagick convert is done with:

linux:~$ convert -append input-pic-*.jpg combined-picture.jpg

Other example use of montage is located on ImageMagick's montage's script site here

 

Linux Mint 14 – “Nadia”: how to Display Trash icon on Desktop

Wednesday, March 13th, 2013

Reading Time: < 1minute

Recently Linux Mint is taking lead among preferred Linux distributions. From my little experience with it mainly installing it on friends PCs I should say Mint develops done a great job to make it more graphically convenient for users migrating from Windows OS.

Though it is generally intuitive, there is one little thing that might be useful for novice Mint user – where from to make Trash icon.

There are two ways to do it.

1. Is by installing / launching gnome-tweak-tool

Linux Mint 12 desktop trashbin screenshot

I personally prefer gnome-tweak-tool, cause it has plenty of nice options related to how GUI environment, behaves. I believe even non Linux-Mint GNOME 3 users should take a look at gnome-tweak-tool if already haven't as it allows user to tailer plenty of desktop nice stuff.

2. Through [Main Menu]

-> Preferences -> Desktop Settings -> Desktop -> Desktop icons

Linux Mint desktop how to visualize trash bin on desktop screenshot

Get more peaceful night sleep on Ubuntu, Mint and Xubuntu Linux using gtk-redshift

Monday, March 11th, 2013

Reading Time: 2minutes

gtk redshift Xubuntu Linux screenshot sleep peacefully when using computer at late

If you want to have more peaceful night sleep when working on Ubuntu or other Debian based Linux distro, be sure to have gtk-redshift installed.
It is a little program that simply changes the color gamma of screen and makes your screen look more reddish at night. According to many scientific research done on how we humans react, whether using computer late at night. It is concluded that less bright colors and especially reddish color gamma relaxes our eye strain and thus makes it easier for us to get a sleep quickly once in bed. gtk-redshift is available in latest Ubuntu 12.04 as well as on other Ubuntu derivatives (Xubuntu, Mint Linux) etc.

Easiest way to install it is via respective GUI Package Manager or via good old Synaptic (GUI aptitude frontend).
I personally prefer to always install Synaptic on new Desktop Linux PCs, use it as package GUI frontend, for the simple reason it offers one very similar "unified" package Installer outlook across different Linux distros.

The quickest way to use GUI version of Redshift is to install with apt:

root@xubuntu:~# apt-get install --yes gtk-redshift
....

To further use it it needs one time to be run with color gamma paraments, launch it first time via terminal with:
user@xubuntu:~$ gtk-redshift  -l 52.5:13.4

It is a good idea to make a tiny shell script wrapper with good settings for gtk-redshift and later use this shell wrapper as launcher :

root@xubuntu:~# echo '#!/bin/sh' >> /usr/local/bin/gtk-redshift
root@xubuntu:~# echo 'gtk-redshift' >> /usr/local/bin/gtk-redshift
root@xubuntu:~# chmod +x /usr/local/bin/gtk-redshift

From then on, to launch it you can directly open it via terminal

user@xubuntu:~$ /usr/local/bin/gtk-redshift

To make the program permanently work, make it run via respective GUI environment startup . In GNOME add it start-up from:

user@xubuntu:~$ gnome-session-manager

Important note to make about gtk-redshift is that on some older monitor screens, very early in morning the screen becomes too red, making screen look like displaying on very old long time used CRT monitors. For people working in fields like; Web Design, Architecture, or any drawing twisted colors effect will be annoying and will probably interfere with your perception of colors. However for programmers, system administrators and people who use computer mainly for typing and reading gtk-redshift is huge blessing.

Enjoy ! 🙂
 

How to change default port 25 to 995 or any other on Postfix mail server

Wednesday, February 27th, 2013

Reading Time: < 1minute

If you need to change default mail delivery port of Postfix mail server from 25 to any other port lets say to 995. Here is how:

Edit /etc/postfix/master.cf  – a small note to make here is master.cf is located on same location on mostly all Linux distributions as well as FreeBSD.

Find line with:

smtp      inet  n       –       n       –       –       smtpd

Change smtp – which is actually a reference for port 25 to whatever port you need – i.e. 995 After change line should look like:

995      inet  n       –       n       –       –       smtpd

Saveconfig file and restart postfix to load new settings:

# /etc/rc.d/postfix restart
postfix/postfix-script: stopping the Postfix mail system
postfix/postfix-script: starting the Postfix mail system

Secure delete files irreverseble on Debian and Fedora GNU / Linux

Thursday, February 21st, 2013

Reading Time: 2minutes

I just read an article in Linux-Magazine on Advanced File Management named – "Beyond the Basics". Most of what the article says is pretty trivial and known by any Linux enthusiast average user and administrator. There was one command mentioned shred which is probably not so well known among Free Software users shred allows the user to "secure delete files" / from the hard disk irreversible.

The tool is part of coreutils package and available across mostly all Linux distributions including Debian / Ubuntu debian derivatives and the RedHat based distros CentOS, Fedora, RHEL etc.

Just for info for those who don't know how to check, to which package a command belongs with rpm and dpkg, here is how;

[hipo@centos ~]$ rpm -qf /usr/bin/shred
coreutils-5.97-23.el5_4.2

hipo@debian:~$ dpkg -S /usr/bin/shred
coreutils: /usr/bin/shred

Here is how to delete a sample file ovewritting 3 times (-n2 – means 3 because in comuters we know we countr from 0 – 0 1 2 3), the z option fills up with zeros after overwritting the file ( just like seen on paste), -v option shows verbose what shred is doing and -u option truncates removes file after overwritting

noah:/var/tmp# shred -n2 -zvu crash20121113021508.txt
shred: crash20121113021508.txt: pass 1/3 (random)…
shred: crash20121113021508.txt: pass 2/3 (random)…
shred: crash20121113021508.txt: pass 3/3 (000000)…
shred: crash20121113021508.txt: removing
shred: crash20121113021508.txt: renamed to 00000000000000000000000
shred: 00000000000000000000000: renamed to 0000000000000000000000
shred: 0000000000000000000000: renamed to 000000000000000000000
shred: 000000000000000000000: renamed to 00000000000000000000
shred: 00000000000000000000: renamed to 0000000000000000000
shred: 0000000000000000000: renamed to 000000000000000000
shred: 000000000000000000: renamed to 00000000000000000
shred: 00000000000000000: renamed to 0000000000000000
shred: 0000000000000000: renamed to 000000000000000
shred: 000000000000000: renamed to 00000000000000
shred: 00000000000000: renamed to 0000000000000
shred: 0000000000000: renamed to 000000000000
shred: 000000000000: renamed to 00000000000
shred: 00000000000: renamed to 0000000000
shred: 0000000000: renamed to 000000000
shred: 000000000: renamed to 00000000
shred: 00000000: renamed to 0000000
shred: 0000000: renamed to 000000
shred: 000000: renamed to 00000
shred: 00000: renamed to 0000
shred: 0000: renamed to 000
shred: 000: renamed to 00
shred: 00: renamed to 0
shred: crash20121113021508.txt: removed
 

One common use of shred is by sysadmins who has to prepare old server containing lets say client data (SQL) – mail boxes or just file data and then sell it to third parties making sure data will be un-restorable for the new owner. Also shred is used a lot by crackers who set up "time bombs" activated on user activity or inactivity to destroy evidences in case of crackers PC is being captured by police. Though shred cannot guarantee 100% that deleted data can't be recoved within a special data recovery lab in most of cases it is enough to assure data with it will be almost impossible to recover.