Archive for August, 2011

How to Secure Apache on FreeBSD against Range header DoS vulnerability (affecting Apache 1.3/2.x)

Tuesday, August 30th, 2011

How to Secure Apache webserver on FreeBSD and CentOS against Range: header Denial of Service attack

Recently has become publicly known for the serious hole found in all Apache webserver versions 1.3.x and 2.0.x and 2.2.x. The info is to be found inside the security CVE-2011-3192 https://issues.apache.org/bugzilla/show_bug.cgi?id=51714

Apache remote denial of service is already publicly cirtuculating, since about a week and is probably to be used even more heavily in the 3 months to come. The exploit can be obtained from exploit-db.com a mirror copy of #Apache httpd Remote Denial of Service (memory exhaustion) is for download here

The DoS script is known in the wild under the name killapache.pl
killapache.pl PoC depends on perl ForkManager and thus in order to be properly run on FreeBSD, its necessery to install p5-Parallel-ForkManager bsd port :


freebsd# cd /usr/ports/devel/p5-Parallel-ForkManager
freebsd# make install && make install clean
...

Here is an example of the exploit running against an Apache webserver host.


freebsd# perl httpd_dos.pl www.targethost.com 50
host seems vuln
ATTACKING www.targethost.com [using 50 forks]
:pPpPpppPpPPppPpppPp
ATTACKING www.targethost.com [using 50 forks]
:pPpPpppPpPPppPpppPp
...

In about 30 seconds to 1 minute time the DoS attack with only 50 simultaneous connections is capable of overloading any vulnerable Apache server.

It causes the webserver to consume all the machine memory and memory swap and consequently makes the server to crash in most cases.
During the Denial of Service attack is in action access the websites hosted on the webserver becomes either hell slow or completely absent.

The DoS attack is quite a shock as it is based on an Apache range problem which started in year 2007.

Today, Debian has issued a new versions of Apache deb package for Debian 5 Lenny and Debian 6, the new packages are said to have fixed the issue.

I assume that Ubuntu and most of the rest Debian distrubtions will have the apache’s range header DoS patched versions either today or in the coming few days.
Therefore work around the issue on debian based servers can easily be done with the usual apt-get update && apt-get upgrade

On other Linux systems as well as FreeBSD there are work arounds pointed out, which can be implemented to close temporary the Apache DoS hole.

1. Limiting large number of range requests

The first suggested solution is to limit the lenght of range header requests Apache can serve. To implement this work raround its necessery to put at the end of httpd.conf config:


# Drop the Range header when more than 5 ranges.
# CVE-2011-3192
SetEnvIf Range (?:,.*?){5,5} bad-range=1
RequestHeader unset Range env=bad-range
# We always drop Request-Range; as this is a legacy
# dating back to MSIE3 and Netscape 2 and 3.
RequestHeader unset Request-Range
# optional logging.
CustomLog logs/range-CVE-2011-3192.log common env=bad-range
CustomLog logs/range-CVE-2011-3192.log common env=bad-req-range

2. Reject Range requests for more than 5 ranges in Range: header

Once again to implement this work around paste in Apache config file:

This DoS solution is not recommended (in my view), as it uses mod_rewrite to implement th efix and might be additionally another open window for DoS attack as mod_rewrite is generally CPU consuming.


# Reject request when more than 5 ranges in the Range: header.
# CVE-2011-3192
#
RewriteEngine on
RewriteCond %{HTTP:range} !(bytes=[^,]+(,[^,]+){0,4}$|^$)
# RewriteCond %{HTTP:request-range} !(bytes=[^,]+(?:,[^,]+){0,4}$|^$)
RewriteRule .* - [F]

# We always drop Request-Range; as this is a legacy
# dating back to MSIE3 and Netscape 2 and 3.
RequestHeader unset Request-Range

3. Limit the size of Range request fields to few hundreds
To do so put in httpd.conf:


LimitRequestFieldSize 200

4. Dis-allow completely Range headers: via mod_headers Apache module

In httpd.conf put:


RequestHeader unset Range
RequestHeader unset Request-Range

This work around could create problems on some websites, which are made in a way that the Request-Range is used.

5. Deploy a tiny Apache module to count the number of Range Requests and drop connections in case of high number of Range: requests

This solution in my view is the best one, I’ve tested it and I can confirm on FreeBSD works like a charm.
To secure FreeBSD host Apache, against the Range Request: DoS using mod_rangecnt, one can literally follow the methodology explained in mod_rangecnt.c header:


freebsd# wget http://people.apache.org/~dirkx/mod_rangecnt.c
..
# compile the mod_rangecnt module
freebsd# /usr/local/sbin/apxs -c mod_rangecnt.c
...
# install mod_rangecnt module to Apache
freebsd# /usr/local/sbin/apxs -i -a mod_rangecnt.la
...

Finally to load the newly installed mod_rangecnt, Apache restart is required:


freebsd# /usr/local/etc/rc.d/apache2 restart
...

I’ve tested the module on i386 FreeBSD install, so I can’t confirm this steps works fine on 64 bit FreeBSD install, I would be glad if I can hear from someone if mod_rangecnt is properly compiled and installed fine also on 6 bit BSD arch.

Deploying the mod_rangecnt.c Range: Header to prevent against the Apache DoS on 64 bit x86_amd64 CentOS 5.6 Final is also done without any pitfalls.


[root@centos ~]# uname -a;
Linux centos 2.6.18-194.11.3.el5 #1 SMP Mon Aug 30 16:19:16 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
[root@centos ~]# /usr/sbin/apxs -c mod_rangecnt.c
...
/usr/lib64/apr-1/build/libtool --silent --mode=link gcc -o mod_rangecnt.la -rpath /usr/lib64/httpd/modules -module -avoid-version mod_rangecnt.lo
[root@centos ~]# /usr/sbin/apxs -i -a mod_rangecnt.la
...
Libraries have been installed in:
/usr/lib64/httpd/modules
...
[root@centos ~]# /etc/init.d/httpd configtest
Syntax OK
[root@centos ~]# /etc/init.d/httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]

After applying the mod_rangecnt patch if all is fine the memory exhaustion perl DoS script‘s output should be like so:


freebsd# perl httpd_dos.pl www.patched-apache-host.com 50
Host does not seem vulnerable

All of the above pointed work-arounds are only a temporary solution to these Grave Apache DoS byterange vulnerability , a few days after the original vulnerability emerged and some of the up-pointed work arounds were pointed. There was information, that still, there are ways that the vulnerability can be exploited.
Hopefully in the coming few weeks Apache dev team should be ready with rock solid work around to the severe problem.

In 2 years duration these is the second serious Apache Denial of Service vulnerability after before a one and a half year the so called Slowloris Denial of Service attack was capable to DoS most of the Apache installations on the Net.

Slowloris, has never received the publicity of the Range Header DoS as it was not that critical as the mod_range, however this is a good indicator that the code quality of Apache is slowly decreasing and might need a serious security evaluation.

Monitoring Windows hosts with Nagios on Debian GNU/Linux

Tuesday, August 30th, 2011

Nagios logo install and configure nagios to monitor Windows hosts with on Debian GNU/Linux

In this article in short, I’ll explain how I configured Nagios on a Debian GNU/Linux release (Squeeze 6) to monitor a couple of Windows hosts running inside a local network. Now let’s start.

1. Install necessery nagios debian packages

apt-get install nagios-images nagios-nrpe-plugin nagios-nrpe-server nagios-plugins nagios-plugins-basic nagios-plugins-standard
nagios3 nagios3-cgi nagios3-common nagios3-core

2. Edit /etc/nagios-plugins/config/nt.cfg

In the File substitute:

define command { command_name check_nt command_line /usr/lib/nagios/plugins/check_nt -H '$HOSTADDRESS$' -v '$ARG1$' }

With:

define command {
command_name check_nt
command_line /usr/lib/nagios/plugins/check_nt -H '$HOSTADDRESS$' -p 12489 -v $ARG1$ $ARG2$
}

3. Modify nrpe.cfg to put in allowd hoss to connect to the Nagions nrpe server

vim /etc/nagios/nrpe.cfg

Lookup inside for nagios’s configuration directive:

allowed_hosts=127.0.0.1

In order to allow more hosts to report to the nagios nrpe daemon, change the value to let’s say:

allowed_hosts=127.0.0.1,192.168.1.4,192.168.1.5,192.168.1.6

This config allows the three IPs 192.168.1.4-6 to be able to report for nrpe.

For the changes to nrpe server to take effect, it has to be restrarted.

debian:~# /etc/init.d/nagios-nrpe-server restart

Further on some configurations needs to be properly done on the nrpe agent Windows hosts in this case 192.168.1.4,192.168.1.5,192.168.1.6

4. Install the nsclient++ on all Windows hosts which CPU, Disk, Temperature and services has to be monitored

Download the agent from http://sourceforge.net/projects/nscplus and launch the installer, click twice on it and follow the installation screens. Its necessery that during installation the agent has the NRPE protocol enabled. After the installation is complete one needs to modify the NSC.ini
By default many of nsclient++ tracking modules are not enabled in NSC.ini, thus its necessery that the following DLLs get activated in the conf:

FileLogger.dll
CheckSystem.dll
CheckDisk.dll
NSClientListener.dll
SysTray.dll
CheckEventLog.dll
CheckHelpers.dll

Another requirement is to instruct the nsclient++ angent to have access to the Linux installed nagios server again with adding it to the allowed_hosts config variable:

allowed_hosts=192.168.1.1

In my case the Nagios runs on Debian Lenny (Squeeze) 6 and possess the IP address of 192.168.1.1
To test the intalled windows nsclient++ agents are properly installed a simple telnet connection from the Linux host is enough:

5. Create necessery configuration for the nagios Linux server to include all the Windows hosts which will be monitored

There is a window.cfg template file located in /usr/share/doc/nagios3-common/examples/template-object/windows.cfg on Debian.

The file is a good start point for creating a conf file to be understand by nagios and used to periodically refresh information about the status of the Windows hosts.

Thus it’s a good idea to copy the file to nagios3 config directory:

debian:~# mkdir /etc/nagios3/objects
debian:~# cp -rpf /usr/share/doc/nagios3-common/examples/template-object/windows.cfg /etc/nagios3/objects/windows.cfg

A sample windows.cfg content, (which works for me fine) and monitor a couple of Windows nodes running MS-SQL service and IIS and makes sure the services are up and running are:

define host{
use windows-server ; Inherit default values from a template
host_name Windows1 ; The name we're giving to this host
alias Iready Server ; A longer name associated with the host
address 192.168.1.4 ; IP address of the host
}
define host{
use windows-server ; Inherit default values from a template
host_name Windows2 ; The name we're giving to this host
alias Iready Server ; A longer name associated with the host
address 192.168.1.4 ; IP address of the host
}
define hostgroup{
hostgroup_name windows-servers ; The name of the hostgroup
alias Windows Servers ; Long name of the group
}
define hostgroup{
hostgroup_name IIS
alias IIS Servers
members Windows1,Windows2
}
define hostgroup{
hostgroup_name MSSQL
alias MSSQL Servers
members Windows1,Windows2
}
define service{
use generic-service
host_name Windows1
service_description NSClient++ Version
check_command check_nt!CLIENTVERSION
}
define service{ use generic-service
host_name Windows1
service_description Uptime
check_command check_nt!UPTIME
}
define service{ use generic-service
host_name Windows1
service_description CPU Load
check_command check_nt!CPULOAD!-l 5,80,90
}
define service{
use generic-service
host_name Windows1
service_description Memory Usage
check_command check_nt!MEMUSE!-w 80 -c 90
define service{
use generic-service
host_name Windows1
service_description C: Drive Space
check_command check_nt!USEDDISKSPACE!-l c -w 80 -c 90
}
define service{
use generic-service
host_name Windows1
service_description W3SVC
check_command check_nt!SERVICESTATE!-d SHOWALL -l W3SVC
}
define service{
use generic-service
host_name Windows1
service_description Explorer
check_command check_nt!PROCSTATE!-d SHOWALL -l Explorer.exe
}
define service{
use generic-service
host_name Windows2
service_description NSClient++ Version
check_command check_nt!CLIENTVERSION
}
define service{ use generic-service
host_name Windows2
service_description Uptime
check_command check_nt!UPTIME
}
define service{ use generic-service
host_name Windows2
service_description CPU Load
check_command check_nt!CPULOAD!-l 5,80,90
}
define service{
use generic-service
host_name Windows2
service_description Memory Usage
check_command check_nt!MEMUSE!-w 80 -c 90
define service{
use generic-service
host_name Windows2
service_description C: Drive Space
check_command check_nt!USEDDISKSPACE!-l c -w 80 -c 90
}
define service{
use generic-service
host_name Windows2
service_description W3SVC
check_command check_nt!SERVICESTATE!-d SHOWALL -l W3SVC
}
define service{
use generic-service
host_name Windows2
service_description Explorer
check_command check_nt!PROCSTATE!-d SHOWALL -l Explorer.exe
}
define service{ use generic-service
host_name Windows1
service_description SQL port Check
check_command check_tcp!1433
}
define service{
use generic-service
host_name Windows2
service_description SQL port Check
check_command check_tcp!1433
}
The above config, can easily be extended for more hosts, or if necessery easily setup to track more services in nagios web frontend.
6. Test if connectivity to the nsclient++ agent port is available from the Linux server

debian:~# telnet 192.168.58.6 12489
Trying 192.168.58.6...
Connected to 192.168.58.6.
Escape character is '^]'.
asd
ERROR: Invalid password.

Another good idea is to launch on the Windows host the NSClient++ (system tray) , e.g.:

Start, All Programs, NSClient++, Start NSClient++ (system tray).

Test Nagios configuration from the Linux host running nagios and nrpe daemons to check if the check_nt, can succesfully authenticate and retrieve data generated from the nsclient++ on the Windows host:

debian:~# /usr/lib/nagios/plugins/check_nt -H 192.168.1.5 -p 12489 -v CPULOAD -w 80 -c 90 -l 5,80,90,10,80,90

If everything is okay and the remote Windows system 192.168.1.5 has properly configured and running NSClient++ the above command should return an output like:

CPU Load 1% (5 min average) 1% (10 min average) | '5 min avg Load'=1%;80;90;0;100 '10 min avg Load'=1%;80;90;0;100

In case of the command returns:

could not fetch information from server

instead this means that probably there is some kind of problem with authentication or handshake of the Linux host’s nagios check_nt to the Windows server’s running on 12489.

This is sometimes caused by misconfigured NSC.ini file, however in other occasions this error is caused by misconfigured Windows Firewall or because the NSClient++ is not running with Administrator user.

By the way important note to make about Windows 2008r2 is that if NSClient++ is running there it’s absolutely required to Login with Windows Administrator and run the NSClient++ /start , if it’s run through the Run As Adminsitrator with an admin privileged user the aforementioned error might appear, so be careful.
I’ve experienced this error myself and it took me about 40 minutes to find that I have to run it directly with Administrator user after logging as Administrator.

7. Create nagios web iface Apache configuration

nagios debian pachage is shipped with a config which is suitable to be set debian:~# cp -rpf /usr/share/doc/nagios3-common/examples/apache2.conf /etc/apache2/sites-avalable/nagios
debian:~# ln -sf /etc/apache2/sites-available/nagios /etc/apache2/sites-enabled/nagios

The /etc/apache2/sites-available/nagios can easily be configured to work on Virtualhost, to do so the above copied file need to be wrapped inside a VirtualHost directive. For that put in the beginning of the file;

<VirtualHost *:80>

and in the end of the file:

<VirtualHost *:80>

8. Restart nagios server and Apache for the new settings to take effect

debian:~# /etc/init.d/apache2 restart
...
debian:~# /etc/init.d/nagios3 restart

If some custom configuration about tracking the Debian Linux nagios host running services needs to be made, its also helpful for one to check in /etc/nagios3/conf.d

Well that’s mostly what I had to do to make the Nagios3 server to keep track of a small Windows network on Debian GNU/Linux Squeeze 6, hope this small article helps. Cheers 😉

Using perl and sed to substitute strings in multiple files on Linux and BSD

Friday, August 26th, 2011

Using perl and sed to replace strings in files on Linux, FreeBSD, OpenBSD, NetBSD and other UnixOn many occasions when had to administer on Linux, BSD, SunOS or any other *nix, there is a need to substitute strings inside files or group of files containing a certain string with another one.

The task is not too complex and many of the senior sysadmins out there would certainly already has faced this requirement and probably had a good idea on files substitution with perl and sed, however I’m quite sure there are dozen of system administrators out there who did not know, how and still haven’t faced a situation where there i a requirement to substitute from a command shell or via a scripting language.

This article tagets exactly these system administrators who are not 100% sys op Gurus 😉

1. Substitute text strings inside files on Linux and BSD with perl

Perl programming language has originally been created to do a lot of text manipulation as well as most of the Linux / Unix based hosts today have installed working copy of perl , therefore using perl as a mean to substitute one string in a file to another one is maybe the best way to completet the task.
Another good thing about perl is that text processing with it is said to be in most cases a bit faster than sed .
However it is still dependent on the string to be substituted I haven’t done benchmark tests to positively say 100% that always perl is quicker, however my common sense suggests perl will be quicker.

Now enough talk here is a very simple way to substitute a reoccuring, text string inside a file with another chosen one is like so:

debian:~# perl -pi -e 's/foo/bar/g' file1 file2

This will substitute the string foo with bar everywhere it’s matched in file1 and file2

However the above code is a bit “dangerous” as it does not preserve a backup copy of the original files, where string is substituted is not made.
Therefore using the above command should only be used where one is 100% sure about the string changes to be made.

Hence a better idea whether conducting the text substitution is to keep also the original file backup under a let’s say .bak extension. To achieve that I use perl as follows:

freebsd# perl -i.bak -p -e 's/syzdarma/magdanoz/g;' file1 file2

This command creates copies of the original files file1 and file2 under the names file1.bak and file2.bak , the files file1 and file2 text occurance of strings syzdarma will get substituted with magdanoz using the option /g which means – (substitute globally).

2. Substitute string in all files inside directory using perl on Linux and BSD

Every now and then the there is a need to do manipulations with large amounts of files, I can’t right now remember a good scenario where I had to change all occuring matching strings to anther one to all files located inside a directory, anyhow I’ve done this on a number of occasions.

A good way to do a mass file string substitution on Linux and BSD hosts equipped with a bash shell is via the commands:

debian:/root/textfiles:# for i in $(echo *.txt); do perl -i.bak -p -e 's/old_string/new_string/g;' $i; done

Where the text files had the default txt file extension .txt

Above bash loop prints each of the files located in /root/textfiles and substitutes everywhere (globally) the old_string with new_string .

Another alternative to the above example to replace multiple occuring text string in all files in multiple directories is possible using a combination of shell commands grep, perl, sort, uniq and xargs .
Let’s say that one wants to match everywhere inside the root directory and all the descendant directories for files with a custom string and substitute it to another one, this can be done with the cmd:

debian:~# grep -R -files-with-matches 'old_string' / | sort | uniq | xargs perl -pi~ -e 's/old_string/new_string/g'

This command will lookup for string old_string in all files in the / – root directory and in case of occurance will substitute with new_string (This command’s idea was borrowed as an idea from http://linuxadmin.org so thx.).

Using the combination of 5 commands, however is not very wise in terms of efficiency.

Therefore to save some system resources, its better in terms of efficiency to take advantage of the find command in combination with xargs , here is how:

debian:~# find / | xargs grep 'old_string' -sl |uniq | xargs perl -pi~ -e 's/old_string/new_string/g'

Once again the find command example will do exactly the same as the substitute method with grep -R …

As enough is said about the way to substitute text strings inside files using perl, I will further explain how text strings can be substituted using sed

The main reason why using sed could be a better choice in some cases is that Unices are not equipped by default with perl interpreter. In general the amount of servers who contains installed sed compared to the ones with perl language interpreter is surely higher.

3. Substitute text strings inside files on Linux and BSD with sed stream editor

In many occasions, wether a website is hosted, one needs to quickly conduct a change in string inside all files located in a directory, to resolve issues with static urls directly encoded in html.
To achieve this task here is a code using two little bash script loops in conjunctions with sed, echo and mv commands:

debian:/var/www/website# for i in $(ls -1); do cat $i |sed -e "s#index.htm#http://www.webdomain.com/#g">$i.new; done
debian:/var/www/website# for i in $(ls *.new); do mv $i $(echo $i |sed -e "s#.new##g"); done

The above command sed -e “s#index.htm#http://www.webdomain.com/#g”, instructs sed to substitute all appearance of the text string index.htm to the new text string http://www.webdomain.com

First for bash loop, creates all the files with substituted string to file1.new, file2.new, file3.new etc.
The second for loop uses mv to overwrite the original input files file1, file2, file3, etc. with the newly created ones file1.new, file2.new, file3.new

There is a a way shorter way to conclude the same text substitutions task using a simpler one liner with only using sed and bash’s eval capabilities, here is how:

debian:/var/www/website# sed -i 's/old_string/new_string/g' *

Above command will change old_string to new_string inside all files in directory /var/www/website

Whether a change has to be made with less than 1024 files using this method might be more efficient, however whether a text substitute has to be done to let’s say 5000+ the above simplistic version will not work. An error of Argument list too long will prevent the sed -i ‘s/old_string/new_string/g’ to complete its task.

The above for loop 2 liner should be also working without problems with FreeBSD and the rest of BSD derivatives, though I have not tested it yet, hence any feedback from FreeBSD guys is mostly welcome.

Consider that in order to have the for loops commands work on FreeBSD or NetBSD, they have to be run under a bash shell.
That’s all folks thanks the Lord for letting me write this nice article, I hope it gives some insights on how multiple files text replace on Unix works .
Cheers 😉

Possible way to increase Linux TCP/IP port thoroughput via sysctl kernel variable

Thursday, August 25th, 2011

Sysctl is a great way to optimize Linux. sysctl has a dozens of values which could drastically improve server networking and overall performance.

One of the many heplful variables to optimize the way the Linuz kernel works on busy servers is net.ipv4.ip_local_port_range .

The default sysctl setting for net.ipv4.ip_local_port_range on Debian, Ubuntu Fedora, RHEL, CentOS is:

net.ipv4.ip_local_port_range = 32768 65536

This means that the kernel and the corresponding server running services instructing the Linuz kernel open new port sockets can only open local ports in the range of 32768 – 65536 .
On a regular Desktop GNU/Linux machine or a not high iron server this settins is perfectly fine, however on a high scale servers the local port range in the interval of 32768-65536 might be insufficient at times, especially if there are programs which require binding of many local ports.

Therefore on a high load servers, generally it’s a good to raise the port range to be assigned by kernel to 8912 – 65536 , to do so the setting has to be changed like shown below:

linux:~# sysctl -w net.ipv4.ip_local_port_range = 8192 65536
...

If changing this setting on the server doesn’t show any negative impact on performance in few hours time or a day or even better decreases the server average load, it’s a good idea that it be added to sysctl.conf to load up the setting on next kernel boot.

linux:~# echo 'net.ipv4.ip_local_port_range' >> /etc/sysctl.conf

Enjoy 😉

Fix to Nagios is currently not checking for external commands

Wednesday, August 24th, 2011

While I was deploying a new Nagios install to Monitor some Windows hosts I’ve came across the following error in Nagios’s web interface:

Sorry, but Nagios is currently not checking for external commands, so your command will not be committed!
Read the documentation for information on how to enable external commands...

This error is caused by an option configuration for /etc/nagios/nrpe.cfg (part of the nrpe-nagios-server Debian package.

The config variable in nrpe.cfg causing the error is check_external_command=0 , the fix comes to changing the variable to:

check_external_command=1

As well as restart the /etc/init.d/nagios-nrpe-server and /etc/init.d/nagios3 services:

debian:~# /etc/init.d/nagios3 restart
...
debian:~# /etc/init.d/nagios-nrpe-server
...

This changes has work out the error Sorry, but Nagios is currently not checking for external commands, so your command will not be committed! , however immediately after another kind of error appared in Nagios web interface when I tried to use the send Nagios commands button. The error was:

Error: Could not stat() command file '/var/lib/nagios3/rw/nagios.cmd'!

This error is due to a deb package, which seems to be affecting the current deb versions of Nagios shipped with Debian 6 Squeeze stable, as well as the Latest Ubuntu release 11.04.

Thanksfully there is a work around to the problem I found online, to fix it up I had to execute the commands:

debian:~# /etc/init.d/nagios3 stop debian:~# dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios3/rw
debian:~# dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios3
debian:~# /etc/init.d/nagios3 start

And hooray Thanks God the error is gone 😉

How to configure Exim to relay mail to remote SMTP server on Debian and Ubuntu

Wednesday, August 24th, 2011

I’m required to do some mail relaying on a Debian Linux host which should use a remote mail server to relay its mails.
Until so far I’ve had not much experience with exim as I prefer using qmail, whever a mail server is needed. However since now only a relaying was necessery and exim is the default installed MTA on Debian, I’ve decided to use exim to take care of the SMTP mail relaying.
After a bit of reading it happened configuring exim to relay via remote SMTP server is more than easy!

All I had to do is run the command:

debian-relay:~# dpkg-reconfigure exim4-config

Next in the Ncruses interface to appear:

Debian Exim relay smtp config screenshot

I had to choose the option:

mail sent by smarthost; no local mail

Next a dialog appears asking for:
System mail name:
Therein it’s necessery to type in the hostname of the remote SMTP to be used for mail relay.
Next dialog asks for:
IP-addresses to listen on for incoming SMTP connections:
and I left it with 127.0.0.1 however if exim is supposed to be visible from external network one might decide to put in real IP address there.

Pressing OK leads to the next dialog:
 Other destinations for which mail is accepted: 
I decided to leave this blank as I don’t want to accept mail for any destinations.
Next pane reads:
Visible domain name for local users:
I’ve typed inside my smtp relay server e.g.:
smtp.myrelaymail.com

Further comes:
IP address or host name of the outgoing smarthost:
There once again I typed my mail relay host smtp.relaymail.com

The next config screen is:
Keep number of DNS-queries minimal (Dial-on-Demand)?
On any modern Linux host the default answer of No is fine.
Following prompt asked if I want to:
Split configuration into small files?
I’ve decided not to tamper with it and choosed No
Afterwards mail relaying works like a charm thx God 😉

How to change mail sent from in Nagios on Debian GNU/Linux 6

Wednesday, August 24th, 2011

I’ve been playing with configuring a new nagios running on a Linux host which’s aim is to monitor few Windows servers.
The Linux host’s exim is configured to act as relay host to another SMTP server, so all email ending up in the Linux localhost on port 25 is forwarded to the remote SMTP.

The remote smtp only allows the Linux to send email only in case if a real existing username@theserverhostname.com is passed it, otherwise it rejects mail and does not sent properly the email.
As the newly configured Nagios installatio is supposed to do e-mail notification, I was looking for a way to change the default user with which Nagios sends mails, which is inherited directly after the username with which /usr/sbin/nagios3 and /usr/sbin/nrpe are running (on Debian this is nagios@theserverhostname.com).

Thanksfully, there is a work around, I’ve red some forum threads explaning that the username with whch nagios sends mail can be easily changed from /etc/nagios3/commands.cfg by passing the -a “From: custom_user@myserverhostname.com” to all occurance of /usr/bin/mail -s , its preferrable that the -a custom_user@myserverhostname.com is inserted before the -s “” subject option. Hence the occurance of mail command should be changed from:

| /usr/bin/mail -s "** $NOTIFICATIONTYPE$

To:

| /usr/bin/mail -a "From: custom_user@theserverhostname.com" -s "** $NOTIFICATIONTYPE$

Now to read it’s new configurations nagios requirs restart:

debian:~# /etc/init.d/nagios3 restart
...

Now in case of failed services or Hosts Down nagios will send it’s mail from the custom user custom_user@theserverhostname.com and nagios can can send mail properly via the remote relay SMTP host 😉

How to make NAT enable hosts in a local network to access the internet, create port forwarding to local IPs behind the router using iptables

Tuesday, August 23rd, 2011

I’m bulding new iptables firewall on one Linux server. The Debian GNU/Linux is required to act as firewall do Network Adress Translation for a small network of office PCs as well as forward some of the inbound ports to hosts from the local network located behind the router.

The local network besides the router had an IP addressing in the class C network e.g. (192.168.1.1-255)

First I procceded and enabled the Network Address Translation via the Linux kernel variable:

linux:~# sysctl -w net.ipv4.ip_forward=1
linux:~# echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.conf

Initially I even forgot to switch on the net.ipv4.ip_forward to 1 (by default this value is set to 0) – GNU/Linux’s default network behaviour is not predetermined to act as network router.
However, since I haven’t configured Network Address Translation for quite some time it completely slipped my mind!

Anyways next the actual iptables rule which makes NAT possible I used is:

linux:~# /sbin/iptables -t nat -A POSTROUTING -s 192.168.1.0/24 ! -d 192.168.1.0/24 -j SNAT --to-source xxx.xxx.xxx.xxx

Whether xxx.xxx.xxx.xxx is the External IP address assigned to the router on eth0

With this very simple rules now Network the local network is capable of accessing the Internet withotu problem.

It’s a good time to say that still many system administrators, still erroneously use MASQUERADE rules instead of SNAT .
IP MASQUERADING is an ancestry from ipchains and these days should be completely abandonded, especially where no often change of primary IP address to access the internet is made.
For dial-ups or other kind of networking, where the IP addresses are often changed still IP MASQUERADING might be a good idea though.

My next goal was to make the Linux router to do port forwarding of Traffic which arrives on port 80 to a IIS server assigned with a local IP address of 192.168.1.5
I did the webserver (port 80), port forwarding from IP xxx.xxx.xxx.xxx to 192.168.1.5 with the iptables rule:

linux:~# /sbin/iptables -t nat -A PREROUTING -d xxx.xxx.xxx.xxx/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.5:80

There was a requirement to do port forwarding for a Windows remote Desktop running on standard port 3389 from the router to the internal Windows IP address running the IIS webserver, however the company required me to only allow access to the rdesktop 3389 port to certain real IP addresses.
Initially I thought about using the above PREROUTING rule which makes the port redirection to the IIS server and only change port 80 to port 3389 , and then use filter table INPUT chain rules like:

/sbin/iptables -A INPUT -s xx1.xx2.xx3.xx4,1xx,2xx,3xx,4xx,xxx.xxx.xxx.xxx -p tcp -m tcp --dport 3389 -j ACCEPT/sbin/iptables -A INPUT -p tcp -m tcp --dport 3389 -j REJECT --reject-with icmp-port-unreachable
32

However this did not work out, so I decided to give a try to do the same within the filter table using the FORWARD chain, like so:

FORWARD/sbin/iptables -A FORWARD -p tcp -m tcp -s xx1.xx2.xx3.xx4,1xx,2xx,3xx,4xx,xxx.xxx.xxx.xxx -p tcp -m tcp --dport 3389 -j ACCEPT
/sbin/iptables -A FORWARD -p tcp -m tcp --dport 3389 -j REJECT --reject-with icmp-port-unreachable

Adding this rules did not added any filtering to the forwarded remote desktop port. I suspected that somehow probably my above PREROUTING nat rules are read before any other rules and therefore automatically allows any IP address to port fortward traffic.
I’ve checked the iptables documentation and it seems my guess was partially right.

When some kind of network traffic enters the iptables firewall it first goes through the PREROUTING channel and then the traffic flows in a certain order.
iptables packet flow diagram

The iptables network packets flow is clearly seen in above’s diagram a thorough looks gives a very good idea on how packet is being processed by iptables

Finally as I couldn’t think about a good solution on how to only filter the port redirected traffic, which always firstly entered in the POSTROUTING chain, I’ve consulted with the guys in irc.freenode.net in #Netfilter.

I’m quite thanksful as a guy nicknamed Olipro has given me a pretty good picture on the port forwarding POSTROUTING problem and has provided me with a very logical easy and great fix.
He suggested that I only do port forwarding for certain IP addresses instead of allowing all IP addresses and then lookup for a way to allow only some of them and filter the rest.

The iptables rule to restrict the incoming traffic to the remote desktop forwarded port 3389 to few only allowed IP addresses looks like so:

linux:~# /sbin/iptables -t nat -A PREROUTING -d xxx.xxx.xxx.xxx/32 -s xx1.xx2.xx3.xx4,1xx,2xx,3xx,4xx,xxx.xxx.xxx.xxx -p tcp -m tcp –dport 3389 -j DNAT –to-destination 192.168.1.5:3389

Now the three sample IPs passed xx1.xx2.xx3.xx4,1xx,2xx,3xx,4xx,xxx.xxx.xxx.xxx has added to port forward traffic on 3389 to 192.168.1.5

By the way I did not know that newer versions of iptables support passing by multiple IP addresses to the –source or –destination IP. This is really great feature I’ve learned from the good guys from #Netfilter. However one should be careful when using the multiple IPs with -s or -d, it’s really important that the passed consequent IPs has no space between the , delimiter.

Now that’s all my task is completed. All computerse inside the Network 192.168.1.1-255 on the Linux router freely can access the Internet, all IPs are also capable to access the IIS server located behind the NAT as well as only certain IPs are capable of accessing to the IIS remote desktop.
Hope the article helps somebody 😉

How to set up Path to .exe GNUWin32 binary files in Windows XP / Vista / 2003 / 2008 (Setting PATH to executables on Windows)

Tuesday, August 23rd, 2011

I’ve been working on a servers running Windows 2003 and Windows 2008 these days.
As I wanted to be more flexible on what I can do from the command line I decided to install GNUwin (provides port of GNU tools), most of which are common part of any Linux distribution).
Having most of the command line flexibility on a Windows server is a great thing, so I would strongly recommend GNUWin to any Windows server adminsitrator out there.

Actually it’s a wonderful thing that most of the popular Linux tools can easily be installed and used on Windows for more check GnuWin32 on sourceforge

One of the reasons I installed Gnuwin was my intention to use the good old Linux tail command to keep an eye interactive on the IIS server access log files, which by the way for IIS webserver are stored by default in C:WindowsSystem32LogFilesW3SVC1*.log

I’ve managed to install the GNUWin following the install instructions, not with too much difficulties. The install takes a bit of time, cause many packs containing different parts of the GNUWin has to be fetched.

To install I downloaded the GNUWin installer available from GNUWin32’s website and instructed to extracted the files into C:Program FilesGnuwin
Then I followed the install instructions suggestions, e.g.:

C:> cd c:Program FilesGnuWin
C:Program FilesGnuWin> download.bat
...
C:Program FilesGnuWin> install c:gnuwin32
...

After the installation was succesfully completed on the two Windows machines, both of which by the way are running 64 bit Windows, it was necessery to add the newly installed GNU .exe files to my regular cmd.exe PATH variable in order to be able to access the sed, tail and the rest of the gnuwin32 command line tools.

In order to add C:GnuWin32bin directory to the windows defined Command line Path , I had to do the following:

a. Select (Properties) for My Computer

Start (button) -> My Computer (choose properties)

b. Select the My Computer Advanced (tab)

Then, from the My Computer pane press on Advanced tab

c. Next press on Environment Variables

Windows environment variables screenshot

You see in above’s screenshot the Environment Variables config dialog, to add the new path location in System Variables sectiom, between the list I had to add the c:GNUwin32bin path locatiion. To find I pressed on Edit button scrolled down to find the Variable and hence added at the end of the long list defined paths.
After adding in GNUwin, the Windows path looks like this:

C:Program Files (x86)EWANAPI;C:WINDOWSsystem32;C:WINDOWS;C:WINDOWSSystem32Wbem;C:Program Files (x86)IntelNGSMSMPFiles;C:Program Files (x86)Microsoft SQL Server100ToolsBinn;C:Program FilesMicrosoft SQL Server100ToolsBinn;C:Program FilesMicrosoft SQL Server100DTSBinn;C:Program Files (x86)Microsoft SQL Server100ToolsBinnVSShellCommon7IDE;C:Program Files (x86)Microsoft Visual Studio 9.0Common7IDEPrivateAssemblies;C:Program Files (x86)Microsoft SQL Server100DTSBinn;C:WINDOWSsystem32WindowsPowerShellv1.0;C:gnuwin32bin

Further on, I launched the tail command to intercatively take an eye on who is accessing the IIS webserver.
Sadly this worked not, trying to use tail with the IIS ex10116.log log;

C:WindowsSystem32LogfilesW3SVC1> tail -f ex10116.log

Spit an error tail: ex10116.log: Bad file descriptor

Since I couldn’t use tail -f I looked for alternative and a quick search led me to Tail 4 Win32 . Lest the name suggests it is supposed to work on 32 bit arch Windows the version on tailforwin32’s website is working perfectly fine on 64 bit Windows as well.
What it does is to simulate a normal tail -f command inside a very simplistic window interface. You see it in action with opened IIS log on below’s screenshot:

GUI Tail for Windows screenshot

Finally my goal is achieved and I can take an eye interactively on IIS logs. End of the article, hope it wasn’t too boring 😉