Archive for April 20th, 2010

Migration of audit messages from snoopy to auditd

Tuesday, April 20th, 2010

his article may be out of date and may be deleted in the future.

This article explains the migration from the previous service "Snoopy" to "Auditd". Only commands that are executed as a user with root rights should be recorded here.

 

Uninstall/disable snoopy
 

Configuration of auditd

Files needed
Auditd start/stop script

/etc/init.d/auditd

Rules for monitoring by auditd

/etc/audit/audit.rules

Auditd plugin for syslog service

/etc/audisp/plugins.d/syslog.conf

Edit the /etc/audit/audit.rules file
Auditd can be specifically configured to capture and exclude messages. The following list is helpful for excluding certain event entries ("msgtype"):

* 1000 – 1099 are for commanding the audit system
* 1100 – 1199 user space trusted application messages
* 1200 – 1299 messages internal to the audit daemon
* 1300 – 1399 audit event messages
* 1400 – 1499 kernel SE Linux use
* 1500 – 1599 AppArmor events
* 1600 – 1699 kernel crypto events
* 1700 – 1799 kernel abnormal records
* 1800 – 1999 future kernel use (maybe integrity labels and related events)
* 2001 – 2099 unused (kernel)
* 2100 – 2199 user space anomaly records
* 2200 – 2299 user space actions taken in response to anomalies
* 2300 – 2399 user space generated LSPP events
* 2400 – 2499 user space crypto events
* 2500 – 2999 future user space (maybe integrity labels and related events)

Adding the rules

In order for auditd to record the desired events, rules must be defined.

List of rules set up
Below is a list and explanation of the rules set up:

-a exclude,always -F msgtype>=2400 -F msgtype<=2499
-a exclude,always -F msgtype=PATH
-a exclude,always -F msgtype=CWD
-a exclude,always -F msgtype=EOE
-a exit,always -F arch=b64 -F auid!=0 -F auid!=4294967295 -S execve
-a exit,always -F arch=b32 -F auid!=0 -F auid!=4294967295 -S execve

The first rule excludes crypto events in user space – these include, for example, messages about a user logging in.
The second through fourth rules remove the information not necessary for monitoring before it is logged.
The fifth and sixth rules capture the commands entered by users moving within an interactive shell. Services etc. executed by the system are therefore not recorded.
It should be noted here that a separate rule must be created for systems that contain both 32- and 64-bit commands and libraries.

Rule syntax

In general, it makes sense to keep the number of existing rules low in order to reduce the load. Therefore, if possible, several rule fields (-F option) should be combined in one rule. Since Auditd obviously has a problem with multiple event entries that are defined in plain text, these have been created in individual rules. The syntax description of the individual rules is given in the next listing:

-a contains the instructions
The action value "exclude" and the list value "always" are specified for rules that should not lead to any log entry
The action values ​​"exit" and "always" have been specified for rules that should lead to a log entry
"exit" stands for a log entry after the command has been executed
-F defines a rules field
Depending on the application, the rules defined here filter by event entry ("msgtype"), architecture ("arch") and login UID ("auid").
-S stands for the syscall. In the rules that should lead to a log entry, the value "execve" is monitored – i.e. when commands are executed.

Redirect to syslog

Within the file /etc/audisp/plugins.d/syslog.conf the value

active = no
on

active = yes
set.

restart auditd with the command

/etc/init.d/auditd restart
the settings are accepted.

Additional information

The following man pages can be consulted for more information:

auditctl
audit.rules
auditd
auditd.conf

Installing qmailanalog and Generating daily qmail statistics with (qmail-stats.py) on Debian Lenny / Daily qmail statististics notification via email

Tuesday, April 20th, 2010

Finding a decent software to generate daily qmail statistics is a really tough job this days. Before time I always used eitherqmailalizer or isoqlog.
Presently qmailalizer is completely abondoned piece of software and I cannot force it to work on 64 bit architecture.Isoqlog is another story, it’s supposed to work with qmailrocks, however my qmail installation is based on Bill’s Linux Qmail Toaster and for some strange reason it’s generating empty statistics. It could be that isoqlog is not generating statistics because the log files’s feed to be processed is not enough. Anyways still I cannot figure out the reason why I cannot make work Isoqlog with the Qmail Toaster.

I needed a way to however at least have an overview statistics of what is happening inside qmail. Of course qmailmrtg which is explained how to be installed in my previous post is providing with some overall information, though the information acquired through it is too general.
I’ve spend some enormous time searching for something that could inform me on various qmail statistics based on the qmail logs, before I could find and tweak the qmail-stats.py report script to become usable with qmailanalog

In the meantime It was necessery for me to investigate into qmailanalog and install it on the Debian system.
Initially I instlaled the qmailanalog from source, latest current source release can be obtained via D.J. Bernstein’s qmaialanlog download page
You won’t be able to compile the qmailanalog piece of code in debian until substitute in the source file: error.h the line:

extern int errno;

with#include <errno.h>

After the above change your source should succesfully compile.
Right after I compile it I realized there is a debian source package installer called:
qmailanalog-installer

So on Debian to install qmailanalog all I had to do was:

debian-server:~# apt-get install qmailanalog-installer
debian-server:~# /usr/bin/build-qmailanalog

Now as I already have qmailanalog properly installed on Debian I decided to test it with a script called qmail-logs.sh
You can download the qmail-logs.sh script from here

Here I quote what exactly is written in the qmail-logs.sh header in order to provide you with a general idea what the script does.

## Purpose:
## Wrapper for qmailanalog scripts. Will analyze qmail multilog
## files for deferrals, failures, overall statistics, or convert
## them to sendmail-style logs.

After executing the script I realized the script is not working properly because of some errors issued by scripts included within the qmailanalog package.
The faced problems and their solution with the qmailanalog: zsenders, zsuccesses, zfailures, zrecipients, zfailures and zrhosts scripts I’ve reported as a Debian bugs The qmailanalog bug report I’ve submitted can be seen here

The whole problem with qmailanalog scripts on Debian is that the “gnu more” binary arguments passed during the script operations are not correct and needs to be fixed.
Similar issues and their solution is explained in Qmailrocks Forums threads

After fixing the issues with qmailanalog I tried once again the qmail-logs.sh script, this time some of the functions provided by the script prooved working however some of them weren’t okay still.
Therefore I took some more time searching on the internet and I’ve found on a mailing list the qmail-stats.py script which worked like a charm with a minor modifications.

In order to have the qmail-stats.py working you need the tai64nfrac binary.You can download the latest current version of tai64nfrac on tai64nfrac’s website
Installation of tai64nfrac is pretty straight forward and comes to the following:

debian-server:~# wget http://archives.eyrie.org/software/system/tai64nfrac-1.4.tar.gz
debian-server:~# tar -zxvf tai64nfrac-1.4.tar.gz
debian-server:~# cd tai64nfrac-1.4
debian-server:~# make
debian-esrver:~# make install

After the install you should have the tai64nfrac in /usr/local/bin/tai64nfrac
Now let’s go back to the qmail-stats.py script. I’ve mirrored the qmail-stats.py script the qmail-stats.py script can be downloaded here

The script reports statistics on Qmail Logs following criterias:

Overall Email Server Statistics
Failure Statistics (Reasons for Failure)
Deferrals Statistics (Reasons for Deferrals)
Top Ten Senders Statistics
Top Ten Recipients Statistics

To make the script working all you have to edit in the script is the LOGFILE_PATH the rest is preconfigured by me already in the version of qmail-stats.py which is provided for download above.
If you choose to compile the qmailanalog from source you might also need to change the CMDS options, which includes the directory locations and commands from qmailanalog.

Now if you want to have the reports generated from qmail-stats.py, you have to setup the qmail-stats.py to be running via the cron daemon.

To do so open your root crontab and put in it:

# report daily qmail statistics05 01 * * * /usr/local/bin/qmail-stats.py | mail -s "Qmail Daily Statistics for $(date)" admin@domain.com

Now you should have the qmail-stats.py mailed to you every day at 01:05 early in the morning! 🙂