Posts Tagged ‘Clamav’

How to check about infected files in clamav log files

Tuesday, October 30th, 2012

How to check about infected files in clamav log files

I've just run clamav with low priority to check the whole drive of a server for infected files Phpshells and other unwanted script kiddie tools. This was part of my check up if the server is compromised, after yesterday's unexpected cracker break in one of our company servers

# nice -n 19 clamav -r /* -l /var/log/clamav-scan.log

This exact server has about 100 Gigabytes of data all contained on one hard disk partition;, thus check up of all files took a few hours. clamav is relatively slow, compared to DrWeb or nod32. But since I'm not in a hurry plus, we can't afford to spend some extra money to buy AV just for one scan I left it scanning in a separate screen sesion.

clamscan execution put some extra load on the server (which btw is used mainly for processing a multitude of SQL queries and provides some HTTP access to few websites via Apache server. After the scan was completed I ended up with enormous very clamav log file, listing all scanned files:

I checked the file content in vim, but as reviewing 119MB of log line by one! – is unthinkable task, e.g.:

debian:~# du -hsc /var/log/clamav_scan.log
119M /var/log/clamav_scan.log
119M total

I did quick review of clamav_scan.log and tailing it displays me::

# tail -n 10 /var/log/clamav_scan.log
----------- SCAN SUMMARY -----------
Known viruses: 1270572
Engine version: 0.97.3
Scanned directories: 18927
Scanned files: 221445
Infected files: 44
Total errors: 287
Data scanned: 12457.43 MB
Data read: 97007.10 MB (ratio 0.13:1)
Time: 1842.362 sec (30 m 42 s) 

Thus I needed a way to not read screen by screen all by screen to see what was detected as Infected Files, but just show only infected files found by clamav.

I didn't know how this done, so did a quick search in Google and found the question how to only grep infected files from clamav.log  answered in Clamav-Users Mailing List read whole thread here

The thread suggests using:

[root@mail clamav]# cat clamd.log | grep -i "found"

Since cat-ing the log is worthless however it is much better to only do grep "found"  clamd.log or as in my case file is clamav_scan.log do:

# grep -i 'found' /var/log/clamav_scan.log

/usr/share/clamav-testfiles/clam.bz2.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.d64.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ppt: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.tnef: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-aspack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.rtf: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.7z: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_IScab_ext.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.odc.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.newc.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.pdf: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-wwpack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ole.doc: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.cab: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-mew.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-petite.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.sis: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-fsg.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_cache_emax.tgz: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.bz2: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_ISmsi_int.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.szdd: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.chm: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.arj: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_IScab_int.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ea05.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.tar.gz: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.html: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.binhex: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.impl.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-upack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bin-be.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.mail: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.mbox.uu: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-nsis.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_ISmsi_ext.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-yc.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bin-le.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-upx.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-pespin.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.mbox.base64: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ea06.exe: ClamAV-Test-File FOUND
 

Surprisingly all the "Infected" files turned to be a regular clamav scan (virus, spyware badware testfiles – i.e. clamav just use this file to check its database definitions works okay). Thus the supposingly  Infected files: 44 turned to be just another false positive.

Actually this grepping and logging of all scanned files, nevertheless they're not infected is completely useless. Thus it would have been much better if instead have run clamscan with cmd options:

debian:~# clamscan -r /* --infected

I hope ppl reading this article wouldn't repeat my "mistake".
In mean time after this thing here, maybe it will be a good idea to schedule 2 weeks or 1 months period clamscan of whole file system to make sure someone doesn't uploaded some malicious PHPShell script, exploit or other unwanted stuff.

My First Blog Entry

Sunday, November 5th, 2006

Today we first met Habib (a colleague from Holland College I study in). We took a newspaper and searched for him a living place. We called from one bulphone to some of the newspaper advertisements but was not able to find any suitable living place for the price he could afford (60 or 70 leva). After that we went to a coffee place called “Central”. Then we went to Mino’s coffee and stayed and talked for 3 or 4 hours with Habib mainly about Bangladej.He told us about different specific things in Bangladej, like for example he explained us the attitude of people to a local people who are musicians (I forgot the exact Bangla’s name of ‘em), he told us Bangla’s people will probably think of us we’re musicians if we go to Bangladej. He said the people appreciate this people as fortune and the musicians live on the hospitality of the people. I went home checked something on the servers. It seems there is a new RC release of clamav 0.90rc2. I installed the new release and tested it but it seemed it has some problems with the clamd.conf’s syntax and I’ve uninstalled the 0.90rc2 and installed the old one which is 0.88.5. Praise God I’m spiritually okay. I’m going to have a sleep now but I’ll first read a little (The Bible). Blessings in the name of Jesus Christ 🙂 !

Back on the Road Again

Saturday, February 17th, 2007

Yesterday I’ve went through a depression. The depression is trying to hunt me periodically. Right now I feel soso.So Praise the Lord. Yesterday I found out I passed the Accounting exam this was a great deal (thanks LORD).Today since it’s Saturday I hoped I will be able to sleep until 2, 3 o’clock unfortunately Tisho came andwake me up. Tisho is a friend I borrowed some money a week and a half ago. I can’t communicate normally with thisguy I don’t know why … We went to Mitko to take him for a coffee. e was playing Mugen ( A StreetFighter likecool fighting game). We stayed their for 30 minutes probably and then went to the fountain for a coffee.It was boring as a whole we smoked some cigaretes then everybody went to his home. When I cameback I upgraded the PHP modules on two of the boxes of DBG and right now I upgraded the clamav release.Meanwhile My boss make some interviews for a new server admin in the firm. One friend Joro went on the Interviewand the Boss approved him for the Job. What I am wandering and feeling about is, will this be me substitutionin the firm or this will be just another local administrator for the office. Actually I hope this thing wouldhappen that God has thought about me. I wait for the Lord to direct my way to somewhere since I’m in theocean as and no Land can be seen, just like I dreamed in one of my dreams before few months.In 5 o’clock I’ll meet Joro to speak about him how his interview in DBG went and possibly what will behis exam. ( I’ll be the examiner :]). Static come home yesterday and told me the dream he had the previousnight. He dreamed the Lord Jesus Christ. Here is the dream.Our saviour was standing on a platform of a sort of cloudsand a lot of people were around him. He was all in light and everything around was light,there were 4 pillars ending in a sort of a sky. Jesus was all in white (white more whiter than any light in the world we can observe), he was with long White beard and Long white hair.Then my friend who dreamed this dream just thought in his mind “Oh Lord that’s you.”Then the Lord Jesus Christ pointed at him and he started ascending into the skies, full of very deep joy.After that he just woke up and prayed. Well Praise the Lord for giving to one of my brothers such a wonderfuldream. I hope his mercy will guard us all. Let Christ’s peace be in our hearts. AmenEND—–

End of Management Games / Lectures

Wednesday, April 16th, 2008

Today was the last day of the lectures with Joop Vinke. Here is how my day passed. I woke up at 9:00, washed my teeth dressed combed my hair and went to the police station to look for our quarter police officer. I need to renew my personal ID card because it has been expired already for 4 years already. Thanks God everything went smoothly in the police station. After that I went to school we had lectures with Joop Vinke. After the school I went home and played around with my FreeBSD system. I succesfully upgraded gnome 2.20 to 2.22.

Using the binary packages from

http://www.marcuscom.com/tb/packages/7-STABLE-FreeBSD/gnome/. I’m trying to upgrade gnome from source for already almost 3 weeks with portmanager. After all of the required ports rebuilded still gnome wasn’t functioning, so in order to make it working I downloaded packages from http://www.marcuscom.com/tb/packages/7-STABLE-FreeBSD/gnome/ and ran a little loop with

for i in *; do pkg_add -vf $i; done

 

to make all the .tbz files install I did that yesterday night today at the afternoon everything was installed and gnome ran just fine I only had to link few libraries because they were searched on a different places. All works just fine now I only have to rebuild few of my games because they’re now linked to an old libraries. In the evening we went out with Javor for a coffee. As very often we went to the fountain we had a nice talk and then we went to his apartment to watch a film. He recommended a film called 1984 and we watched that. My impression is that this film is totally psychodelic and freaky, but still interesting to see. After I went home I went to see my grandma and now I’m home tired on a few steps of my bed 🙂 I should also mention that today I upgraded clamav on 3 of the servers I maintain. It seems there are few configuration options which changed in the new clamav release (0.93). It was an easy day as a whole if we don’t count my physical infirmity.

END—–

Cause and solution for Qmail sent error “Requested action aborted: error in processing Server replied: 451 qq temporary problem (#4.3.0)”

Friday, October 28th, 2011

One of the qmail servers I manage today has started returning strange errors in Squirrel webmail and via POP3/IMAP connections with Thunderbird.

What was rather strange is if the email doesn’t contain a link to a webpage or and attachment, e.g. mail consists of just plain text the mail was sent properly, if not however it failed to sent with an error message of:

Requested action aborted: error in processing Server replied: 451 qq temporary problem (#4.3.0)

After looking up in the logs and some quick search in Google, I come across some online threads reporting that the whole issues are caused by malfunction of the qmail-scanner.pl (script checking mail for viruses).

After a close examination on what is happening I found out /usr/sbin/clamd was not running at all?!
Then I remembered a bit earlier I applied some updates on the server with apt-get update && apt-get upgrade , some of the packages which were updated were exactly clamav-daemon and clamav-freshclam .
Hence, the reason for the error:

451 qq temporary problem (#4.3.0)

was pretty obvious qmail-scanner.pl which is using the clamd daemon to check incoming and outgoing mail for viruses failed to respond, so any mail which contained any content which needed to go through clamd for a check and returned back to qmail-scanner.pl did not make it and therefore qmail returned the weird error message.
Apparently for some reason apparently the earlier update of clamav-daemon failed to properly restart, the init script /etc/init.d/clamav-daemon .

Following fix was very simple all I had to do is launch clamav-daemon again:

linux:~# /etc/inid.d/clamav-daemon restart

Afterwards the error is gone and all mails worked just fine 😉

Scanning shared hosting servers to catch abusers, unwanted files, phishers, spammers and script kiddies with clamav

Friday, August 12th, 2011

Clamav scanning shared hosting servers to catch abusers, phishers, spammers, script kiddies etc. logo

I’m responsible for some GNU/Linux servers which are shared hosting and therefore contain plenty of user accounts.
Every now and then our company servers gets suspended because of a Phishing websites, Spammers script kiddies and all the kind of abusers one can think of.

To mitigate the impact of the server existing unwanted users activities, I decided to use the Clamav Antivirus – open source virus scanner to look up for potentially dangerous files, stored Viruses, Spammer mailer scripts, kernel exploits etc.

The Hosting servers are running latest CentOS 5.5. Linux and fortunately CentOS is equipped with an RPM pre-packaged latest Clamav release which of the time of writting is ver. (0.97.2).

Installing Clamav on CentOS is a piece of cake and it comes to issuing:

[root@centos:/root]# yum -y install clamav
...

After the install is completed, I’ve used freshclam to update clamav virus definitions

[root@centos:/root]# freshclam
ClamAV update process started at Fri Aug 12 13:19:32 2011
main.cvd is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven)
WARNING: getfile: daily-13357.cdiff not found on remote server (IP: 81.91.100.173)
WARNING: getpatch: Can't download daily-13357.cdiff from db.gb.clamav.net
WARNING: getfile: daily-13357.cdiff not found on remote server (IP: 163.1.3.8)
WARNING: getpatch: Can't download daily-13357.cdiff from db.gb.clamav.net
WARNING: getfile: daily-13357.cdiff not found on remote server (IP: 193.1.193.64)
WARNING: getpatch: Can't download daily-13357.cdiff from db.gb.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
Downloading daily.cvd [100%]
daily.cvd updated (version: 13431, sigs: 173670, f-level: 60, builder: arnaud)
Downloading bytecode.cvd [100%]
bytecode.cvd updated (version: 144, sigs: 41, f-level: 60, builder: edwin)
Database updated (1019925 signatures) from db.gb.clamav.net (IP: 217.135.32.99)

In my case the shared hosting hosted websites and FTP user files are stored in /home directory thus I further used clamscan in the following way to check report and log into file the scan results for our company hosted user content.

[root@centos:/root]# screen clamscan -r -i --heuristic-scan-precedence=yes --phishing-scan-urls=yes --phishing-cloak=yes --phishing-ssl=yes --scan-archive=no /home/ -l /var/log/clamscan.log
home/user1/mail/new/1313103706.H805502P12513.hosting,S=14295: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/new/1313111001.H714629P29084.hosting,S=14260: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/new/1305115464.H192447P14802.hosting,S=22663: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/new/1311076363.H967421P17372.hosting,S=13114: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/domain.com/infos/cur/859.hosting,S=8283:2,S: Heuristics.Phishing.Email.SSL-Spoof FOUND/home/user1/mail/domain.com/infos/cur/131.hosting,S=6935:2,S: Heuristics.Phishing.Email.SSL-Spoof FOUND

I prefer running the clamscan in a screen session, because it’s handier, if for example my ssh connection dies the screen session will preserve the clamscan cmd execution and I can attach later on to see how scan went.

clamscan of course is slower as it does not use Clamav antivirus daemon clamd , however I prefer running it without running the daemon, as having a permanently running clamd on the servers sometimes creates problems or hangs and it’s not really worthy to have it running since I’m intending to do a clamscan no more than once per month to see some potential users which might need to be suspended.

Also later on, after it finishes all possible problems are logged to /var/log/clamscan.log , so I can read the file report any time.

A good idea might also be to implement the above clamscan to be conducted, once per month via a cron job, though I’m still in doubt if it’s better to run it manually once per month to search for the malicious users content or it’s better to run it via cron schedule.

One possible pitfall with automating the above clamscan /home virus check up, might be the increased load it puts to the system. In some cases the Webserver and SQL server might be under a heavy load at the exactly same time the clamscan cron work is running, this might possible create severe issues for users websites, if it’s not monitored.
Thus I would probably go with running above clamscan manually each month and monitor the server performance.
However for people, who have “iron” system hardware and clamscan file scan is less likely to cause any issues, probably a cronjob would be fine. Here is sample cron job to run clamscan:

10 05 01 * * clamscan -r -i --heuristic-scan-precedence=yes --phishing-scan-urls=yes --phishing-cloak=yes --phishing-ssl=yes --scan-archive=no /home/ -l /var/log/clamscan.log >/dev/null 2>&1

I’m interested to hear if somebody already is using a clamscan to run on cron without issues, once I’m sure that running it on cron would not lead to server down-times, i’ll implement it via cron job.

Anyone having experience with running clamscan directory scan through crond? 🙂