Archive for the ‘Various’ Category

Collecting all website contact e-mails on Windows – Contact Email Extractor

Thursday, June 13th, 2013

For second week, I'm partially working in E-marketing agency. Its an education institution, so they need to harvest e-mails for possible contacts to send them advertisements about University Scholarships, offered Study Streams in university, events Uni corriculum etc. Hence I did a quick research to see if there is program capable to extract all e-mails from website. I doubt there is any for Linux and as mostly all here are working on Windows I researched for domain e-mail harvester program that runs on Windows. Of course some websites, do put their contact e-mails in picture format to prevent spammers to collect e-mails, however bigger percentage on the net doesn't, so such program does a great job :)
Quick research in Sourceforce revealed few nice softwares. First I one tried Contact e-mail Extractor done perfectly extracting all e-mails on domain I typed, below is screenshot of Contact E-mail Extractor

Extract Collect all emails from website email extractor on Windows

Surely program will be of interest to anyone, somehow involved in E-marketing or SPAM :)
Probably, there are scripts to extract all emails after crawling a website on Linux too and when I have little more time its interesting to research on that too. If someone knows of something to extract e-mails from site on Linux please drop a comment.

Share this on

Converting jpeg / png pictures to mp4 format videos on Linux with avconv

Wednesday, June 12th, 2013

making mp4 videos from jpeg and png pictures gnu linux
Here in my job, there is one internet (TV) around which can only read video formats in Mp4. Henceforth I needed a way to convert a number of JPEG / PNG format Picture files to Mp4. In my situation I needed to make video out of pictures and script it as this was required to be later pushed via FTP protocol to the Internet TV. Ater doing some research I figured out this is possible to build video out of pictures by using together ImageMagick and avconv.

First thing is to send pictures in certain resolution. There is a tool from imagemagick package called mogrify (which can do that). To convert a number of different dimension pictures to lets say 1024×768, used:

noah:~# mogrify -resize 1024×768 *.png

In earlier Linux distributions to create a movie from pictures ffmpeg was used like so:

# ffmpeg -qscale 5 -r 20 -b 9600 -i SDC%04d.png movie.mp4

However in newer version of ffmpeg support for this is removed, and to make video from images, there is other tool avconv. On most Linux distributions avconv is part of libav-tools package.

To install on avconv Debian and Ubuntu;
noah:~# apt-get install –yes libav-tools

Once installed to create single mp4 video from JPEG or PNG pictures:

noah:~# avconv -i SDC106001.JPG SDC10595.JPG SDC13611.JPG SDC13612.JPG SDC13614.JPG  movie-from-pictures.mp4

I found there is also a nice GUI,software Open Movie Editor, which can create video out of number of pictuers stored in a directory. 
Open Movie Editor is available in most Linux distributions (unfortunately as of time of writting on Debian Wheeze there is no install candidate of openmovieeditor), there is however substitute package doing the same job called openshot);
On latest Debian stable to install Openshot:

noah:~# apt-get install –yes openshot

openshot running on Debian Wheezy Linux create video from pictures

Openshot has an option (Import Image Sequence) from File -> Import Image Sequence. Using this option you can select a directory with fles with common prefix name lets say IMG*.png and create Video based on the photos inside.

Linux create  video from images import image sequence Debian / Fedora and Ubuntu

On Ubuntu Linux or other distro where openmovieeditor is avialable install it with:

ubuntu:~# apt-get install –yes openmovieeditor

Openmovieeditor Linux create movies from images screenshot
To create a video out of pictures click on Media Browser tab. Choose the Folder with pictures from which video will be generated and drag them to the Video bar
(window).

To save produced video navigate to menus:

Project -> Render

and select format to save it as. One good codec to save output is Quicktime's. To convert later Quicktime MOV Video to MP4 video,
there is the  Pytube Media Converter script.

Well that's all enjoy your new videos from pictures :)

Share this on

Checking port security on Linux with Nmap – Just another Nmap examples tutorial

Sunday, June 9th, 2013

Scanning with nmap checking computer network security Linux FreeBSD Windows Nmap logo
Nmap
(Network Mapper) is one of the most essential tools for checking server security. As a penetration testing instrument it is both used by SysAdmins / Crackers and Security Specialists. Its perfect too to make periodic port audits and determine how good is configured server firewall or even in time of building one. Often with time Firewall rules grow bigger and bigger and as a consequence there is a risk of loopholes in FW rules, nmap routine host checks (i.e. run as a cronjob and logging port status on server is IMHO a good preventive measure).

I first get introduced to Nmap in the early days of my careers as IT Geek and System Administrator around year 2000. Back then Computer Security and hacking culture was a common thing across IT geeks and ppl hanging in IRC ;) This article will not say much of news for those accustomed to Nmap, but hope interesting for people newly introduced to Computer Security it will be of use.


1. Checking host status with Nmap (Is remote scanned host up).

There is plenty of ways to check, whether remote host is reachable, ping is classics, but not always relevant as many network admins decide to filter ping for security reasons. Of course one can do manual try outs with telnet on common Services Ports (Apache, Mail, Squid, MySQL etc. / 80,25,8080, 3306), or even write on own prog to do so but its worthless as Nmap is already there with options for this and its report in about 90% of cases is relevant:

To check whether host is up with Nmap:

pcfreak:~# nmap -sP google.com

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 11:58 EEST
Nmap scan report for google.com (173.194.39.227)
Host is up (0.013s latency).
Other addresses for google.com (not scanned): 173.194.39.238 173.194.39.231 173.194.39.226 173.194.39.232 173.194.39.230 173.194.39.233 173.194.39.228 173.194.39.225 173.194.39.229 173.194.39.224
rDNS record for 173.194.39.227: sof01s02-in-f3.1e100.net
Nmap done: 1 IP address (1 host up) scanned in 0.74 seconds

2. Port map with Quick remote host (connect) scan

Most classical way of scanning, since the early days of computing is to  attempt connecting to remote host ports opening connection via creating new TCP or UDP protocol socket with C's connect(); function. Hence nmap's "default" way of scanning is like so. Anyways it doesn't scan all possible 65534 ports, when run with no extra arguments, but instead scans only those more popular widespread used.

noah:~# nmap -sT pc-freak.net

 

Starting Nmap 5.00 ( http://nmap.org ) at 2013-06-08 15:05 EEST
Stats: 0:00:01 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 2.00% done; ETC: 15:07 (0:01:38 remaining)
Stats: 0:00:02 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 24.40% done; ETC: 15:05 (0:00:09 remaining)
Stats: 0:00:03 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 77.25% done; ETC: 15:05 (0:00:01 remaining)
Interesting ports on pc-freak.net (83.228.93.76):
Not shown: 985 filtered ports
PORT     STATE  SERVICE
20/tcp   closed ftp-data
21/tcp   open   ftp
22/tcp   open   ssh
25/tcp   open   smtp
53/tcp   open   domain
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  closed https
465/tcp  open   smtps
631/tcp  closed ipp
993/tcp  open   imaps
995/tcp  closed pop3s
8022/tcp open   unknown
9001/tcp open   tor-orport

Nmap done: 1 IP address (1 host up) scanned in 4.69 seconds
 

During scan, pressing Enter, prints on screen statistics on how many percentage of scan is completed. In older Nmap, releases this was not so, it is very convenient stuff, as some host scans (with specific firewalls), can have anti port scan rules making the scan time ultra luggish. If this is the case nmap can be run in different scan mode, I'm gonna say few words on that later.

3. Nmap – Scanning only selected ports of interest and  port range

a) Scanning only desired ports
Whether scanning a complete range of IPs from C or B class network, it is handy to only scan only ports of interests for example (Apache, SMTP, POP3, IMAP etc.).
Here is how to scan those 4;

noah:~# nmap -sT pc-freak.net -p 80,25,110,143

 

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 15:49 EEST
Stats: 0:00:00 elapsed; 0 hosts completed (0 up), 1 undergoing Ping Scan
Ping Scan Timing: About 100.00% done; ETC: 15:49 (0:00:00 remaining)
Stats: 0:00:00 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 100.00% done; ETC: 15:49 (0:00:00 remaining)
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.20s latency).
PORT    STATE SERVICE
25/tcp  open  smtp
80/tcp  open  http
110/tcp open  pop3
143/tcp open  imap

Nmap done: 1 IP address (1 host up) scanned in 1.00 seconds

List of all common network services with port number is located in /etc/services

b) Scanning a port range

By default nmap does not scan all the ports in the low ports range 1-1024. This port range according to RFC standards are reserved for standard more often and high priority network services. Default's nmap scan does not scan all of the 1-1024 ports and sometimes, some people prefer to run services in non-standard port numbers on some obscure ports in those port range. It is common that some "hacked (cracked is proper word here)", have secretly install Connect Shell or Connect back shell services running in those port range. Thus scanning those port range on administrated servers (especially whether there is suspicion for intrusion).

noah:~# nmap -sT pc-freak.net -p 1-1024

 

 

Starting Nmap 5.00 ( http://nmap.org ) at 2013-06-08 15:47 EEST
Stats: 0:00:04 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 77.44% done; ETC: 15:47 (0:00:01 remaining)
Stats: 0:00:04 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 84.86% done; ETC: 15:47 (0:00:01 remaining)
Interesting ports on pc-freak.net (83.228.93.76):
Not shown: 1011 filtered ports
PORT    STATE  SERVICE
20/tcp  closed ftp-data
21/tcp  open   ftp
22/tcp  open   ssh
25/tcp  open   smtp
53/tcp  open   domain
80/tcp  open   http
110/tcp open   pop3
143/tcp open   imap
443/tcp closed https
465/tcp open   smtps
631/tcp closed ipp
993/tcp open   imaps
995/tcp closed pop3s

4. Scanning all possible ports to make complete node port audit

As I said prior, if no extra port arguments nmap scans only number of pre-selected high use ports. However it is always nice to run complete port scan. Doing complete port scan on host, can reveal unusual open ports for cracker backdoors or ports or whether on Windows (ports open by Viruses and Trojans). As the complete number of possible remote ports to attempt to connect to is (65536), such a scan is much slower and sometimes can take literally "ages". To scan all ports on my home router in a local 100 M/Bit network with my notebook it takes about 23 minutes. On remote hosts it can take from 30 / 40 minutes to many hours – depending on firewall type on remote scanned host. Also by scanning all ports, there is risk remote host add you to its FW reject rules, whether its running some kind of automated software for Intrusion Detection (IDS) like Snort or AIDE.
To run complete port scan with nmap;

noah:~# nmap -sT pc-freak.net -p 0-65535
 

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 22:28 EEST
Stats: 0:00:01 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 0.03% done
Stats: 0:00:01 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 0.05% done
Stats: 0:06:35 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 31.23% done; ETC: 22:50 (0:14:28 remaining)
Stats: 0:06:35 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 31.24% done; ETC: 22:50 (0:14:27 remaining)
Stats: 0:08:21 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 37.41% done; ETC: 22:51 (0:13:57 remaining)
Stats: 0:08:21 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 37.43% done; ETC: 22:51 (0:13:56 remaining)
Stats: 0:08:21 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 37.46% done; ETC: 22:51 (0:13:56 remaining)
Stats: 0:08:22 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 37.50% done; ETC: 22:51 (0:13:55 remaining)
Stats: 0:08:22 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 37.53% done; ETC: 22:51 (0:13:56 remaining)
Stats: 0:08:28 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 37.96% done; ETC: 22:51 (0:13:50 remaining)
Stats: 0:11:55 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 53.22% done; ETC: 22:51 (0:10:28 remaining)
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.0023s latency).
Not shown: 65518 filtered ports
PORT     STATE  SERVICE
20/tcp   closed ftp-data
21/tcp   open   ftp
22/tcp   open   ssh
25/tcp   open   smtp
53/tcp   open   domain
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  closed https
465/tcp  open   smtps
631/tcp  closed ipp
993/tcp  open   imaps
995/tcp  closed pop3s
2060/tcp open   unknown
2070/tcp open   ah-esp-encap
2207/tcp closed unknown
8022/tcp open   oa-system
9001/tcp open   tor-orport

Nmap done: 1 IP address (1 host up) scanned in 1367.73 seconds

5. Scanning a network range of IPs with NMAP

It is common thing to scan a network range in C class network, especially as usually we admins have to administrate a number of hosts running in a local network:

 

noah:~# nmap -sP '192.168.0.*'

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 22:29 EEST
Stats: 0:00:01 elapsed; 0 hosts completed (0 up), 256 undergoing Ping Scan
Ping Scan Timing: About 0.98% done
Stats: 0:00:09 elapsed; 0 hosts completed (0 up), 256 undergoing Ping Scan
Parallel DNS resolution of 256 hosts. Timing: About 0.00% done
Nmap scan report for 192.168.0.16
Host is up (0.00029s latency).
Nmap done: 256 IP addresses (1 host up) scanned in 9.87 seconds

You can also scan class C network with:

>noah:~# nmap -sP 192.168.1.0/24

6. Obtaining network services version numbers

Nmap is capable digging version numbers of remote running application binding to port:. Option to try to guess obtain version number is -sV (Show Version).

noah:~# nmap -sV pc-freak.net

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 22:35 EEST
Stats: 0:00:05 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Service scan Timing: About 90.91% done; ETC: 22:37 (0:00:09 remaining)
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.0083s latency).
Not shown: 985 filtered ports
PORT     STATE  SERVICE         VERSION
20/tcp   closed ftp-data
21/tcp   open   ftp             ProFTPD 1.3.3a
22/tcp   open   ssh             OpenSSH 5.5p1 Debian 6+squeeze3 (protocol 2.0)
25/tcp   open   smtp            qmail smtpd
53/tcp   open   domain?
80/tcp   open   http            Apache httpd
110/tcp  open   pop3            qmail pop3d
143/tcp  open   imap            Courier Imapd (released 2005)
443/tcp  closed https
465/tcp  open   ssl/smtp        qmail smtpd
631/tcp  closed ipp
993/tcp  open   tcpwrapped
995/tcp  closed pop3s
8022/tcp open   http            ShellInABox httpd
9001/tcp open   ssl/tor-orport?
Service Info: Host: mail.pc-freak.net; OSs: Unix, Linux; CPE: cpe:/o:linux:kernel

Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 126.37 seconds

 

7. Checking remote server OS version

 noah:~# nmap -O pc-freak.net

 

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 22:42 EEST
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.0017s latency).
Not shown: 985 filtered ports
PORT     STATE  SERVICE
20/tcp   closed ftp-data
21/tcp   open   ftp
22/tcp   open   ssh
25/tcp   open   smtp
53/tcp   open   domain
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  closed https
465/tcp  open   smtps
631/tcp  closed ipp
993/tcp  open   imaps
995/tcp  closed pop3s
8022/tcp open   oa-system
9001/tcp open   tor-orport
Device type: general purpose|broadband router|WAP|media device
Running (JUST GUESSING): Linux 2.6.X|2.4.X|3.X (94%), Gemtek embedded (89%), Siemens embedded (89%), Netgear embedded (88%), Western Digital embedded (88%), Comtrend embedded (88%)
OS CPE: cpe:/o:linux:kernel:2.6 cpe:/o:linux:kernel:2.4.20 cpe:/o:linux:kernel:3 cpe:/o:linux:kernel:2.4
Aggressive OS guesses: Linux 2.6.32 – 2.6.35 (94%), Vyatta 4.1.4 (Linux 2.6.24) (94%), Linux 2.6.32 (93%), Linux 2.6.17 – 2.6.36 (93%), Linux 2.6.19 – 2.6.35 (93%), Linux 2.6.30 (92%), Linux 2.6.35 (92%), Linux 2.4.20 (Red Hat 7.2) (92%), Linux 2.6.22 (91%), Gemtek P360 WAP or Siemens Gigaset SE515dsl wireless broadband router (89%)
No exact OS matches for host (test conditions non-ideal).

OS detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 8.76 seconds

As you can see from above output OS version guess is far from adequate, as my home router is running a Debian Squeeze. However in some older Linux releases, where services return OS version nr., it reports proper.

8. Scanning silently with Nmap SYN (Stealth Scan)

As many servers run some kind of IDS logging attempts to connect to multiple ports on the host and add scanning IP to filtering CHAIN. It is generally good idea to always scan with SYN Scan. SYN scan is not a guarantee that scanning attempt will not be captured by well configured IDS, or admin snorting on network with tcpdump,trafshow or iptraf. Stealth scan is useful to prevent IDS from raising red lamps.

noah:~# nmap -sS pc-freak.net

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-08 22:57 EEST
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.0075s latency).
Not shown: 985 filtered ports
PORT     STATE  SERVICE
20/tcp   closed ftp-data
21/tcp   open   ftp
22/tcp   open   ssh
25/tcp   open   smtp
53/tcp   open   domain
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  closed https
465/tcp  open   smtps
631/tcp  closed ipp
993/tcp  open   imaps
995/tcp  closed pop3s
8022/tcp open   oa-system
9001/tcp open   tor-orport

Nmap done: 1 IP address (1 host up) scanned in 7.73 seconds

 

9. Nmap Scan Types (Paranoid | sneaky | polite | normal | insane)

Nmap has 6 modes of scanning. Whether no Type of scan is passed on with (-T) arg. , it scans in normal mode. Paranoid and sneaky are the slowest but lest aggressive and less likely to be captured by automated firewall filtering rules soft or IDS.

Insane mode is for people, who want to scan as quickly as possible not caring about consequences. Usually whether scanning your own hosts Insane is nice as it saves you time.

Paranoid scan is ultra, slow so in general, such scan is helpful if you're going to sleep and you  want to scan your concurrent company servers, without being identified. Paraonid scan, takes hours and depending on where remote scanned host is located can sometimes take maybe 12 to 24 hours.
noah:~# nmap -T0 pc-freak.net

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-09 00:23 EEST
Stats: 0:15:00 elapsed; 0 hosts completed (1 up), 1 undergoing SYN Stealth Scan
SYN Stealth Scan Timing: About 0.05% done
Almost always -T3 or T4 is reasonable.

10. Scanning hosts in verbose mode

pcfreak:~# nmap -vv localhost

Starting Nmap 5.00 ( http://nmap.org ) at 2013-06-09 01:14 EEST
NSE: Loaded 0 scripts for scanning.
Initiating SYN Stealth Scan at 01:14
Scanning localhost (127.0.0.1) [1000 ports]
Discovered open port 21/tcp on 127.0.0.1
Discovered open port 111/tcp on 127.0.0.1
Discovered open port 22/tcp on 127.0.0.1
Discovered open port 53/tcp on 127.0.0.1
Discovered open port 993/tcp on 127.0.0.1
Discovered open port 143/tcp on 127.0.0.1
Discovered open port 110/tcp on 127.0.0.1
Discovered open port 80/tcp on 127.0.0.1
Discovered open port 3306/tcp on 127.0.0.1
Discovered open port 25/tcp on 127.0.0.1
Discovered open port 783/tcp on 127.0.0.1
Discovered open port 8022/tcp on 127.0.0.1
Discovered open port 9001/tcp on 127.0.0.1
Discovered open port 465/tcp on 127.0.0.1
Completed SYN Stealth Scan at 01:14, 0.09s elapsed (1000 total ports)
Host localhost (127.0.0.1) is up (0.0000070s latency).
Scanned at 2013-06-09 01:14:27 EEST for 1s
Interesting ports on localhost (127.0.0.1):
Not shown: 986 closed ports
PORT     STATE SERVICE
21/tcp   open  ftp
22/tcp   open  ssh
25/tcp   open  smtp
53/tcp   open  domain
80/tcp   open  http
110/tcp  open  pop3
111/tcp  open  rpcbind
143/tcp  open  imap
465/tcp  open  smtps
783/tcp  open  spamassassin
993/tcp  open  imaps
3306/tcp open  mysql
8022/tcp open  unknown
9001/tcp open  tor-orport

Read data files from: /usr/share/nmap
Nmap done: 1 IP address (1 host up) scanned in 0.21 seconds
           Raw packets sent: 1000 (44.000KB) | Rcvd: 2014 (84.616KB)

 

11. Nmap typical scan arguments combinations

noah:~# nmap -sS -P0 -sV pc-freak.net

Stats: 0:01:46 elapsed; 0 hosts completed (1 up), 1 undergoing Service Scan
Service scan Timing: About 90.91% done; ETC: 01:22 (0:00:10 remaining)
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.0063s latency).
Not shown: 985 filtered ports
PORT     STATE  SERVICE         VERSION
20/tcp   closed ftp-data
21/tcp   open   ftp             ProFTPD 1.3.3a
22/tcp   open   ssh             OpenSSH 5.5p1 Debian 6+squeeze3 (protocol 2.0)
25/tcp   open   smtp            qmail smtpd
53/tcp   open   domain?
80/tcp   open   http            Apache httpd
110/tcp  open   pop3            qmail pop3d
143/tcp  open   imap            Courier Imapd (released 2005)
443/tcp  closed https
465/tcp  open   ssl/smtp        qmail smtpd
631/tcp  closed ipp
993/tcp  open   tcpwrapped
995/tcp  closed pop3s
8022/tcp open   http            ShellInABox httpd
9001/tcp open   ssl/tor-orport?
Service Info: Host: mail.pc-freak.net; OSs: Unix, Linux; CPE: cpe:/o:linux:kernel

Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 106.23 seconds
 

12. Logging nmap output

Nmap can output logs in Plain Text (TXT) / GNMAP and XML. I prefer logging to TXT, as plain text is always better:
noah:~# nmap pc-freak.net -o nmap-log.txt

Starting Nmap 6.00 ( http://nmap.org ) at 2013-06-09 01:32 EEST
Stats: 0:00:01 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 4.60% done; ETC: 01:32 (0:00:21 remaining)
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.013s latency).
Not shown: 985 filtered ports
PORT     STATE  SERVICE
20/tcp   closed ftp-data
21/tcp   open   ftp
22/tcp   open   ssh
25/tcp   open   smtp
53/tcp   open   domain
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  closed https
465/tcp  open   smtps
631/tcp  closed ipp
993/tcp  open   imaps
995/tcp  closed pop3s
3306/tcp closed mysql
8022/tcp open   oa-system

Nmap done: 1 IP address (1 host up) scanned in 5.23 seconds

Below is also a paste from nmap man page (Example section) nmap -Pn -p80 -oX logs/pb-port80scan.xml -oG logs/pb-port80scan.gnmap 216.163.128.20/20

This scans 4096 IPs for any web servers (without pinging them) and saves the output in grepable and XML formats.

13. Other good Nmap scanning examples and arguments

One very useful Nmap option is;
-A – Enables OS detection and Version detection, Script scanning and Traceroute

Whether you have a list of all IPs administrated by you and you would like to scan all of them;

noah:~# nmap -iL /root/scan_ip_addresses.txt

Other useful option is -sA (This does TCP ACK Scan), it is useful way to determine if remote host is running some kind of stateful firewall. Instead of connecting to ports to check whether opened, ACKs are send.

- Fast port Scan

noah:~# nmap -F pc-freak.net
...

-D argument (Decoy scanning
Nmap has option for simulating port scan from multiple IPs, the so called Decoyed scanning. Using Decoys, one can hide real IP address from which Nmap scan is initiated

# nmap -n -D192.168.1.5,10.5.1.2,172.1.2.4,3.4.2.1 192.168.1.5

- Scan firewall for security weaknesses

(TCP Null Scan to full firewall to generate responce)
# nmap -sN 10.10.10.1

(TCP Fin scan to check firewall)

 # nmap -sF 10.10.10.1

(TCP Xmas scan to check firewall)

# nmap -sX 10.10.10.1

- Scan UDP ports

# nmap -sU hostname

- Scan remote host using IP (ping) Protocol

noah:~# nmap -P0 pc-freak.net

Connect Scan Timing: About 96.20% done; ETC: 23:16 (0:00:00 remaining)
Nmap scan report for pc-freak.net (83.228.93.76)
Host is up (0.0099s latency).
Not shown: 985 filtered ports
PORT     STATE  SERVICE
20/tcp   closed ftp-data
21/tcp   open   ftp
22/tcp   open   ssh
25/tcp   open   smtp
53/tcp   open   domain
80/tcp   open   http
110/tcp  open   pop3
143/tcp  open   imap
443/tcp  closed https
465/tcp  open   smtps
631/tcp  closed ipp
993/tcp  open   imaps
995/tcp  closed pop3s
8022/tcp open   oa-system
9001/tcp open   tor-orport

Nmap done: 1 IP address (1 host up) scanned in 4.97 seconds

 

Share this on

Fixing Clamav error: “WARNING: Can’t download daily.cvd from database.clamav.net”

Thursday, June 6th, 2013

On one of the Debian Squeeze Servers, where I have Running QMAIL Server, I haven't checked logs for a long time. Cause Qmail is configured and all runs smoothly. Just today while checking logs, I've noticed in /var/log/clamav/clamav.log, clamav database fails to be updated with an error, e.g.:

qmail:~# tail -n 28 /var/log/clamav/clamav.log

ClamAV update process started at Thu Jun 6 20:47:14 2013
main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven)
WARNING: getpatch: Can't download daily-16682.cdiff from db.local.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from db.local.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from db.local.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from db.local.clamav.net
ERROR: getpatch: Can't download daily-16682.cdiff from db.local.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
ERROR: Can't download daily.cvd from db.local.clamav.net
Giving up on db.local.clamav.net…
ClamAV update process started at Thu Jun 6 20:47:15 2013
main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven)
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
WARNING: Can't download daily.cvd from database.clamav.net
Trying again in 5 secs…
ClamAV update process started at Thu Jun 6 20:47:20 2013
main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven)
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
WARNING: Can't download daily.cvd from database.clamav.net

On host Freshclam is configured to run in background as a service i.e.:

qmail:~#
 ps ax|grep -i fresh|grep -v grep
13615 ? Ss 0:02 /usr/bin/freshclam -d –quiet
 

I stopped clamav and tried running it manually through its script: qmail:~# /etc/init.d/clamav-freshclam restart

The error was reoccuring, so I decided to kill it and try running freshclam manually:

qmail:~# kill -9 freshclam

qmail:~# freshclam
I got same error again:
 

Thu Jun 6 16:46:20 2013 -> ClamAV update process started at Thu Jun 6 16:46:20 2013 Thu Jun 6 16:46:20 2013 -> main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven) Thu Jun 6 16:46:20 2013 -> WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net Thu Jun 6 16:46:20 2013 -> WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net Thu Jun 6 16:46:20 2013 -> WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net Thu Jun 6 16:46:20 2013 -> WARNING: getpatch: Can't download daily-16682.cdiff from database.clamav.net Thu Jun 6 16:46:20 2013 -> ERROR: getpatch: Can't download daily-16682.cdiff from database.clamav.net Thu Jun 6 16:46:20 2013 -> WARNING: Incremental update failed, trying to download daily.cvd Thu Jun 6 16:46:20 2013 -> ERROR: Can't download daily.cvd from database.clamav.net Thu Jun 6 16:46:20 2013 -> Giving up on database.clamav.net… Thu Jun 6 16:46:20 2013 -> Update failed. Your network may be down or none of the mirrors listed in /etc/clamav/freshclam.conf is working. Check http://www.clamav.net/support/mirror-problem for possible reasons.

The solution was to delete clamav database filedaily.cvd and then run another freshclam Clamav DB virus update:

qmail:~# rm -f /var/lib/clamav/daily.cvd
qmail:~# freshclam
root@pcfreak:/etc/init.d# freshclam ClamAV update process started at Thu Jun 6 22:07:21 2013
main.cvd is up to date (version: 54, sigs: 1044387, f-level: 60, builder: sven)
Downloading daily.cvd [100%]
daily.cvd updated (version: 17309, sigs: 1302714, f-level: 63, builder: neo)
bytecode.cld is up to date (version: 214, sigs: 41, f-level: 63, builder: neo)
Database updated (2347142 signatures) from db.local.clamav.net (IP: 195.222.33.229)

Finally, to make freshclam work as daemon, restarted init script:

qmail:~# /etc/init.d/clamav-freshclam restart
[ ok ] Stopping ClamAV virus database updater: freshclam.
[ ok ] Starting ClamAV virus database updater: freshclam.

 

Share this on

Maldetect – Malware web content file scanner for GNU / Linux – Keep your hosting servers Malware clean

Tuesday, June 4th, 2013

Linux malware detect scan for malware from commandline / Fedora, CentOS, Debian, Ubuntu 

It is so common nowadays that Shared hosting clients upload PHP / Javascript / Ajax scripts carelessly downloaded from somewhere containing malicious features or infected by third party script kiddie tools which replicate themselves after succesfully exploit some common PHP or Perl vulnerability. I'm sure even as time of writing this post probably millions of old un-updated Hosting Linux servers are silent Malware hives.
Therefore For Shared Hosting server servers it is useful to know about the existence of Maldetect – Linux Malware scanner also known under the name LMD.

Linux Maldetect – in what it does is very similar to Windows good Spyware Detect and Clean tool Malware Bytes. LMD uses Spyware definition database collected from network edge Intrusion detection systems who caught Web bugs commonly exploited as well as from custom user submissions of Malicious stuff. Maldetect's database can easily be exported and plays well together with ClamAV antivirus. LMD is very precious and is one of the must have outfits for hosting admins, as its use allows you to determine succesful cracking before system is rootkited and you have to audit for Backdoors or Rookit with rkhunter and chkroot

1. Install Linux MalDetect

LMD is young project so it does not still have a package deb and rpm package builds. Installation is done from source;

debian:~# wget http://www.rfxn.com/downloads/maldetect-current.tar.gz
debian:~# tar -xzf maldetect-current.tar.gz
debian:~# cd maldetect-*
debian:~# ./install.sh

Linux Malware Detect v1.4.1
            (C) 2002-2013, R-fx Networks <proj@r-fx.org>
            (C) 2013, Ryan MacDonald <ryan@r-fx.org>
inotifywait (C) 2007, Rohan McGovern <rohan@mcgovern.id.au>
This program may be freely redistributed under the terms of the GNU GPL

installation completed to /usr/local/maldetect
config file: /usr/local/maldetect/conf.maldet
exec file: /usr/local/maldetect/maldet
exec link: /usr/local/sbin/maldet
exec link: /usr/local/sbin/lmd
cron.daily: /etc/cron.daily/maldet

maldet(3143): {sigup} performing signature update check…
maldet(3143): {sigup} local signature set is version 201205035915
maldet(3143): {sigup} new signature set (2013060217799) available
maldet(3143): {sigup} downloaded http://www.rfxn.com/downloads/md5.dat
maldet(3143): {sigup} downloaded http://www.rfxn.com/downloads/hex.dat
maldet(3143): {sigup} downloaded http://www.rfxn.com/downloads/rfxn.ndb
maldet(3143): {sigup} downloaded http://www.rfxn.com/downloads/rfxn.hdb
maldet(3143): {sigup} downloaded http://www.rfxn.com/downloads/maldet-clean.tgz
maldet(3143): {sigup} signature set update completed
maldet(3143): {sigup} 11509 signatures (9641 MD5 / 1868 HEX)

2. Maldetect configs and binaries

Config is default installed in -  /usr/local/maldetect/conf.maldet
Main executed binary is placed in -  /usr/local/maldetect/maldet
There is a cron skele file placed in /etc/cron.daily/maldet. Its useful to run maldet via cron to check all sites on server and get e-mail reports.

3. Keep maldet up2date

debian:~# maldet --update-ver

Linux Malware Detect v1.4.2
            (C) 2002-2013, R-fx Networks <proj@r-fx.org>
            (C) 2013, Ryan MacDonald <ryan@r-fx.org>
inotifywait (C) 2007, Rohan McGovern <rohan@mcgovern.id.au>
This program may be freely redistributed under the terms of the GNU GPL v2

maldet(3511): {update} checking for available updates...
maldet(3511): {update} hashing install files and checking against server...
maldet(3511): {update} latest version already installed.

4. Update Maldetect definitions manually

Maldetect Malware definitions are designed to auto-update via cron. For people who don't like to waste CPU time and scrape on HDD with cronjob;

debian:~# maldet --update

5. Configure LMD

Tune according to your needs in config (/usr/local/maldetect/conf.maldet)

maxfilesize="768k"
email_alert=1
email_subj="Attention Malware found! Check your server!"
email_addr="hipo@pc-freak.net"

6. Scanning for Malware manually

debian:~# maldetect -m /home,/var/,/sbin,/opt
....

Just for a close up below is a list of common 60 Malwares found on Hosting servers (taken from Maldetect Website);

base64.inject.unclassed     perl.ircbot.xscan
bin.dccserv.irsexxy         perl.mailer.yellsoft
bin.fakeproc.Xnuxer         perl.shell.cbLorD
bin.ircbot.nbot             perl.shell.cgitelnet
bin.ircbot.php3             php.cmdshell.c100
bin.ircbot.unclassed        php.cmdshell.c99
bin.pktflood.ABC123         php.cmdshell.cih
bin.pktflood.osf            php.cmdshell.egyspider
bin.trojan.linuxsmalli      php.cmdshell.fx29
c.ircbot.tsunami            php.cmdshell.ItsmYarD
exp.linux.rstb              php.cmdshell.Ketemu
exp.linux.unclassed         php.cmdshell.N3tshell
exp.setuid0.unclassed       php.cmdshell.r57
gzbase64.inject             php.cmdshell.unclassed
html.phishing.auc61         php.defash.buno
html.phishing.hsbc          php.exe.globals
perl.connback.DataCha0s     php.include.remote
perl.connback.N2            php.ircbot.InsideTeam
perl.cpanel.cpwrap          php.ircbot.lolwut
perl.ircbot.atrixteam       php.ircbot.sniper
perl.ircbot.bRuNo           php.ircbot.vj_denie
perl.ircbot.Clx             php.mailer.10hack
perl.ircbot.devil           php.mailer.bombam
perl.ircbot.fx29            php.mailer.PostMan
perl.ircbot.magnum          php.phishing.AliKay
perl.ircbot.oldwolf         php.phishing.mrbrain
perl.ircbot.putr4XtReme     php.phishing.ReZulT
perl.ircbot.rafflesia       php.pktflood.oey
perl.ircbot.UberCracker     php.shell.rc99
perl.ircbot.xdh             php.shell.shellcomm


 

Share this on

Debian Linux: Installing and monitoring servers with Icanga (Nagios fork soft)

Monday, June 3rd, 2013

icinga-monitoring-processes-and-servers-linux-logo

There is plenty of software for monitoring how server performs and whether servers are correctly up and running. There is probably no Debian Linux admin who didn't already worked or at least tried Nagios and Mointor to monitor and notify whether server is unreachable or how server services operate. Nagios and Munin are play well together to prevent possible upcoming problems with Web / Db / E-mail services or get notify whether they are completely inaccessible. One similar "next-generation" and less known software is Icanga.
The reason, why to use Icinga  instead of Nagios is  more features a list of what does Icinga supports more than Nagios is on its site here
I recently heard of it and decided to try it myself. To try Icanga I followed Icanga's install tutorial on Wiki.Icanga.Org here
In Debian Wheezy, Icinga is already part of official repositories so installing it like in Squeeze and Lenny does not require use of external Debian BackPorts repositories.

1. Install Icinga pre-requirement packages

debian:# apt-get --yes install php5 php5-cli php-pear php5-xmlrpc php5-xsl php5-gd php5-ldap php5-mysql

2. Install Icanga-web package

debian:~# apt-get --yes install icinga-web

Here you will be prompted a number of times to answer few dialog questions important for security, as well as fill in MySQL server root user / password as well as SQL password that will icinga_web mySQL user use.

icinga-choosing-database-type

configuring-icinga-web-debian-linux-configuring-database-shot

debian-config-screenshot-configuring-icinga-idoutils

icinga-password-confirmation-debian-linux
….

Setting up icinga-idoutils (1.7.1-6) …
dbconfig-common: writing config to /etc/dbconfig-common/icinga-idoutils.conf
granting access to database icinga for icinga-idoutils@localhost: success.
verifying access for icinga-idoutils@localhost: success.
creating database icinga: success.
verifying database icinga exists: success.
populating database via sql…  done.
dbconfig-common: flushing administrative password
Setting up icinga-web (1.7.1+dfsg2-6) …
dbconfig-common: writing config to /etc/dbconfig-common/icinga-web.conf

Creating config file /etc/dbconfig-common/icinga-web.conf with new version
granting access to database icinga_web for icinga_web@localhost: success.
verifying access for icinga_web@localhost: success.
creating database icinga_web: success.
verifying database icinga_web exists: success.
populating database via sql…  done.
dbconfig-common: flushing administrative password

Creating config file /etc/icinga-web/conf.d/database-web.xml with new version
database config successful: /etc/icinga-web/conf.d/database-web.xml

Creating config file /etc/icinga-web/conf.d/database-ido.xml with new version
database config successful: /etc/icinga-web/conf.d/database-ido.xml
enabling config for webserver apache2…
Enabling module rewrite.
To activate the new configuration, you need to run:
  service apache2 restart
`/etc/apache2/conf.d/icinga-web.conf' -> `../../icinga-web/apache2.conf'
[ ok ] Reloading web server config: apache2 not running.
root password updates successfully!
Basedir: /usr Cachedir: /var/cache/icinga-web
Cache already purged!

3. Enable Apache mod_rewrite
 

 

debian:~# a2enmod rewrite
debian:~# /etc/init.d/apache2 restart


4. Icinga documentation files

Some key hints on Enabling some more nice Icinga features are mentioned in Icinga README files, check out, all docs files included with Icinga separate packs are into:
 

debian:~# ls -ld *icinga*/
drwxr-xr-x 3 root root 4096 Jun  3 10:48 icinga-common/
drwxr-xr-x 3 root root 4096 Jun  3 10:48 icinga-core/
drwxr-xr-x 3 root root 4096 Jun  3 10:48 icinga-idoutils/
drwxr-xr-x 2 root root 4096 Jun  3 10:48 icinga-web/

debian:~# less /usr/share/doc/icinga-web/README.Debian debian:~# less /usr/share/doc/icinga-idoutils/README.Debian

5. Configuring Icinga

Icinga configurations are separated in two directories:

debian:~# ls -ld *icinga*

drwxr-xr-x 4 root root 4096 Jun  3 10:50 icinga
drwxr-xr-x 3 root root 4096 Jun  3 11:07 icinga-web

>

etc/icinga/ – (contains configurations files for on exact icinga backend server behavior)

 

/etc/icinga-web – (contains all kind of Icinga Apache configurations)
Main configuration worthy to look in after install is /etc/icinga/icinga.cfg.

6. Accessing newly installed Icinga via web

To access just installed Icinga, open in browser URL – htp://localhost/icinga-web

icinga web login screen in browser debian gnu linux

logged in inside Icinga / Icinga web view and control frontend

7. Monitoring host services with Icinga (NRPE)

As fork of Nagios. Icinga has similar modular architecture and uses number of external plugins to Monitor external host services list of existing plugins is on Icinga's wiki here.
Just like Nagios Icinga supports NRPE protocol (Nagios Remote Plugin Executor). To setup NRPE, nrpe plugin from nagios is used (nagios-nrpe-server). 

To install NRPE on any of the nodes to be tracked;
debian: ~# apt-get install –yes nagios-nrpe-server

 Then to configure NRPE edit /etc/nagios/nrpe_local.cfg


Once NRPE is supported in Icinga, you can install on Windows or Linux hosts NRPE clients like in Nagios to report on server processes state and easily monitor if server disk space / load or service is in critical state.

Share this on

Enable cyrillic in irssi IRC client on Linux

Sunday, June 2nd, 2013

I sometimes, still use IRC, mostly in Freenode. However sometimes I enter in Bulgarian IRC Network UniBG via IRC access node irc.spnet.net
Problem is by default irssi does not recognize Cyrillic so Bulgarian, Russian, Ukrainian and other Slavonic languages are unreadable. Same goes also for Chineese, Arabic and other "unstandard" characters. Hence from to make irssi support cyrillic, I had to launch it once:

linux:~$ irssi

And then type in it:

/set recode_fallback cp1251
/set recode_out_default_charset cp1251
/set recode_transliterate off
/set recode_autodetect_utf8 on
/set term_charset utf-8
/exit

linux:~$ irssi
After next irssi Launch Cyrillic displays proper Cyrillic letters.

how to make irssi support cyrillic Linux, irssi cyrillic letters screenshot
 

Share this on

Remove password prompt on GNOME Shutdown / Restart on Debian and Ubuntu Linux

Saturday, June 1st, 2013

It is ultra annoying, that in newest Debian and Ubuntu releases with GNOME 3 Desktop environment on every shutdown or restart you need to type in Super User (root) password, to authorize shutdown / restart.

Generally prompting for root password on GNOME restart is obviously a good think from security point of view, but from usability one – especially on notebooks it is useless annoyance…

So after changing this behavior I came up with this tiny article on how to get rid of GNOME Shutdown / Restart password prompt.

There is a click button (on left of Auth prompt on Shutdown showing URL to XML policy rule from where this behavior is controlled. A really good hint to where to look for to change those annoying behavior…

 Here is how to change this new annoying behavior to old GNOME 2 default restart with no root password prompt .
linux:~# gedit /usr/share/polkit-1/actions/org.freedesktop.consolekit.policy
 

Find in XML source sections:
 

Restart the system when multiple users are logged in System policy prevents restarting the system when other users are logged in no auth_admin_keep
Stop the system when multiple users are logged in System policy prevents stopping the system when other users are logged in no auth_admin_keep

To change Restart and Shutdown GUI behavior to not prompt for password, you need to modify in above code:

auth_admin_keep
To:

yes

After changes both sections should look like so:

<action id="org.freedesktop.consolekit.system.restart">
<description>Restart the system</description>
<message>System policy prevents restarting the system</message>
<defaults>
<allow_inactive>no</allow_inactive>
<allow_active>yes</allow_active>
</defaults>
</action>

<action id="org.freedesktop.consolekit.system.restart-multiple-users">
<description>Restart the system when multiple users are logged in</description>
<message>System policy prevents restarting the system when other users are logged in</message>
<defaults>
<allow_inactive>no</allow_inactive>
<allow_active>yes</allow_active>
</defaults>
</action>
 

That's all you finally get rid of the annoying prompt for root password. Enjoy :)

Share this on

Install Microsoft Office 2007 on Debian Linux 7.0 Wheezy

Monday, May 27th, 2013

Install Microsoft Office 2007 on Debian GNU / Linux Wheezy / Debian and Microsoft Office logo

Here is few steps I followed to install succesfully Microsoft.Office.Enterprise.2007.bg-BG.SP2.Integrated on my recently installed Debian stable Wheezy Linux.

1. Install Wine and Winetricks

linux:~# apt-get install --yes winetricks wine
....

2. Download Microsoft XML support – msxml3.msi

Download URL for MS XML from CNET is here
-

For simplicity I've mirrored msxml3.msi .
linux:~$ cd ~/.cache/winetricks/msxml3/
linux:~/.cache/winetricks/msxml3$ wget -q http://www.pc-freak.net/files/msxml3.msi
linux:~/cache/winetricks/msxml3$ winetricks msxml3

3. Insert Microsoft Office 2007 CD or  ISO

To Mount it you need to go in root shell:

To mount CD;

linux:~# mount -t iso9660 -o unhide /dev/cdrom /media/cdrom

To mount MS Office 2007 from ISO file:

linux:~# mkdir /mnt/ISO
linux:~# mount -t iso9660 -o loop Microsoft.Office.Enterprise.2007.bg-BG.SP2.Integrated.iso /mnt/ISO/

4. Launch MS Office installation with WINE emulator

linux:~# wine /mnt/ISO/Microsoft.Office.Enterprise.2007.bg-BG.SP2.Integrated/setup.exe

microsoft office 2007 on debian linux installation in progress

Follow normal installations steps Install and Enjoy.

To start Microsoft Office further follow menus:

Applications -> Wine -> Programs -> Microsoft Office
 

Share this on