Posts Tagged ‘logins’

Rsync copy files with root privileges between servers with root superuser account disabled

Tuesday, December 3rd, 2019

Reading Time: 5minutes



Sometimes on servers that follow high security standards in companies following PCI Security (Payment Card Data Security) standards it is necessery to have a very weird configurations on servers,to be able to do trivial things such as syncing files between servers with root privileges in a weird manners.This is the case for example if due to security policies you have disabled root user logins via ssh server and you still need to synchronize files in directories such as lets say /etc , /usr/local/etc/ /var/ with root:root user and group belongings.

Disabling root user logins in sshd is controlled by a variable in /etc/ssh/sshd_config that on most default Linux OS
installations is switched on, e.g. 

grep -i permitrootlogin /etc/ssh/sshd_config
PermitRootLogin yes

Many corporations use Vulnerability Scanners such as Qualys are always having in their list of remote server scan for SSH Port 22 to turn have the PermitRootLogin stopped with:


PermitRootLogin no

In this article, I'll explain a scenario where we have synchronization between 2 or more servers Server A / Server B, whatever number of servers that have already turned off this value, but still need to
synchronize traditionally owned and allowed to write directories only by root superuser, here is 4 easy steps to acheive it.


1. Add rsyncuser to Source Server (Server A) and Destination (Server B)

a. Execute on Src Host:


groupadd rsyncuser
useradd -g 1000 -c 'Rsync user to sync files as root src_host' -d /home/rsyncuser -m rsyncuser


b. Execute on Dst Host:


groupadd rsyncuser
useradd -g 1000 -c 'Rsync user to sync files dst_host' -d /home/rsyncuser -m rsyncuser


2. Generate RSA SSH Key pair to be used for passwordless authentication

a. On Src Host

su – rsyncuser

ssh-keygen -t rsa -b 4096


b. Check .ssh/ generated key pairs and make sure the directory content look like.


[rsyncuser@src-host .ssh]$ cd ~/.ssh/;  ls -1



3. Copy to Destination host server under authorized_keys


scp ~/.ssh/  rsyncuser@dst-host:~/.ssh/authorized_keys


Next fix permissions of authorized_keys file for rsyncuser as anyone who have access to that file (that exists as a user account) on the system
could steal the key and use it to run rsync commands and overwrite remotely files, like overwrite /etc/passwd /etc/shadow files with his custom crafted credentials
and hence hack you 🙂

Hence, On Destionation Host Server B fix permissions with:

su – rsyncuser; chmod 0600 ~/.ssh/authorized_keys
[rsyncuser@dst-host ~]$

An alternative way for the lazy sysadmins is to use the ssh-copy-id command


$ ssh-copy-id rsyncuser@
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed — if you are prompted now it is to install the new keys
root@'s password: 


For improved security here to restrict rsyncuser to be able to run only specific command such as very specific script instead of being able to run any command it is good to use little known command= option
once creating the authorized_keys


4. Test ssh passwordless authentication works correctly

For that Run as a normal ssh from rsyncuser

On Src Host


[rsyncuser@src-host ~]$ ssh rsyncuser@dst-host

Perhaps here is time that for those who, think enabling a passwordless authentication is not enough secure and prefer to authorize rsyncuser via a password red from a secured file take a look in my prior article how to login to remote server with password provided from command line as a script argument / Running same commands on many servers 

5. Enable rsync in sudoers to be able to execute as root superuser (copy files as root)


For this step you will need to have sudo package installed on the Linux server.

Then, Execute once logged in as root on Destionation Server (Server B)


[root@dst-host ~]# grep 'rsyncuser ALL' /etc/sudoers|wc -l || echo ‘rsyncuser ALL=NOPASSWD:/usr/bin/rsync’ >> /etc/sudoers


Note that using rsync with a ALL=NOPASSWD in /etc/sudoers could pose a high security risk for the system as anyone authorized to run as rsyncuser is able to overwrite and
respectivle nullify important files on Destionation Host Server B and hence easily mess the system, even shell script bugs could produce a mess, thus perhaps a better solution to the problem
to copy files with root privileges with the root account disabled is to rsync as normal user somewhere on Dst_host and use some kind of additional script running on Dst_host via lets say cron job and
will copy gently files on selective basis.

Perhaps, even a better solution would be if instead of granting ALL=NOPASSWD:/usr/bin/rsync in /etc/sudoers is to do ALL=NOPASSWD:/usr/local/bin/
that will get triggered, once the files are copied with a regular rsyncuser acct.


6. Test rsync passwordless authentication copy with superuser works

Do some simple copy, lets say copy files on Encrypted tunnel configurations located under some directory in /etc/stunnel on Server A to /etc/stunnel on Server B

The general command to test is like so:

rsync -aPz -e 'ssh' '–rsync-path=sudo rsync' /var/log rsyncuser@$dst_host:/root/tmp/

This will copy /var/log files to /root/tmp, you will get a success messages for the copy and the files will be at destination folder if succesful.


On Src_Host run:


[rsyncuser@src-host ~]$ dst=FQDN-DST-HOST; user=rsyncuser; src_dir=/etc/stunnel; dst_dir=/root/tmp;  rsync -aP -e 'ssh' '–rsync-path=sudo rsync' $src_dir  $rsyncuser@$dst:$dst_dir;


7. Copying files with root credentials via script

The simlest file to use to copy a bunch of predefined files  is best to be handled by some shell script, the most simple version of it, could look something like this.

# On server1 use something like this
# On server2 dst server
# add in /etc/sudoers
# rsyncuser ALL=NOPASSWD:/usr/bin/rsync




for i in $(echo ${src[@]}); do
rsync -aPvz –delete –dry-run -e 'ssh' '–rsync-path=sudo rsync' "$i" $rsyncuser@$dst_host:$dst_dir"$i";

In above script as you can see, we define a bunch of files that will be copied in bash array and then run a loop to take each of them and copy to testination dir.
A very sample version of the script 


Lets do short overview on what we have done here. First Created rsyncuser on SRC Server A and DST Server B, set up the key pair on both copied the keys to make passwordless login possible,
set-up rsync to be able to write as root on Dst_Host / testing all the setup and pinpointing a small script that can be used as a backbone to develop something more complex
to sync backups or keep system configurations identicatial – for example if you have doubts that some user might by mistake change a config etc.
In short it was pointed the security downsides of using rsync NOPASSWD via /etc/sudoers and few ideas given that could be used to work on if you target even higher
PCI standards.


Linux: /var/log/wtmp – No such file or directory quick fix and why it might be missing on a server

Thursday, May 4th, 2017

Reading Time: 2minutes


If you have to occasionally log  into some client old inherited (not installed by you) Linux servers on and just out of curiosity and for security sake dediced do a quick security (last user login) evaluation, e.g. issued the
last command just to find out you get the error:

last: /var/log/wtmp: No such file or directory

Perhaps this file was removed by the operator to prevent logging last info.

Then this might be a sure indicator that some malicious script kiddie (hax0r) activity has been run over the server or the ex-system administrator if fired recently decided to wipe out all his login tracks among with installing some other nasty rootkit or backdoor.

Under some circumstances the error might be caused also by badly written end user rotate script bugs (like shell or perl script) bugs or by a buggy deployment of Linux OS virtual machine.
The last: /var/log/wtmp: No such file or directory error is likely to happen on Ubuntu / Debian / Redhat / CentOS Linux distributions running on a Cloud PaaS service such as Amazon EC2, some of the Cloud services vendors do choose to explicitly remove /var/log/wtmp for the reason that many of end customers are using their Linux VM servers (Xen Virtualization / OpenVZ / LXC – Linux Containers) etc. irresponsibly and hence become a victim of script kiddie attacks and the failed logins attempts logged in /var/log/wtmp grow to many gigabytes.

Even some Linux distributions or system administrators of Linux server login hosts that has to keep tens of thousands of  login records monthly or are concentrating on simplicity and on an attempt to reduce size has purposefully deleted the last login entry file /var/log/wtmp file to save space.

But anyways if you happen to be missing this file always bear in mind that you might have been a victim of intrusion and you better run chkrootkit and rkhunter

Run below commands to fix the missing /var/log/wtmp

touch /var/log/wtmp
chmod 0664 /var/log/wtmp
chown root:utmp /var/log/wtmp

On some Linux distributions such as Ubuntu and Fedora you might also want to create /var/log/btmp(which is used to log failed login attempts to server)

touch /var/log/btmp
chmod 0664 /var/log/btmp
chown root:utmp /var/log/btmp

Once the files are created the last command will start logging server in logins and logouts as it is supposed to be again, e.g.:

linux:~# last -15
root pts/0 Fri May 5 16:41 still logged in

This article was inspired by a prior article found on the site is in Bulgarian so unfortunately you might not be able to read it, but as a content and concept it is pretty similar to, actually the site authorNikolay Nikolov (known in Internet Relay Chat IRC under the pseudonym Joni-B, happened to be an old friend from youth geek IT years 🙂


Maximal protection against SSH attacks. If your server has to stay with open SSH (Secure Shell) port open to the world

Thursday, April 7th, 2011

Reading Time: 5minutes
Brute Force Attack SSH screen, Script kiddie attacking
If you’re a a remote Linux many other Unix based OSes, you have defitenily faced the security threat of many failed ssh logins or as it’s better known a brute force attack

During such attacks your /var/log/messages or /var/log/auth gets filled in with various failed password logs like for example:

Feb 3 20:25:50 linux sshd[32098]: Failed password for invalid user oracle from port 51490 ssh2
Feb 3 20:28:30 linux sshd[32135]: Failed password for invalid user oracle1 from port 42778 ssh2
Feb 3 20:28:55 linux sshd[32141]: Failed password for invalid user test1 from port 51072 ssh2
Feb 3 20:30:15 linux sshd[32163]: Failed password for invalid user test from port 47481 ssh2
Feb 3 20:33:20 linux sshd[32211]: Failed password for invalid user testuser from port 51731 ssh2
Feb 3 20:35:32 linux sshd[32249]: Failed password for invalid user user from port 38966 ssh2
Feb 3 20:35:59 linux sshd[32256]: Failed password for invalid user user1 from port 55850 ssh2
Feb 3 20:36:25 linux sshd[32268]: Failed password for invalid user user3 from port 36610 ssh2
Feb 3 20:36:52 linux sshd[32274]: Failed password for invalid user user4 from port 45514 ssh2
Feb 3 20:37:19 linux sshd[32279]: Failed password for invalid user user5 from port 54262 ssh2
Feb 3 20:37:45 linux sshd[32285]: Failed password for invalid user user2 from port 34755 ssh2
Feb 3 20:38:11 linux sshd[32292]: Failed password for invalid user info from port 43146 ssh2
Feb 3 20:40:50 linux sshd[32340]: Failed password for invalid user peter from port 46411 ssh2
Feb 3 20:43:02 linux sshd[32372]: Failed password for invalid user amanda from port 59414 ssh2
Feb 3 20:43:28 linux sshd[32378]: Failed password for invalid user postgres from port 39228 ssh2
Feb 3 20:43:55 linux sshd[32384]: Failed password for invalid user ftpuser from port 47118 ssh2
Feb 3 20:44:22 linux sshd[32391]: Failed password for invalid user fax from port 54939 ssh2
Feb 3 20:44:48 linux sshd[32397]: Failed password for invalid user cyrus from port 34567 ssh2
Feb 3 20:45:14 linux sshd[32405]: Failed password for invalid user toto from port 42350 ssh2
Feb 3 20:45:42 linux sshd[32410]: Failed password for invalid user sophie from port 50063 ssh2
Feb 3 20:46:08 linux sshd[32415]: Failed password for invalid user yves from port 59818 ssh2
Feb 3 20:46:34 linux sshd[32424]: Failed password for invalid user trac from port 39509 ssh2
Feb 3 20:47:00 linux sshd[32432]: Failed password for invalid user webmaster from port 47424 ssh2
Feb 3 20:47:27 linux sshd[32437]: Failed password for invalid user postfix from port 55615 ssh2
Feb 3 20:47:54 linux sshd[32442]: Failed password for www-data from port 35554 ssh2
Feb 3 20:48:19 linux sshd[32448]: Failed password for invalid user temp from port 43896 ssh2
Feb 3 20:48:46 linux sshd[32453]: Failed password for invalid user service from port 52092 ssh2
Feb 3 20:49:13 linux sshd[32458]: Failed password for invalid user tomcat from port 60261 ssh2
Feb 3 20:49:40 linux sshd[32464]: Failed password for invalid user upload from port 40236 ssh2
Feb 3 20:50:06 linux sshd[32469]: Failed password for invalid user debian from port 48295 ssh2
Feb 3 20:50:32 linux sshd[32479]: Failed password for invalid user apache from port 56437 ssh2
Feb 3 20:51:00 linux sshd[32492]: Failed password for invalid user rds from port 45540 ssh2
Feb 3 20:51:26 linux sshd[32501]: Failed password for invalid user exploit from port 53751 ssh2
Feb 3 20:51:51 linux sshd[32506]: Failed password for invalid user exploit from port 33543 ssh2
Feb 3 20:52:18 linux sshd[32512]: Failed password for invalid user postgres from port 41350 ssh2
Feb 3 21:02:04 linux sshd[32652]: Failed password for invalid user shell from port 54454 ssh2
Feb 3 21:02:30 linux sshd[32657]: Failed password for invalid user radio from port 35462 ssh2
Feb 3 21:02:57 linux sshd[32663]: Failed password for invalid user anonymous from port 44290 ssh2
Feb 3 21:03:23 linux sshd[32668]: Failed password for invalid user mark from port 53285 ssh2
Feb 3 21:03:50 linux sshd[32673]: Failed password for invalid user majordomo from port 34082 ssh2
Feb 3 21:04:43 linux sshd[32684]: Failed password for irc from port 50918 ssh2
Feb 3 21:05:36 linux sshd[32695]: Failed password for root from port 38577 ssh2
Feb 3 21:06:30 linux sshd[32705]: Failed password for bin from port 53564 ssh2
Feb 3 21:06:56 linux sshd[32714]: Failed password for invalid user dev from port 34568 ssh2
Feb 3 21:07:23 linux sshd[32720]: Failed password for root from port 43799 ssh2
Feb 3 21:09:10 linux sshd[32755]: Failed password for invalid user bob from port 50026 ssh2
Feb 3 21:09:36 linux sshd[32761]: Failed password for invalid user r00t from port 58129 ssh2
Feb 3 21:11:50 linux sshd[537]: Failed password for root from port 58358 ssh2

This brute force dictionary attacks often succeed where there is a user with a weak a password, or some old forgotten test user account.
Just recently on one of the servers I administrate I have catched a malicious attacker originating from Romania, who was able to break with my system test account with the weak password tset .

Thanksfully the script kiddie was unable to get root access to my system, so what he did is he just started another ssh brute force scanner to crawl the net and look for some other vulnerable hosts.

As you read in my recent example being immune against SSH brute force attacks is a very essential security step, the administrator needs to take on a newly installed server.

The easiest way to get read of the brute force attacks without using some external brute force filtering software like fail2ban can be done by:

1. By using an iptables filtering rule to filter every IP which has failed in logging in more than 5 times

To use this brute force prevention method you need to use the following iptables rules:
linux-host:~# /sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state -state NEW -m recent -set
linux-host:~# /sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state -state NEW
-m recent -update -seconds 60 -hitcount 5 -j DROP

This iptables rules will filter out the SSH port to an every IP address with more than 5 invalid attempts to login to port 22

2. Getting rid of brute force attacks through use of hosts.deny blacklists

sshbl – The SSH blacklist, updated every few minutes, contains IP addresses of hosts which tried to bruteforce into any of currently 19 hosts (all running OpenBSD, FreeBSD or some Linux) using the SSH protocol. The hosts are located in Germany, the United States, United Kingdom, France, England, Ukraine, China, Australia, Czech Republic and setup to report and log those attempts to a central database. Very similar to all the spam blacklists out there.

To use sshbl you will have to set up in your root crontab the following line:

*/60 * * * * /usr/bin/wget -qO /etc/hosts.deny

To set it up from console issue:

linux-host:~# echo '*/60 * * * * /usr/bin/wget -qO /etc/hosts.deny' | crontab -u root -

These crontab will download and substitute your system default hosts with the one regularly updated on , thus next time a brute force attacker which has been a reported attacker will be filtered out as your Linux or Unix system finds out the IP matches an ip in /etc/hosts.deny

The /etc/hosts.deny filtering rules are written in a way that only publicly known brute forcer IPs will only be filtered for the SSH service, therefore other system services like Apache or a radio, tv streaming server will be still accessible for the brute forcer IP.

It’s a good practice actually to use both of the methods 😉
Thanks to Static (Multics) a close friend of mine for inspiring this article.

How to encrypt files with GPG and OpenSSL on GNU / Linux

Friday, November 25th, 2011

Reading Time: 2minutes
Encrypt files and directories with OpenSSL and GPG (GNUPG), OpenSSL and GPG encryption logo

I have just recently found out that it is possible to use openssl to encrypt files to tighten your security.
Why would I want to encrypt files? Well very simple, I have plain text files where I write down my passwords for servers or account logins for services I use on the internet.

Before this very day I use gpg to encrypt and decrypt my sensitive information files and archives. The way to encrypt files with GPG is very simple, here is an example:

server:~# ls -al test.txt
-rw-r--r-- 1 root root 12 Nov 25 16:50 test.txt
server:~# gpg -c test.txt > test.txt.gpg
Enter passphrase:
Repeat passphrase:

Typing twice the same password produces the encrypted file test.txt.gpg . In order to later decrypt the gpg password protected file I use cmd:

server:~# gpg -d test.txt.gpg >test.txt
Enter passphrase:
Repeat passphrase:
gpg: CAST5 encrypted data
gpg: encrypted with 1 passphrase
gpg: WARNING: message was not integrity protected

As one can see from above output by default gpg uses the CAST5 algorithm to encrypt the data. For all those curious on what kind of encryption does CAST5 provide and where the CAST5 origins are, in short CAST5 is a GNU invented cryptographic algorithm, the short description of the algorithm is as follows:

“…a DES-like Substitution-Permutation Network (SPN) cryptosystem which appears to have good resistance to differential cryptanalysis, linear cryptanalysis, and related-key cryptanalysis. This cipher also possesses a number of other desirable cryptographic properties, including avalanche, Strict Avalanche Criterion (SAC), Bit Independence Criterion (BIC), no complementation property, and an absence of weak and semi-weak keys.”

Anyways, for all those who trust more the DES128 encryption as an encryption algorithm to keep your data secret, the openssl command tool provides another mean to encrypt sensitive data.
To encrypt a file using the openssl’s DES encryption capabilities:

server:~# openssl des -salt -in test.txt -out test.txt.des
enter des-cbc encryption password:
Verifying - enter des-cbc encryption password:

As you can see to encrypt with the DES-CBC its necessery to type twice the secret password “salt” keyword which will be used as an encryption key.

To decrypt later on the DES encrypted file the cmd is:

server:~# openssl des -d -salt -in file.des -out file

In order to encrypt a whole directory earlier compressed with tar zip:

server:~# tar -czf - directory | openssl des -salt -out directory.tar.gz.des

Where directory is the name of directory which will be tarred and crypted.

To later decrypt with openssl the above encrypted tar.gz.des file:

server:~# openssl des -d -salt -in directory.tar.gzdes | tar -x