Posts Tagged ‘are’

Fix “init: Id “ad” respawning too fast: disabled for 5 minutes” – Reload /etc/inittab changes in memory apply without rebooting Linux server

Thursday, April 15th, 2021

Reading Time: 5minutes

inittab-logo-reload-inittab-without-reboot

During my daily sysadmin tasks I've been contacted by a colleague, reporting issues with missing logs in rsyslog on a very old Redhat Server release 5.11.
Exact version is:

root@linux-server:~# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.11 (Tikanga)

After checking the logs, I have confirmed his finding that in reality since about more than a year logs were not produced and al I could find multiple messages in /var/log/messages reading like:

init: Id "ad" respawning too fast: disabled for 5 minutes
init: Id "ad" respawning too fast: disabled for 5 minutes
init: Id "ad" respawning too fast: disabled for 5 minutes
init: Id "ad" respawning too fast: disabled for 5 minutes
init: Id "ad" respawning too fast: disabled for 5 minutes
init: Id "ad" respawning too fast: disabled for 5 minutes

I've checked the status of rsyslog which seemed to be fine

root@linux-server:~# /etc/init.d/rsyslog status
rsyslogd (pid  13709) is running…

The redhat version on the system was

root@linux-server:~# rpm -qa |grep -i rsyslog
rsyslog-3.22.1-7.el5

 

root@linux-server:~# tail -n 16 /var/log/messages
Apr 15 17:21:25 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:26:26 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:31:27 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:36:28 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:41:29 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:46:30 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:51:31 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 17:56:32 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 18:01:33 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 18:06:34 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 18:11:35 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 18:16:38 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 18:21:39 linux-server init: Id "ad" respawning too fast: disabled for 5 minutes

 

root@linux-server:~# /etc/init.d/rsyslog status
rsyslogd (pid  13709) is running…

Since the system is so old and I've seen this message and experienced this "respawning too fast: disabled for 5 minutes" myself in the past on some old Redhat 6.0 before RHEL was born as well as on Slackware Linux. The /etc/inittab which is nowadays obsoleted in newer Linux distributions was used to keep respawing a processes which have the chance to die out for some reason. 

For those unfamiliar with inittab there is a short extract from man inittab to get idea what it is.

 

NAME
       inittab  –  format of the inittab file used by the sysv-compatible init
       process

DESCRIPTION
       The inittab file describes which processes are started  at  bootup  and
       during  normal  operation  (e.g. /etc/init.d/boot, /etc/init.d/rc, get-
       tys…).  Init(8) distinguishes multiple runlevels, each of  which  can
       have  its  own  set of processes that are started.  Valid runlevels are
       0-6 plus A, B, and C for ondemand entries.  An  entry  in  the  inittab
       file has the following format:

              id:runlevels:action:process
 

So for example the use of /etc/inittab was very handy to configure a separate TTY12 (physical console) in the text environment of Linux to log all your messages. Another good use if you had a bash / perl / python script that you wanted to respawn (resurrect itself if it does out) on OS level without adding additional software like Dan Bernstein's all famous daemontools inittab was the right thing to use. It is a pity nowadays inittab is obsoleted in modern Linux OSes but the most likely reason to remove it is if you put some broken script that overeats CPU or memory if it runs multiple times you can easily get into a hung system.

Thus the logical thing to do is to check /etc/inittab content for any strange issues with less /etc/inittab and near the end of file found the problematic process which was triggering a never ending error messages to rsyslog and the module to protect from such messages in rsyslog by values $SystemLogRateLimitInterval and $SystemLogRateLimitBurst

# configure rsyslog rate limiting
# Rate-limiting
$SystemLogRateLimitInterval 5
$SystemLogRateLimitBurst 50000

The problem causing respawning too fast: disabled for 5 minutes

Was an old version of TivSM IBM Tivoli Service Manager /opt/tivoli/tsm/client/ba/bin/dsmc, set in the past in /etc/inittab it seems some colleague after updating to a more recent version has either changed the location of dsmc binary either the architecture of old tsm itself required a record in /etc/inittab in case if for some reasons or bugs the dsmc during backup creation was dying.

root@linux-server:~# tail -8 /etc/inittab
6:2345:respawn:/sbin/mingetty tty6

# Run xdm in runlevel 5
x:5:respawn:/etc/X11/prefdm -nodaemon

#ad:2345:respawn:/opt/tivoli/tsm/client/ba/bin/dsmc sched >/dev/null 2>&1

root@linux-server:~# rpm -qa |grep -i tivsm
TIVsm-API-5.3.4-0
TIVsm-stagent-5.3.4-0
TIVsm-BA-5.3.4-0
TIVsm-API64-5.3.4-0


The logical thing to do was to check whether this binary exist at all here is the result:

root@linux-server:~$ ls -al /opt/tivoli/tsm/client/ba/bin/dsmc
ls: /opt/tivoli/tsm/client/ba/bin/dsmc: No such file or directory

Obviously someone decided to comment out the inittab support for /opt/tivoli/tsm/client/ba/bin/dsmc as the binary was not present and the dsmc backup was executed via a separate one time cron job or the service itself was configured to run continue, but forgot to reread its configuration so in the kernel memory inittab was still having the instruction to loop over the dsmc binary, since the Linux machine was not rebooted ages (1472 days) or 4.8 years time.

root@linux-server:~#  uname -a; echo; uptime
Linux linux-server2.6.18-419.el5 #1 SMP Wed Feb 22 22:40:57 EST 2017 x86_64 x86_64 x86_64 GNU/Linux

 19:04:34 up 1472 days,  5:20,  1 user,  load average: 0.12, 0.07, 0.06


So what really happens is <b>inittab</b> is trying to kind of re-run all the time dsmc process in a similar way like it would in a bash never ending loop;


while [ 1 ]; do 
/opt/tivoli/tsm/client/ba/bin/dsmc sched
done

Since the $PATH location to the binary returns 'No such file or directory' message this message floods up the rsyslog every second which triggers the LimitBurst protection of rsyslog causing rsyslog to disable completely logging for 5 minutes. The next 5 minutes when the time expires for blocking out logging due to reached limit burst.
dsmc binary sends again few ten thousand of messages for few seconds which are already waiting in a queue of rsyslog and the LimitBurst anti DDoS protection activates again. The reason for the LimitBurst is simply because if it logging is not disabled quickly the repeating message is going to fill the hard drive of the system and noone will be able to login. So rsyslog activated the good protection.

It seems noone from support colleagues, never ever noticed this init: Id "ad" respawning too fast: disabled for 5 minutes in /var/log/messages. So since the syslog was continuesly blocked by overflow of non-sense messages, systems  normal logging was interruped and respectively prevented any other meaningful error messages and warnings from the system to get properly logged  and perhaps flooed the remote rsyslog logging servers @logging-servers:514 in /etc/rsyslog.conf


Fix to respawning too fast: disabled for 5 minutes

Very simply make /etc/inittab get reloaded in memory with:

root@linux-server:~# /sbin/init q

or with the linked telnet, which was so much used by us sys admins in the past

root@linux-server:~# /sbin/telinit q

To make the rsyslog suspension disabled of course we need to restart it again.

root@linux-server:~# /etc/init.d/rsyslog restart

root@linux-server:~# /etc/init.d/rsyslog status
rsyslogd (pid  13710) is running…

And Voila logs from services are being delivered normally via configured stuff in /etc/rsyslog.conf, to make sure this is so:

root@linux-server:~# tail -8 /var/log/messages
Apr 15 14:36:29 linux-serverinit: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 14:41:37 linux-serverinit: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 14:51:22 linux-serverinit: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 14:56:30 linux-serverinit: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 15:01:38 linux-serverinit: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 15:06:45 linux-serverinit: Id "ad" respawning too fast: disabled for 5 minutes
Apr 15 18:21:49 linux-server init: Re-reading inittab
Apr 15 18:21:54 linux-server kernel: imklog 3.22.1, log source = /proc/kmsg started.
Apr 15 18:21:54 linux-server rsyslogd: [origin software=”rsyslogd” swVersion=”3.22.1″ x-pid=”13709″ x-info=”http://www.rsyslog.com”] (re)start
Apr 15 18:41:54 linux-server rsyslogd: — MARK —
Apr 15 19:01:54 linux-server rsyslogd: — MARK —
Apr 15 19:21:54 linux-server rsyslogd: — MARK —
Apr 15 19:41:54 linux-server rsyslogd: — MARK —
Apr 15 20:01:54 linux-server rsyslogd: — MARK —

Adding custom user based host IP aliases load custom prepared /etc/hosts from non root user on Linux – Script to allow define IPs that doesn’t have DNS records to user preferred hostname

Wednesday, April 14th, 2021

Reading Time: 3minutes

adding-custom-user-based-host-aliases-etc-hosts-logo-linux

Say you have access to a remote Linux / UNIX / BSD server, i.e. a jump host and you have to remotely access via ssh a bunch of other servers
who have existing IP addresses but the DNS resolver recognized hostnames from /etc/resolv.conf are long and hard to remember by the jump host in /etc/resolv.conf and you do not have a way to include a new alias to /etc/hosts because you don't have superuser admin previleges on the hop station.
To make your life easier you would hence want to add a simplistic host alias to be able to easily do telnet, ssh, curl to some aliased name like s1, s2, s3 … etc.


The question comes then, how can you define the IPs to be resolvable by easily rememberable by using a custom User specific /etc/hosts like definition file? 

Expanding /etc/hosts predefined host resolvable records is pretty simple as most as most UNIX / Linux has the HOSTALIASES environment variable
Hostaliases uses the common technique for translating host names into IP addresses using either getaddrinfo(3) or the obsolete gethostbyname(3). As mentioned in hostname(7), you can set the HOSTALIASES environment variable to point to an alias file, and you've got per-user aliases

create ~/.hosts file

linux:~# vim ~/.hosts

with some content like:
 

g google.com
localhostg 127.0.0.1
s1 server-with-long-host1.fqdn-whatever.com 
s2 server5-with-long-host1.fqdn-whatever.com
s3 server18-with-long-host5.fqdn-whatever.com

linux:~# export HOSTALIASES=$PWD/.hosts

The caveat of hostaliases you should know is this will only works for resolvable IP hostnames.
So if you want to be able to access unresolvable hostnames.
You can use a normal alias for the hostname you want in ~/.bashrc with records like:

alias server-hostname="ssh username@10.10.10.18 -v -o stricthostkeychecking=no -o passwordauthentication=yes -o UserKnownHostsFile=/dev/null"
alias server-hostname1="ssh username@10.10.10.19 -v -o stricthostkeychecking=no -o passwordauthentication=yes -o UserKnownHostsFile=/dev/null"
alias server-hostname2="ssh username@10.10.10.20 -v -o stricthostkeychecking=no -o passwordauthentication=yes -o UserKnownHostsFile=/dev/null"

then to access server-hostname1 simply type it in terminal.

The more elegant solution is to use a bash script like below:

# include below code to your ~/.bashrc
function resolve {
        hostfile=~/.hosts
        if [[ -f “$hostfile” ]]; then
                for arg in $(seq 1 $#); do
                        if [[ “${!arg:0:1}” != “-” ]]; then
                                ip=$(sed -n -e "/^\s*\(\#.*\|\)$/d" -e "/\<${!arg}\>/{s;^\s*\(\S*\)\s*.*$;\1;p;q}" "$hostfile")
                                if [[ -n “$ip” ]]; then
                                        command "${FUNCNAME[1]}" "${@:1:$(($arg-1))}" "$ip" "${@:$(($arg+1)):$#}"
                                        return
                                fi
                        fi
                done
        fi
        command "${FUNCNAME[1]}" "$@"
}

function ping {
        resolve "$@"
}

function traceroute {
        resolve "$@"
}

function ssh {
        resolve "$@"
}

function telnet {
        resolve "$@"
}

function curl {
        resolve "$@"
}

function wget {
        resolve "$@"
}

 

Now after reloading bash login session $HOME/.bashrc with:

linux:~# source ~/.bashrc

ssh / curl / wget / telnet / traceroute and ping will be possible to the defined ~/.hosts IP addresses just like if it have been defined global wide on System in /etc/hosts.

Enjoy
 

6 of April Short Living and Death of Saint Methodius a brother of Constantin Cyril – Christian Enlighteners of the Bulgarian and Slavonic Lands

Wednesday, April 7th, 2021

Reading Time: 7minutes

Saints_Cyril_and_Methodius_Bulgarian_Enlighteners_Icon_by_Avram_Dichov,_1892

Saint Methodius was the bigger child of the two brothers enlighteners of Bulgarian Lands and the Slavonic nationsSaint Cyril (known as The Philosopher) and Saint Methodius born in Thessaloniki
a region of the Byzantine empire. Thessaloniki and its regions according to manuscripts has been mostly inhabited by Slavs at IX century.

In his young age he has chosen to be a Byzantine army man. For 10 years he has been assigned to govern a region of Slavonic lands part of Byzantine Eastern Empire. 
Even though he had been a successful in his worldly career and has been taking a respectful service for the Byzantine Emperor he felt the calling of God, gave up his high position in
aristocracy and entered one of the many monasteries of the empire in mount of Olympus Asia Minor region. Soon after to the same monastery came his younger brother Constantine
Since that moment the two brothers never departed until death of Cyril who died in Rome. And even in his death has been highly venerated by the Pope and considered saint even in
time of his burial at the famous Basillica Church San Clemente where they reside for veneration by Slavs even to this day.

San_clemente_fresco_Saint_Cyril-And-Methodius-in-Rome.

The receiving of Saint Cyril and Methodius in Rome with the Relics of Saint Clement 4th Pope of Rome

The two God chosen man also achieved a glorious deed which was not achieved by anyone before. The creation of the Glagolic Alphabet which become the foundation of the Cyrillic alphabet
currently used of about 300 000 000 people worldwide including by us Bulgarians. The new language was developed to be able to share the high truths of Christ the Gospel and his Church services and books  and 
most impotantly the eternal salvation prepared by God The Holy Trinity The Father the Son and the Holy Spirit for all those who love him and servce him.

The_Grave-of-Saint-Cyril-in-San-Clemente

The Holy relics of Saint Cyril in San Clemente Crypt

As knowing the truth about Heaven and Hell and the Heavenly
kingdom they were highly conserned for the high see of Slavs in the Empire who still have been in darkness of idolatry and paganism. Inspired by God they soon creating an Alphabet in which  translated the whole Holy Bible (The Gospel of Christ the Epistle books and Old Testament book  and Church service books translating the Eastern Orthodox Holy Liturgy of Saint John The Crysostom and Saint John The Basil.
in language understandable for the Slavs. The glagolic alphabet has been soon translated to the Cyrillic Alphabet by the 7 pupils of Saint Cyril and Methodius and has been spread in Bulgarian lands first then in Serbia, Kiev
and rest of Slavonic nations.

Saint_Methodius-Archibishop-of-Moravia-and_Pannonia

In Great Moravia a region today situated in Chech Republic the two saint brothers teached and baptized the local Slavs , teaching them the Eternal heavenly truths of the Gospel.
The German clergy envied the two apostle equal mans mission great success and tried to rebaptize and teach the already baptized according to a Roman Rite and spread their own influence pushed by a lust for power.
Hence Methodius (also called by Western histographs Method) has been chased away from Moravia by the German clergy to the nearby Pannonia, where king Kocel  who appreciated St. Methodius great deeds for his people and
not understanding why the German-clergy is chasing such a saintly man has moved him to Rome to Pope Adrian II-nd with an enquiry to ordain him a Bishop. The Pope ordained him almost immediately and accepted St. Methodius
and St Cyrilus with all his clergy in honour, because the 2 saints were bringing a highly esteemed gift the Holy Relics who were attained by them earlier in their Baptizmal Mission to The Hazars (an assignment given to them by the Eastern Byzantine Emperor
and successfully achieved).
During their travel in Herzon the two saints has found the Holy Relics of Saint Clement Ist the 4th Pope of Rome who has been exiled and passed away to Christ as a martyr in Herzon (Today's Crimea) in the distant 2nd century.
After his consecration in Rome saint  Metho has been raised from a simple Gospel preacher to the high clergy rank of Archibishop for Pannonia and Moravia, a region who has been under the rule of the Bishop of Rome.

The fight against the clean life of the saint continued with full power even after his raise to the throne of archibishopship. The envious pseudo spiritual missioners sent by the Pope hungering not for Christ but for earthly power slandered
frequently against Method. to the local prince. because Methodius was publicly rebuke their unclean life being filled with the power of Christ spirit who detests all uncleanness in man. At other times suggested to the pope that Methodius is
against the Western Church and eagers to cut Moravia and Pannonia from the diocese of Rome and give it to the Byzantines. In parallel the German "clergy" was frequetnly sending letters slandering against Methodius accusing him in inobedience to the Eastern Christian Emperor
and the The Great Church of Byzantium the homechurch of st. Cyril and Method. and to whom they were obedient still …
Methodius being already in advanced age had to go everywhere by person and rebuke and testify about himself that all this is a far away from truth. His enemies succeeded eventually and for 2 years and a half he has been thrown in prison without the knowledge of the pope.
Finding out about this ungodly deed of his inobedient power-lust german clergy Pope John VIII issued a special order bulla for the immediate liberation of his archibishop.

Saints-Cyril-Methodius-Zahari-Zograph-1848.

Icon of Saint Cyril and Methodius painting from 1848 y.

Even in this hardships saint Methodius never stopped to preach the Gospel to the Slavs and spread the Good news of Jesus Christ The Son of God enbodiment, life, Crucifix for the sins of all mankind anf Glorious Resurrection through which all those who believe are saved if they persist in the good deeds of Christ and stick to the trutfullness  of the Church of Christ which at the time of St. Cyrilus was before the Great Schism in year 1054, when the Roman Catholics has fallen away from the One Church who kept and preserved the head of the Church Jesus Christ teaching the Holy Eastern Orthodox Church.
Being even in prison Saint Methodius continued to translate The Holy books from Greek to the new language invented and adjusted for the Slavs talk. 
He left us as a heritage the Translation Copy of All Books of Holy Bible except the book of Maccabees which is today only present in the Orthodox Bible and considered one of the non-canonical but God inspired books, as well as the Nomocanon (the book with rules and Church laws of the Church and the clergy).
Nomocanon comes from .(Greek: Νομοκανών, Nomokanōn; from the Greek nomos – law and kanon – a rule). It is  a collection of ecclesiastical law, consisting of the elements from both the Civil law and the Canon law.
Nomocanons even today forms part of the canon law of the Eastern Catholic Churches (through the Eastern Catholic canon law) and of the Eastern Orthodox Churches.

Sveti_Sveti-Saints_Kyril-i-Metodij

Saint Methodius the Slav-Bulgarian enlighter passed silently to Christ full of God''s Grace in saintship on 6th of April year 885 A.D. His burial was in Velegrad Chech Republic, and according to the Church tradition about his living  he "lays in the Great Moravian Church in the left side of the wall behind the alter dedicated to the Mother of God the Holy Theotokos Virgin Maria". A church service dedicated to him is still presented in the Bulgarian Church's manuscripts and a copies of it in preservation museums in Russia.

The deeds of Saint Methodius and Saint Cyrilus has a mark on the whole world's events. Even to this day as in Bulgaria, we have a translation of the Bible based on the Cyrillic language named after saint Cyrilus the brother of Methodius based on his started work to translate Glagolic alphabet to Cyrillic, continued by his pupils and fully completed by Saint Clement (Kliment) of Ohrid, who was warmly accepted by Saint Apostle equal King Boris I of Bulgaria. Thanks to him the deeds of Saint Methodius and their pupils and we have the cyrillic alphabet and Bulgarian Orthodox Church and spirituality even today. The deeds of Saint Cyrilus and the Cyrillic are recognizable worldwide in the face of the strange for the Western and the Asians Cyrillic language used by Russians.

Two Saints Cyril and Methodius has were officially proclaimed as a protectors of Europe recently by Pope John Paul II, for a reason as the ex-Pope has seen and experienced the powerness of prayers of the two saints while he was a cardinal in Bulgaria before his raise to the Roman Popeship throne about little know nowadays.

The Memory and veneration of Saint Methodius and Cyrilus logically are mostly observed in Bulgaria logically as Bulgaria was the first Empire country historically to accept the spiritual fruits the new language.  Many Schools and institutions, streets are named after the two saints.
The official Church veneration of St. St. Cyril and Methodius is on 11 in Church and  May in secular tradition. The Bulgarian Orthodox church has established a number of feasts connected to the two saints as they''re the prime patron saints thanks to whose prayers the Bulgarian Church has been established survived through centuries and conitnues its triumphal march towards the kingdom of heaven.

Let by the two saints Methodius and Cyril Bulgaria and Europe have Peace, Faith and Love and Enlightenment about the true teaching of Christ the Holy Orthodoxy!
 

How to redirect / forward all postfix emails to one external email address?

Thursday, October 29th, 2020

Reading Time: 3minutes

Postfix_mailserver-logo-howto-forward-email-with-regular-expression-or-maildrop

Lets say you're  a sysadmin doing email migration of a Clustered SMTP and due to that you want to capture for a while all incoming email traffic and redirect it (forward it) towards another single mailbox, where you can review the mail traffic that is flowing for a few hours and analyze it more deeper. This aproach is useful if you have a small or middle sized mail servers and won't be so useful on a mail server that handels few  hundreds of mails hourly. In below article I'll show you how.

How to redirect all postfix mail for a specific domain to single external email address?

There are different ways but if you don't want to just intercept the traffic and a create a copy of email traffic using the always_bcc integrated postfix option (as pointed in my previous article postfix copy every email to a central mailbox).  You can do a copy of email flow via some custom written dispatcher script set to be run by the MTA on each mail arriva, or use maildrop filtering functionality below is very simple example with maildrop in case if you want to filter out and deliver to external email address only email targetted to specific domain.

If you use maildrop as local delivery agent to copy email targetted to specifidc domain to another defined email use rule like:

if ( /^From:.*domain\.com/:h ) {
  cc "!someothermail@domain2.com"
}


To use maildrop to just forward email incoming from a specific sender towards local existing email address on the postfix to an external email address  use something like:

if ( /^From: .*linus@mail.example.com.*/ )
{
        dotlock "forward.lock" {
          log "Forward mail"
          to "|/usr/sbin/sendmail linuxbox@collector.example.com"
        }
}

Then to make the filter active assuming the user has a physical unix mailbox, paste above to local user's  $HOME/.mailfilter.

What to do if your mail delivered via your Email-Server.com are sent from a monitoring and alarming scripts that are sending towards many mailboxes that no longer exist after the migration?

To achive capturing all normal attempted to be sent traffic via the mail server, we can forward all served mails towards a single external mail address you can use the nice capability of postfix to understand PCRE perl compatible regular expressions. Regular expressions in postfix of course has its specific I recommend you take a look to the postfix regexp table documentation here, as well as check the Postfix Regex / Tester / Debugger online tool – useful to validate a regexp you want to implement.

How to use postfix regular expression to do a redirect of all sent emails via your postfix mail relayhost towards external mail servers?

 

In main.cf /etc/postfix/main.cf include this line near bottom or as a last line:

virtual_maps = hash:/etc/postfix/virtual, regexp:/etc/postfix/virtual-regexp

One defines the virtual file which can be used to define any of your virtual domains you want to simulate as present on the local postfix, the regexp: does load the file which is read by postfix where you can type the regular expression applied to every incoming email via SMTP port 25 or encrypted MTA ports 385 / 995 etc.

So how to redirect all postfix mail to one external email address for later analysis?

Create file /etc/postfix/virtual-regexp

/.+@.+/ external-forward-email@gmail.com

Next build the mapfile (this will generate /etc/postfix/virtual-regexp.db )
 

# postmap /etc/postfix/virtual-regexp

This also requires a virtual.db to exist. If it doesn't create an empty file called virtual and run again postmap postfix .db generator

# touch /etc/postfix/virtual && postmap /etc/postfix/virtual


Note in /etc/postfix/virtual you can add your postfix mail domains for which you want the MTA to accept mail as a local mail.

In case you need to view all postfix defined virtual domains configured to accept mail locally on the mail server.
 

$ postconf -n | grep virtual
virtual_alias_domains = mydomain.com myanotherdomain.com
virtual_alias_maps = hash:/etc/postfix/virtual


The regexp /.+@.+/ external-forward-email@gmail.com applied will start forwarding mails immediately after you reload the MTA with:

# systemctl restart postfix


If you want to exclude target mail domains to not be captured by above regexp, in /etc/postfix/virtual-regexp place:

/.+@exclude-domain1.com/ @exclude-domain1.com
/.+@exclude-domain2.com/ @exclude-domain2.com

Time for a test. Send a test email


Next step is to Test it mail forwarding works as expected
 

# echo -e "Tseting body" | mail -s "testing subject" -r "testing@test.com" whatevertest-user@mail-recipient-domain.com

Postfix copy every email to a central mailbox (send a copy of every mail sent via mail server to a given email)

Wednesday, October 28th, 2020

Reading Time: 2minutes

Postfix-logo-always-bcc-email-option-send-all-emails-to-a-single-address-with-postfix.svg

Say you need to do a mail server migration, where you have a local configured Postfix on a number of Linux hosts named:

Linux-host1
Linux-host2
Linux-host3

etc.


all configured to send email via old Email send host (MailServerHostOld.com) in each linux box's postfix configuration's /etc/postfix/main.cf.
Now due to some infrastructure change in the topology of network or anything else, you need to relay Mails sent via another asumably properly configured Linux host relay (MailServerNewHost.com).

Usually such a migrations has always a risk that some of the old sent emails originating from local running scripts on Linux-host1, Linux-Host2 … or some application or anything else set to send via them might not properly deliver emails to some external Internet based Mailboxes via the new relayhost MailServerNewHost.com.

E.g. in /etc/postfix/main.cfLinux-Host* machines, you have below config after the migration:

relayhost = [MailServerNewHost.com]

Lets say that you want to make sure, that you don't end up with lost emails as you can't be sure whether the new email server will deliver correctly to the old repicient emails. What to do then?

To make sure will not end up in undelivered state and get lost forever after a week or so (depending on the mail queue configuration retention period made on Linux sent MTAs and mailrelay MailServerNewHost.com, it is a very good approach to temprorary set all email communication that will be sent via MailServerNewHost.com a BCC emaills (A Blind Carbon Copy) of each sent mail via relay that is set on your local configured Postfix-es on Linux-Host*.

In postfix to achieve that it is very easy all you have to do is set on your MailServerNewHost.com a postfix config variable always_bcc smartly included by postfix Mail Transfer Agent developers for cases exactly like this.

To forward all passed emails via the mail server just place in the end of /etc/postfix/mail.conf after login via ssh on MailServerNewHost.com

always_bcc=All-Emails@your-diresired-redirect-email-address.com


Now all left is to reload the postfix to force the new configuration to get loaded on systemd based hosts as it is usually today do:

# systemctl reload postfix


Finally to make sure all works as expected and mail is sent do from do a testing via local MTAs. 
 

Linux-Host:~# echo -e "Testing body" | mail -s "testing subject" -r "testing@test.com" georgi.stoyanov@remote-user-email-whatever-address.com

Linux-Host:~# echo -e "Testing body" | mail -s "testing subject" -r "testing@test.com" georgi.stoyanov@sample-destination-address.com


As you can see I'm using the -r to simulate a sender address, this is a feature of mailx and is not available on older Linux Os hosts that are bundled with mail only command.
Now go to and open the All-Emails@your-diresired-redirect-email-address.com in Outlook (if it is M$ Office 365 MX Shared mailbox), Thunderbird or whatever email fetching software that supports POP3 or IMAP (in case if you configured the common all email mailbox to be on some other Postfix / Sendmail / Qmail MTA). and check whether you started receiving a lot of emails 🙂

That's all folks enjoy ! 🙂

Improve wordpress admin password encryption authentication keys security with WordPress Unique Authentication Keys and Salts

Friday, October 9th, 2020

Reading Time: 5minutes

wordpress-improve-security-logo-linux

Having a wordpress blog or website with an admistrator and access via a Secured SSL channel is common nowadays. However there are plenty of SSL encryption leaks already out there and many of which are either slow to be patched or the hosting companies does not care enough to patch on time the libssl Linux libraries / webserver level. Taking that in consideration many websites hosted on some unmaintained one-time run not-frequently updated Linux servers are still vulneable and it might happen that, if you paid for some shared hosting in the past and someone else besides you hosted the website and forget you even your wordpress installation is still living on one of this SSL vulnerable hosts. In situations like that malicious hackers could break up the SSL security up to some level or even if the SSL is secured use MITM (MAN IN THE MIDDLE) attack to simulate your well secured and trusted SSID Name WIFi network to  redirects the network traffic you use (via an SSL transparent Proxy) to connect to WordPress Administrator Dashbiard via https://your-domain.com/wp-admin. Once your traffic is going through the malicious hax0r even if you haven't used the password to authenticate every time, e.g. you have saved the password in browser and WordPress Admin Panel authentication is achieved via a Cookie the cookies generated and used one time by Woddpress site could be easily stealed one time and later from the vicious 1337 h4x0r and reverse the hash with an interceptor Tool and login to your wordpress …

Therefore to improve the wordpress site security it very important to have configured WordPress Unique Authentication Keys and Salts (known also as the WordPress security keys).

They're used by WordPress installation to have a uniquely generated different key and Salt from the default one to the opened WordPress Blog / Site Admin session every time.

So what are the Authentication Unique Keys and Salts and why they are Used?

Like with almost any other web application, when PHP session is opened to WordPress, the code creates a number of Cookies stored locally on your computer.

Two of the cookies created are called:

 wordpress_[hash]
wordpress_logged_in_[hash]

First  cookie is used only in the admin pages (WordPress dashboard), while the second cookie is used throughout WordPress to determine if you are logged in to WordPress or not. Note: [hash] is a random hashed value typically assigned to your session, therefore in reality the cookies name would be named something like wordpress_ffc02f68bc9926448e9222893b6c29a9.

WordPress session stores your authentication details (i.e. WordPress username and password) in both of the above mentioned cookies.

The authentication details are hashed, hence it is almost impossible for anyone to reverse the hash and guess your password through a cookie should it be stolen. By almost impossible it also means that with today’s computers it is practically unfeasible to do so.

WordPress security keys are made up of four authentication keys and four hashing salts (random generated data) that when used together they add an extra layer to your cookies and passwords. 

The authentication details in these cookies are hashed using the random pattern specified in the WordPress security keys. I will not get into too much details but as you might have heard in Cryptography Salts and Keys are important – an indepth explanation on Salts Cryptography (here). A good reading for those who want to know more on how does the authentication based and salts work is on stackexchange.

How to Set up Salt and Key Authentication on WordPress
 

To be used by WP Salts and Key should be configured under wp-config.php usually they look like so:

wordpress-website-blog-salts-keys-wp-config-screenshot-linux

!!! Note !!!  that generating (manually or generated via a random generator program), the definition strings you have to use a random string value of more than 60 characters to prevent predictability 

The default on any newly installed WordPress Website is to have the 4 definitions with _KEY and the four _SALTs to be unconfigured strings looks something like:

default-WordPress-security-keys-and-salts-entries-in-wordPress-wp-config-php-file

Most people never ever take a look at wp-config.php as only the Web GUI Is used for any maintainance, tasks so there is a great chance that if you never heard specifically by some WordPress Security Expert forum or some Security plugin (such as WP Titan Anti Spam & Security) installed to report the WP KEY / SALT you might have never noticed it in the config.

There are 8 WordPress security keys in current WP Installs, but not all of them have been introduced at the same time.
Historically they were introduced in WP versions in below order:

WordPress 2.6: AUTH_KEY, SECURE_AUTH_KEY, LOGGED_IN_KEY
WordPress 2.7: NONCE_KEY
WordPress 3.0: AUTH_SALT, SECURE_AUTH_SALT, LOGGED_IN_SALT, NONCE_SALT

Setting a custom random generated values is an easy task as there is already online Wordpress Security key Random generator.
You can visit above address and you will get an automatic randomly generated values which could be straight copy / pasted to your wp-config.php.

Howeever if you're a paranoic on the guessability of the random generator algorithm, I would advice you use the generator and change some random values yourself on each of the 8 line, the end result in the configuration should be something similar to:

 

define('AUTH_KEY',         '|w+=W(od$V|^hy$F5w)g6O-:e[WI=NHY/!Ez@grd5=##!;jHle_vFPqz}D5|+87Q');
define('SECURE_AUTH_KEY',  'rGReh.<%QBJ{DP )p=BfYmp6fHmIG~ePeHC[MtDxZiZD;;_OMp`sVcKH:JAqe$dA');
define('LOGGED_IN_KEY',    '%v8mQ!)jYvzG(eCt>)bdr+Rpy5@t fTm5fb:o?@aVzDQw8T[w+aoQ{g0ZW`7F-44');
define('NONCE_KEY',        '$o9FfF{S@Z-(/F-.6fC/}+K 6-?V.XG#MU^s?4Z,4vQ)/~-[D.X0<+ly0W9L3,Pj');
define('AUTH_SALT',        ':]/2K1j(4I:DPJ`(,rK!qYt_~n8uSf>=4`{?LC]%%KWm6@j|aht@R.i*ZfgS4lsj');
define('SECURE_AUTH_SALT', 'XY{~:{P&P0Vw6^i44Op*nDeXd.Ec+|c=S~BYcH!^j39VNr#&FK~wq.3wZle_?oq-');
define('LOGGED_IN_SALT',   '8D|2+uKX;F!v~8-Va20=*d3nb#4|-fv0$ND~s=7>N|/-2]rk@F`DKVoh5Y5i,w*K');
define('NONCE_SALT',       'ho[<2C~z/:{ocwD{T-w+!+r2394xasz*N-V;_>AWDUaPEh`V4KO1,h&+c>c?jC$H');

 


Wordpress-auth-key-secure-auth-salt-Linux-wordpress-admin-security-hardening

Once above defines are set, do not forget to comment or remove old AUTH_KEY / SECURE_AUTH_KEY / LOGGED_IN_KEY / AUTH_SALT / SECURE_AUTH_SALT / LOGGED_IN_SALT /NONCE_SALT keys.

The values are configured one time and never have to be changed, WordPress installation automatic updates or Installed WP Plugins will not tamper the value with time.
You should never expand or show your private generated keys to anyone otherwise this could be used to hack your website site.
It is also a good security practice to change this keys, especially if you have some suspects someone has somehow stolen your wp-onfig keys. 
 

Closure

Having AUTH KEYs and Properly configured is essential step to improve your WordPress site security. Anytime having any doubt for a browser hijacked session (or if you have logged in) to your /wp-admin via unsecured public Computer with a chance of a stolen site cookies you should reset keys / salts to a new random values. Setting the auth keys is not a panacea and frequent WP site core updates and plugins should be made to secure your install. Always do frequent audits to WP owned websites with a tool such as WPScan is essential to keep your WP Website unhacked.

 

 

Report haproxy node switch script useful for Zabbix or other monitoring

Tuesday, June 9th, 2020

Reading Time: 3minutes

zabbix-monitoring-logo
For those who administer corosync clustered haproxy and needs to build monitoring in case if the main configured Haproxy node in the cluster is changed, I've developed a small script to be integrated with zabbix-agent installed to report to a central zabbix server via a zabbix proxy.
The script  is very simple it assumed DC1 variable is the default used haproxy node and DC2 and DC3 are 2 backup nodes. The script is made to use crm_mon which is not installed by default on each server by default so if you'll be using it you'll have to install it first, but anyways the script can easily be adapted to use pcs cmd instead.

Below is the bash shell script:

UserParameter=active.dc,f=0; for i in $(sudo /usr/sbin/crm_mon -n -1|grep -i 'Node ' |awk '{ print $2 }'); do ((f++)); DC[$f]="$i"; done; \
DC=$(sudo /usr/sbin/crm_mon -n -1 | grep 'Current DC' | awk '{ print $1 " " $2 " " $3}' | awk '{ print $3 }'); \
if [ “$DC” == “${DC[1]}” ]; then echo “1 Default DC Switched to ${DC[1]}”; elif [ “$DC” == “${DC[2]}” ]; then \
echo "2 Default DC Switched to ${DC[2]}”; elif [ “$DC” == “${DC[3]}” ]; then echo “3 Default DC: ${DC[3]}"; fi


To configure it with zabbix monitoring it can be configured via UserParameterScript.

The way I configured  it in Zabbix is as so:


1. Create the userpameter_active_node.conf

Below script is 3 nodes Haproxy cluster

# cat > /etc/zabbix/zabbix_agentd.d/userparameter_active_node.conf

UserParameter=active.dc,f=0; for i in $(sudo /usr/sbin/crm_mon -n -1|grep -i 'Node ' |awk '{ print $2 }'); do ((f++)); DC[$f]="$i"; done; \
DC=$(sudo /usr/sbin/crm_mon -n -1 | grep 'Current DC' | awk '{ print $1 " " $2 " " $3}' | awk '{ print $3 }'); \
if [ “$DC” == “${DC[1]}” ]; then echo “1 Default DC Switched to ${DC[1]}”; elif [ “$DC” == “${DC[2]}” ]; then \
echo "2 Default DC Switched to ${DC[2]}”; elif [ “$DC” == “${DC[3]}” ]; then echo “3 Default DC: ${DC[3]}"; fi

Once pasted to save the file press CTRL + D


The version of the script with 2 nodes slightly improved is like so:
 

UserParameter=active.dc,f=0; for i in $(sudo /usr/sbin/crm_mon -n -1|grep -i 'Node ' |awk '{ print $2 }' | sed -e 's#:##g'); do DC_ARRAY[$f]=”$i”; ((f++)); done; GET_CURR_DC=$(sudo /usr/sbin/crm_mon -n -1 | grep ‘Current DC’ | awk ‘{ print $1 ” ” $2 ” ” $3}’ | awk ‘{ print $3 }’); if [ “$GET_CURR_DC” == “${DC_ARRAY[0]}” ]; then echo “1 Default DC ${DC_ARRAY[0]}”; fi; if [ “$GET_CURR_DC” == “${DC_ARRAY[1]}” ]; then echo “2 Default Current DC Switched to ${DC_ARRAY[1]} Please check “; fi; if [ -z “$GET_CURR_DC” ] || [ -z “$DC_ARRAY[1]” ]; then printf "Error something might be wrong with HAProxy Cluster on  $HOSTNAME "; fi;


The haproxy_active_DC_zabbix.sh script with a bit of more comments as explanations is available here 
2. Configure access for /usr/sbin/crm_mon for zabbix user in sudoers

 

# vim /etc/sudoers

zabbix          ALL=NOPASSWD: /usr/sbin/crm_mon


3. Configure in Zabbix for active.dc key Trigger and Item

active-node-switch1

Linux: Howto Disable logging for all VirtualHosts on Apache and NGINX Webservers one liner

Wednesday, July 1st, 2020

Reading Time: 2minutes

disable-apache-nginx-logging-for-all-virtualhosts
Did you happen to administer Apache Webservers or NGINX webservers whose logs start to grow so rapidly that are flooding the disk too quickly?
Well this happens sometimes and it also happens that sometimes you just want to stop logging especially, to offload disk writting.

There is an easy way to disable logging for requests and errors (access_log and error_log usually residing under /var/log/httpd or /var/log/nginx ) for  all configured Virtual Domains with a short one liner, here is how.

Before you start  Create backup of /etc/apache2/sites-enabled / or /etc/nginx to be able to revert back to original config.

# cp -rpf /etc/apache2/sites-enabled/ ~/

# cp -rpf /etc/nginx/ ~/


1. Disable Logging for All  Virtual Domains configured for Apache Webserver

First lets print what the command will do to make sure we don't mess something

# find /home/hipo/sites-enabled/* -exec echo sed -i 's/#*[Cc]ustom[Ll]og/#CustomLog/g' {} \;


You will get some output like

find /home/hipo//sites-enabled/* -exec echo sed -i 's/#*[Cc]ustom[Ll]og/#CustomLog/g' {} \;

find /etc/apache2/sites-enabled/* -exec sed -i 's/#*[Cc]ustom[Ll]og/#CustomLog/g' {} \;
find /etc/apache2/sites-enabled/* -exec sed -i 's/#*[Ee]rror[Ll]og/#ErrorLog/g' {} \;

2. Disable Logging for All configured Virtual Domains for NGINX Webserver
 

find /etc/nginx/sites-enabled/* -exec sed -i 's/#*access_log/#access_log/g' {} \;
find /etc/nginx/sites-enabled/* -exec sed -i 's/#*error_log/#error_log/g' {} \;

f course above substituations that will comment out with '#' occurances from file configs of only default set access_log and error_log / access.log, error.log 
for machines where there is no certain convention on file naming and there are multiple domains in custom produced named log files this won't work.

This one liner was inspired from a friend's daily Martin Petrov. Martin blogged initially about this nice tip for those reading Cyrillic check out mpetrov.net, so. Thanks Marto ! 🙂

Christ is Risen Eastern Orthodox Resurrection Paschal Greeting in Different Languages

Friday, April 24th, 2020

Reading Time: 10minutes

Resurrection-of-Christ-holy-orthodox-icon-Voskresenie-Hristovo

Happy Easter to All Orthodox Christians worldwide ! 
We are the the bright week –  this is the week after The Feast of Feasts Resurrection of Christ. This year in 2020, we Orthodox Christians celebrated this feast on 19th, 20th and 21st of April. The celebrations of the Feast of Christians and the Paschal joy continues for the whole week, so I found some time to quickly blog to share the Joy of the Resurrection of the Savior Jesus Christ who has freed all People from the Fear of the Death by Manifestating Death has been overtaken by Life Eternally.
Earlier years, I've blogged shortly on the Christ is risen in many languages. But this time I decided to extend my previous blog by adding some more details on which are the Member Churches consisting the Christ body of Holy Orthodoxy, What is the Creed of Faith (Symbol of Faith) difference Between Orthodox Christians and Roman Catholics and why we're not catholics and do celebrate Easter on a different date from Roman Catholics. Finally I will post the Paschal Greeting translated to as many languages I could find.

In the Eastern Orthodox Christian world which is the True Church of Christ consists of 15 National Churches each traced back to the Holy Apostles of Christ, each of Churches is in eucharistic Communion with the other. 

Canonical Orthodox Christ Churches as of year 2020 are the following:

1. Orthodox Church of Constantinople
2. Orthodox Church of Alexandria
3. Orthodox Church of Antiochia
4. Jerusalem Patriarchal Orthodox Church
5. Bulgarian Orthodox Church
6. Georgian Orthodox Church
7. Serbian Orthodox Church
8. Russian Orthodox Church
9. Romanian Orthodox Church
10. Orthodox Church of Cyprus (archibishopship)
11. Greek Orthodox Church
12. Albanian Orthodox Church
13. Polish Orthodox Church
14. Orthodox Church of Chech Lands and Slovakia
15. American Orthodox Church


Historically Christ Church was one before the Great Schism that was perhaps the greatest tragedy of mankind after Christ's Crucifix it occured in 1054 A.D. About this saddening events, the manuscripts and contemporary saints says with this terrible event, the whole world shaked its basis. The result of the Bulla brought by Pope's messangers in Hagia Sophia Cathedral in Constantinople in the Holy Alter of the Church putting in document of official schism and the Church of the east condeming the Western Church Cuhrch action headed by the pope due to the false Creed of faith inclusion legallized by the pope with the so called 'Filioque' word formula that changed the original agreement of Church fathers decisions on the First Ecumenical Council of Constantinople in 381 A.D. (which by the way puts Anathemas on anyone who dares to change the Creed of Faith as well change by the Popes in the well known ancient Baptism Formulas like oilment (receving the sign of the holy spirit during baptism).

The historical number of Orthodox Churche dioceses were much more numerous but with the time and the hardships this are the only ones that left as official Churches, many dioceses were destroyed by the Muslims Conquests and Roman Catholics orders like the Jesuits whose fight against orthodoxy has been severe in their attempt to make the whole world to turn to the pope, this is very well known by the many remains of Uniates around Europe, especially in nowadays Ukraine. There is a lot of nations like Chechz and Hungarians whose for many centuries confessed orthodoxy but due to the economic relations with the West and the converion of their rulers (princes / Kings) etc. to Roman Catholicism has gradually converted their Eastern Churches to Roman Catholics.

The origional Nicean Creed (Nicea-Constantinople) creed of faith reads as this:

Nicean Creed of Faith ( Agreed on 381 Anno Dommini in Emperor Constantine City of Byzantine Empire Constantinople)

We believe in one God, the Father Almighty, Maker of heaven and earth, and of all things visible and invisible.

And in one Lord Jesus Christ, the only-begotten Son of God, begotten of the Father before all worlds (æons), Light of Light, very God of very God, begotten, not made, consubstantial with the Father;

by whom all things were made;

who for us men, and for our salvation, came down from heaven, and was incarnate by the Holy Ghost and of the Virgin Mary, and was made man;

he was crucified for us under Pontius Pilate, and suffered, and was buried, and the third day he rose again, according to the Scriptures, and ascended into heaven, and sitteth on the right hand of the Father;

from thence he shall come again, with glory, to judge the quick and the dead. ;

whose kingdom shall have no end.

* And in the Holy Ghost, the Lord and Giver of life, who proceedeth from the Father, who with the Father and the Son together is worshiped and glorified, who spake by the prophets.

In one Holy Catholic and apostolic Church; we acknowledge one baptism for the remission of sins; we look for the resurrection of the dead, and the life of the world to come. Amen.

The Western Church head Bishop the Pope and local priests due to some historical regions of Spain and other parts of Western Europe's aim to fight heresies included the word Latin word Filioque in above translated text (Word which is translated as "And from the Son") in above starred line 'And in te Holy Ghost, The Lord Giver of Life who proceedeth from the Father' become 'And in the Holy Ghost, The Lord Giver of Life who proceedeth from the Father (Filioque) = and from the Son.' this was acceptable for the Eastern Churches until the moment when this Confession of Faith has been legalized for the Whole Western Church with a decree so called pope 'Bulla' with which it become the official confession of faith for the whole Catholic Church. The Eastern Church of course was following the accepted Canon rules from the first Ecumenical Council in 381 A.D. and rejected to accept the definition of the Pope at first in the Face of Saint Patriarch Photios I of Constantinople (year 810 –  893) and become official in 1054 by the rule of Pope Leo whose legates tried to claim Headship of the Pope over the whole Church and questioned the title of the Constantinople Ecumenical Patriarch Michael I Cerularius.
Along with the chages of the Creed of Faith the West, the years during centuries VII and IX centuries has already put a lot of differences in the East and West Church along doctrinal, theological, linguistic, political, and geographical lines so the split was a reflection of all this. The Latin Church was much more power hungry and more progressive for its time and authoritarian, trying to combine the Worldly power with the Spiritual one given by the line of Apostoles from Christ Ceasaris-Papism, where the Eastern Church was governed in the ancient model of the Worldly power in face of Eastern Roman empire Emperor and the Patriarch who was a governor of the Spiritual power. The schism was worsened also by the many Latins raids in the Eastern Empire Christian brothers and the sacking of Constantinople in 8-13 April year 1204. Of course both Wester and Eastern Roman Empire had an appetite for a conquest over the other and often this has lead the secular rulers on both sides to try to manipulate activities of the spiritual leaders of both to work for their interests, but the schism would never occur if the spiritual establishment of the Church which are the Holy Canons (decision of the Ecumenical Councils) were not breached by the Western Church.
One of this breaches of the Ancient canons is the Celebration of Eastern Pascha which says the Christian Pascha should never coincide with Jewish Pascha. However in the Western Church this rule was breached and nowadays The Eastern (The Day of the Resurrection of Christ) in the Roman Catholic Church (Western Church) coincides most of the years with Jewish Pascha (both Roman Catholics and the executors of Christ who never accepted him the Jews celebrate together … a sad fact).

Nowadays most of the Ancient Churches of the East together with the Eastern Orthodox Churches, who are confessing the Faith of Christ such as it was handed by the Saint Fathers has a very specific ancient way of confession of faith similar to the Creed of Faith which was a very common short ancient way to confess the faith when two Christians met it is perhaps originating from the times of the Heresies in the 1st century right after the Christ Crucifix, when the pupil of Christ used it to confirm the Glorious and unexplainable Miracle of the Resurrection of the Lord Jesus Christ from the Death in Real Body in the 3rd day from the Grave in the Cave where his body was buried.

Resurrection-of-Christ-Anastasis-Greek-orthodox-icon

The Greeting Formula is the well known in the Eastern Orthodox Churches such as in Bulgaria / Greece / Russia / Serbia etc. Christ is Risen.
On every easter Almost everyone in the Orthodox Christian Countires greats everyone else both in homes on the street at work or anywhere relatives friends and even unfamiliar people who has to do business deeds with the immersely joyful greeting.

ХРИСТОС ВОСКРЕСЕ / CHRIST IS RISEN !!!!! !!!

Then the greeted Person answers back

ВОЙСТИНУ ВОСКРЕСЕ / TRULY HE IS RISEN (INDEED HE IS RISEN) !!!!!!!!

In the Orthodox Churches, believers do greet themselves with this heartful joyful greeting for the whole 40 days after the Feast of Resurrection of Christ.

In Russia, Ukrain, Belarus and the surrounding Slavonic lands there is this tradition that the greeting is repeated 3 times as an interaction between person A and person B, for example.

Person A (3 times) greets:
ХРИСТОС ВОСКРЕСЕ = CHRIST IS RISEN !!!
Person B (3 times) answers:
ВОЙСТИНУ ВОСКРЕСЕ = TRULY HE IS RISEN !!!

Resurrection-of-Christ-Velikden-orthodox-holy-icon

Below is a good list with Paschal Resurrection Greeting in multiple languages, for those who has curious polyglot minds who want to learn few words in different languages.

Indo-European languages

Greek: Χριστὸς ἀνέστη! Ἀληθῶς ἀνέστη! (Khristós anésti! Alithós anésti!)

Voskresenie-Gospoda-Nashego-Iisusa-Hrista-Mosaic

Slavic languages

Church Slavonic: Хрїсто́съ воскре́се! Вои́стинꙋ воскре́се! (Xristósŭ voskrése! Voístinu voskrése!)

Belarusian: Хрыстос уваскрос! Сапраўды ўваскрос! (Chrystos uvaskros! Sapraŭdy ŭvaskros!)

Bulgarian: Христос воскресе! Воистину воскресе! (Khristos voskrese! Voistinu voskrese!), as if in Church Slavonic; Христос възкресе! Наистина възкресе! (Khristos vâzkrese! Naistina vâzkrese!) in Modern Bulgarian

Croatian: Krist uskrsnu! Uistinu uskrsnu!

: Kristus vstal z mrtvých! Vpravdě vstal z mrtvých!

Macedonian: Христос воскресе! Навистина воскресе! (Hristos voskrese! Navistina voskrese!), traditional; or Христос воскресна! Навистина воскресна! (Hristos voskresna! Navistina voskresna!)

Polish: Chrystus zmartwychwstał! Prawdziwie zmartwychwstał!

Russian: Христос воскрес(-е)! Воистину воскрес(-е)! (Khristos voskres(-е)! Voistinu voskres(-е)!) (the version with -e is in Church Slavonic, one without it is in modern Russian; both are widely used)

Rusyn: Хрістос воскрес! Воістину воскрес! (Hristos voskres! Voistynu voskres!)

Serbian: Христос васкрсе! Ваистину васкрсе! (Hristos vaskrse! Vaistinu vaskrse!) or Христос воскресе! Ваистину воскресе! (Hristos voskrese! Vaistinu voskrese!)

Slovak: Kristus vstal z mŕtvych! Skutočne vstal (z mŕtvych)! (though the Church Slavonic version is more often used)

Slovene: Kristus je vstal! Zares je vstal!

Ukrainian: Христос воскрес! Воістину воскрес! (Khrystos voskres! Voistynu voskres!)

Tosk Albanian: Krishti u ngjall! Vërtet u ngjall!

Armenian

Western Armenian: Քրիստոս յարեա՜ւ ի մեռելոց: Օրհնեա՜լ է Յարութիւնն Քրիստոսի: (Krisdos haryav i merelotz! Orhnyal e Haroutyunen Krisdosi!)

eastern dialect, Քրիստոս հարյա՜վ ի մեռելոց: Օրհնյա՜լ է Հարությունը Քրիստոսի: (Khristos haryav i merelotz! Orhnyal e Harouthyoune Khristosi!); literally "Christ is risen! Blessed is the resurrection of Christ!")
 

Christus-Resurrexit-Latin-icon-Christ_Resurrection

 

Germanic languages

 

Anglic languages

Scots: Christ has ryssyn! Hech aye, he his ain sel!

English: Christ is risen! He is risen indeed! Or Christ is risen! Truly, he is risen!

Old English: Crist is ārisen! Hē is sōþlīċe ārisen!

Middle English: Crist is arisen! Arisen he sothe!

Danish: Kristus er opstanden! Sandelig Han er Opstanden!

West Frisian: Kristus is opstien! Wis is er opstien!

German: Christus ist auferstanden! Er ist wahrhaft auferstanden! or Der Herr ist auferstanden! Er ist wahrhaftig auferstanden!

Icelandic: Kristur er upprisinn! Hann er sannarlega upprisinn!

Faroese: Kristus er upprisin! Hann er sanniliga upprisin!

Low Franconian languages

Dutch: Christus is opgestaan! Hij is waarlijk opgestaan! (Netherlands) or Christus is verrezen! Hij is waarlijk verrezen! (Belgium)

Afrikaans: Christus het opgestaan! Hy het waarlik opgestaan!

Norwegian

Bokmål: Kristus er oppstanden! Han er sannelig oppstanden!

Nynorsk: Kristus er oppstaden! Han er sanneleg oppstaden!

Swedish: Kristus är uppstånden! Han är sannerligen uppstånden!

Italic languages

Latin: Christus resurrexit! Resurrexit vere!

Romance languages

Aromanian: Hristolu anyie! Di alihea anyie!

Resurrection-of-Christ-Coptic-christian-icon

Catalan: Crist ha ressuscitat! Veritablement ha ressuscitat!

French: Le Christ est ressuscité ! En vérité il est ressuscité! Or Le Christ est ressuscité ! Vraiment il est ressuscité !

Galician: Cristo resucitou! De verdade resucitou!

Italian: Cristo è risorto! È veramente risorto!

Portuguese: Cristo ressuscitou! Em verdade ressuscitou! or Cristo ressuscitou! Ressuscitou verdadeiramente!

Arpitan: Lo Crist es ressuscitat! En veritat es ressuscitat!

Romanian: Hristos a înviat! Adevărat a înviat!

Romansh: Cristo es rinaschieu! In varded, el es rinaschieu!

Sardinian: Cristu est resuscitadu! Aberu est resuscitadu!

Sicilian: Cristu arrivisciutu esti! Pibbiru arrivisciutu esti!

Spanish: ¡Cristo resucitó! ¡En verdad resucitó!

Walloon: Li Crist a raviké! Il a raviké podbon!

Baltic languages

Latvian: Kristus (ir) augšāmcēlies! Patiesi (viņš ir) augšāmcēlies!

Lithuanian: Kristus prisikėlė! Tikrai prisikėlė!
 

Celtic languages

Goidelic languages

Old Irish: Asréracht Críst! Asréracht Hé-som co dearb!

Irish: Tá Críost éirithe! Go deimhin, tá sé éirithe!

Manx: Taw Creest Ereen! Taw Shay Ereen Guhdyne!

Scottish Gaelic: Tha Crìosd air èiridh! Gu dearbh, tha e air èiridh!

 

Brythonic languages

 

Breton:Dassoret eo Krist! E wirionez dassoret eo!

Cornish: Thew Creest dassorez! En weer thewa dassorez!

Welsh: Atgyfododd Crist! Yn wir atgyfododd!

Indo-Iranian languages

Ossetian:Чырысти райгас! Æцæгæй райгас! Or бæлвырд райгас! (Ḱyrysti rajgas! Æcægæj rajgas or bælvyrd rajgas!)

Persian: مسیح برخاسته است! به راستی برخاسته است!‎ (Masih barkhaste ast! Be rasti barkhaste ast!)

Hindi: येसु मसीह ज़िन्दा हो गया है! हाँ यक़ीनन, वोह ज़िन्दा हो गय یسوع مسیح زندہ ہو گیا ہے! ہاں یقیناً، وہ زندہ ہو گیا ہے!‎ (Yesu Masīh zindā ho gayā hai! Hā̃ yaqīnan, voh zindā ho gayā hai!)

Marathi: Yeshu Khrist uthla ahe! Kharokhar uthla ahe!

Abkhazian: Kyrsa Dybzaheit! Itzzabyrgny Dybzaheit!
 

Jesus-Christ-Resurrected-arabic-coptic-icon

Afro-Asiatic languages

 

Semitic languages

 

Standard Arabic: المسيح قام! حقا قام!‎ (al-Masīḥ qām! Ḥaqqan qām!) or المسيح قام! بالحقيقة قام! (al-Masīḥ qām! Bi-l-ḥaqīqati qām!)

Aramaic languages

 

Classical Syriac: ܡܫܝܚܐ ܩܡ! ܫܪܝܪܐܝܬ ܩܡ!‎ (Mshiḥa qām! sharīrāīth qām! or Mshiḥo Qom! Shariroith Qom!)

Assyrian Neo-Aramaic: ܡܫܝܚܐ ܩܡܠܗ! ܒܗܩܘܬܐ ܩܡܠܗ!‎ (Mshikha qimlih! bhāqota qimlih!)

Turoyo: ܡܫܝܚܐ ܩܝܡ! ܫܪܥܪܐܝܬ ܩܝܡ!‎ (Mshiḥo qāyem! Shariroith qāyem!)

 

East African languages

 

Tigrinya: Christos tensiou! Bahake tensiou!

Amharic: Kristos Tenestwal! Bergit Tenestwal!

Hebrew: המשיח קם! באמת קם!‎ (Hameshiach qam! Be'emet qam!)

Maltese: Kristu qam! Huwa qam tassew! or Kristu qam mill-mewt! Huwa qam tassew!

Egyptian

Coptic: (Pi'Christos aftonf! Khen oumetmi aftonf!)

Judeo-Berber: Lmasih yahye-d ger lmeytin! Stidet yahye-d ger lmeytin!


Dravidian languages

Tamil: கிறிஸ்து உயிர்த்தெழுந்தார், மெய்யாகவே அவர் உயிர்த்தெழுந்தார்.

Malayalam: ക്രിസ്തു ഉയിര്ത്തെഴുന്നേറ്റു! തീര്ച്ചയായും ഉയിര്ത്തെഴുന്നേറ്റു! (Christu uyirthezhunnettu! Theerchayayum uyirthezhunnettu!)
 

Eskimo–Aleut languages

Aleut: Kristusaaq Aglagikuk! Angangulakan Aglagikuk!

Pacific Gulf Yupik: Kristusaq ungwektaq! Pichinuq ungwektaq!

Central Yupik: Kristuussaaq unguirtuq! Ilumun unguirtuq!

Mayan languages

Tzotzil: Icha'kuxi Kajvaltik Kristo! Ta melel icha'kuxi!

Tzeltal: Cha'kuxaj Kajwaltik Kristo! Ta melel cha'kuxaj!

Christ-resurrection-Anastasis

 

Austronesian languages

Malayo-Polynesian

Batak: Tuhan nunga hehe! Tutu do ibana hehe!

Carolinian: Lios a melau sefal! Meipung, a mahan sefal!

Cebuano: Nabanhaw Si Kristo! Nabanhaw gayud!

Waray: Hi Kristo nabanwaw! Matuod nga Hiya nabanhaw!

Chamorro: La'la'i i Kristo! Magahet na luma'la' i Kristo!

Fijian: Na Karisito tucake tale! Io sa tucake tale!

Filipino: Nabuhay muli Si Kristo! Nabuhay talaga!

Hawaiian: Ua ala hou ʻo Kristo! Ua ala ʻiʻo nō ʻo Ia!

Indonesian: Kristus telah bangkit! Dia benar-benar telah bangkit!

Kapampangan: Y Kristû sinûbli yáng mèbié! Sinûbli ya pin mèbié!

Malagasy: Nitsangana tamin'ny maty i Kristy! Nitsangana marina tokoa izy!

Cook Islands Māori: Kuo toetu’u ‘ae Eiki! ‘Io kuo toetu’u mo’oni!

 Austroasiatic languages: Mon-Khmer

: Preah Christ mean preah choan rous leong vinh! trung mean preah choan rous leong vinh men!

 Vietnamese

: Chúa Ki-tô đã sống lại! Ngài đã sống lại thật!

 Thai

Thai: พระคริสต์เป็นขึ้นจากความตาย! or พระคริสต์ทรงกลับคืนพระชนม์ชีพ!

Basque

Basque: Cristo Berbiztua! Benetan Berbiztua!

Japanese

Japanese: ハリストス復活!実に復活! (Harisutosu fukkatsu! Jitsu ni fukkatsu!)

 Korean

Korean그리스도 부활하셨네! 참으로 부활하셨네! (Geuriseudo buhwalhasyeonne! Chameuro buhwalhasyeonne!)

 Na-Dené languages

Athabaskan languages

Navajo: Christ daaztsą́ą́dę́ę́ʼ náádiidzáá! Tʼáá aaníí daaztsą́ą́dę́ę́ʼ náádiidzáá!

Tlingit: Xristos Kuxwoo-digoot! Xegaa-kux Kuxwoo-digoot!

Niger–Congo languages

: Kristo Ajukkide! Kweli Ajukkide!

Swahili: Kristo Amefufuka! Amefufuka kweli kweli!

Gikuyu: Kristo ni muriuku! Ni muriuku nema!

Quechuan languages

Quechua: Cristo causarimpunña! Ciertopuni causarimpunña!

Mongolic languages

Classical Mongolian: Есүс дахин амилсан, Тэр үнэхээр амилсан! (Yesus dahin amilsan, ter uneheer amilsan)

Turkic languages

Turkish: Mesih dirildi! Hakikaten dirildi!

Uyghur: ‫ئەيسا تىرىلدى! ھەقىقەتىنلا تىرىلدى!‬‎ (Əysa tirildi! Ⱨəⱪiⱪətinla tirildi!)

Azerbaijani: Məsih dirildi! Həqiqətən dirildi!

Chuvash: Христос чĕрĕлнĕ! Чăн чĕрĕлнĕ! (Hristos čĕrĕlnĕ! Čyn čĕrĕlnĕ!)

Khakas: Христос тірілді! Сыннаң тірілді! (Hristos tíríldí! Sınnañ tíríldí!)

Uzbek: Масих тирилди! Хақиқатдан тирилди! (Masih tirildi! Haqiqatdan tirildi!)

Sino-Tibetan languages

Chinese: 基督復活了!他確實復活了! (Jīdū fùhuó-le! Tā quèshí fùhuó-le!) or 耶穌復活了,真的他復活了! (Yēsū fùhuó-le, Zhēnde tā fùhuó-le!)

Uralic languages

Estonian: Kristus on üles tõusnud! Tõesti on üles tõusnud!

Finnish: Kristus nousi kuolleista! Totisesti nousi!

Hungarian: Krisztus feltámadt! Valóban feltámadt!

Karelian: Hristos nouzi kuollielois! Tovessah nouzi!

Glorious-Resurrection-of-Jesus-Christ-icon

Constructed languages

International auxiliary languages

Esperanto: Kristo leviĝis! Vere Li leviĝis!

Ido: Kristo riviveskabas! Ya Il rivivesakabas!

Interlingua: Christo ha resurgite! Vermente ille ha resurgite! or Christo ha resurrecte! Vermente ille ha resurrecte!

Quenya: (Hristo Ortane! Anwave Ortanes!)

Klingon: Hu'ta' QISt! Hu'bejta'!