Posts Tagged ‘second time’

How to Copy large data directories between 2 Linux / Unix servers without direct ssh / ftp access between server1 and server2 other by using SSH, TAR and Unix pipes

Monday, April 27th, 2015

Reading Time: 3minutes

how-to-copy-large-data-directories-between-2-linux-unix-servers-without-direct-ssh-ftp-access-btween-each-other

In a Web application data migration project, I've come across a situation where I have to copy / transfer 500 Gigabytes of data from Linux server 1 (host A) to Linux server 2 (host B). However the two machines doesn't have direct access to each other (via port 22) for security reasons and hence I cannot use sshfs to mount remotely host dir via ssh and copy files like local ones.

As this is a data migration project its however necessery to migrate the data finding a way … Normal way companies do it is to copy the data to External Hard disk storage and send it via some Country Post services or some employee being send in Data center to attach the SAN to new server where data is being migrated However in my case this was not possible so I had to do it different.

I have access to both servers as they're situated in the same Corporate DMZ network and I can thus access both UNIX machines via SSH.

Thanksfully there is a small SSH protocol + TAR archiver and default UNIX pipe's capabilities hack that makes possible to transfer easy multiple (large) files and directories. The only requirement to use this nice trick is to have SSH client installed on the middle host from which you can access via SSH protocol Server1 (from where data is migrated) and Server2 (where data will be migrated).

If the hopping / jump server from which you're allowed to have access to Linux  servers Server1 and Server2 is not Linux and you're missing the SSH client and don't have access on Win host to install anything on it just use portable mobaxterm (as it have Cygwin SSH client embedded )

Here is how:
 

jump-host:~$ ssh server1 "tar czf – /somedir/" | pv | ssh server2 "cd /somedir/; tar xf


As you can see from above command line example an SSH is made to server1  a tar is used to archive the directory / directories containing my hundred of gigabytes and then this is passed to another opened ssh session to server 2  via UNIX Pipe mechanism and then TAR archiver is used second time to unarchive previously passed archived content. pv command which is in the middle is not obligitory though it is a nice way to monitor status about data transfer like below:
 

500GB 0:00:01 [10,5MB/s] [===================================================>] 27%


P.S. If you don't have PV installed install it either with apt-get on Debian:

 

debian:~# apt-get install –yes pv

 

Or on CentOS / Fedora / RHEL etc.

 

[root@centos ~]# yum -y install pv

 

Below is a small chunk of PV manual to give you better idea of what it does:

NAME
       pv – monitor the progress of data through a pipe

SYNOPSIS
       pv [OPTION] [FILE]…
       pv [-h|-V]

DESCRIPTION
       pv  allows  a  user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage
       completed (with progress bar), current throughput rate, total data transferred, and ETA.

       To use it, insert it in a pipeline between two processes, with the appropriate options.  Its standard input will be passed
       through to its standard output and progress will be shown on standard error.

       pv  will  copy  each  supplied FILE in turn to standard output (- means standard input), or if no FILEs are specified just
       standard input is copied. This is the same behaviour as cat(1).

       A simple example to watch how quickly a file is transferred using nc(1):

              pv file | nc -w 1 somewhere.com 3000

       A similar example, transferring a file from another process and passing the expected size to pv:

              cat file | pv -s 12345 | nc -w 1 somewhere.com 3000


Note that with too big file transfers using PV will delay data transfer because everything will have to pass through another 2 pipes, however for file transfers up to few gigabytes its really nice to include it.

If you only need to transfer huge .tar.gz archive and you don't bother about traffic security (i.e. don't care whether transferred traffic is going through encrypted SSH tunnel and don't want to put an overhead to both systems for encrypting the data and you have some unfiltered ports between host 1 and host 2 you can run netcat on host 2 to listen for connections and forward .tar.gz content via netcat's port like so:
 

linux2:~$ nc -l -p 12345 > /path/destinationfile
linux2:~$ cat /path/sourcfile | nc desti.nation.ip.address 12345


Another way to transfer large data without having connection with server1 and server2 but having connection to a third host PC is to use rsync and good old SSH Tunneling, like so:
 

jump-host:~$ ssh -R 2200:Linux-server1:22 root@Linux-server2 "rsync -e 'ssh -p 2200' –stats –progress -vaz /directory/to/copy root@localhost:/copy/destination/dir"

Saint George’s day in Pomorie Monastery Bulgaria

Thursday, May 6th, 2010

Reading Time: 4minutes
St. George Bulgarian icon

I’m in pomorie monastery right now. Pityle pomorie’s monastery’s website can only be accessed in Russian and Bulgarian language.
Pomorie Monastery is named after the greatly honoured saint George, he is highy honoured here in Bulgaria as well as in many countries in the orthodox christian world, he is also venerated in some other non-orthodox countries one of which for instance is England.
Some of the monks here in the monastery used to say it’s St. George himself who invited me to the monastery for a blessing since I myself am named after the saint.
Many people from Pomorie and near Burgas’s region and even from afar are gathered to commemorate St. George’s Martyrdom and celebrate the Saint George’s day . It’s the second time I’m in Pomorie’s monastery which by the way is a Male monastery. I’m really happy that such a cloisters still exists in this troubles times in sense of spirituality.
The place is really peaceful and the fact that it’s georgraphically located near the sea makes it a must see destination if you’re visiting Bulgaria or travelling through the country.
The history of the monastery is also very interesting. Currently the brotherhood includes 6 monks. The monastery abbot is a really kind and gentle man and the brothers are united in comparison to the observed dividement between monks walking the way of salvation in spiritual abbeys nowadays.
The main reason people are coming on the feast of st. George is to beg for the prayer intercession of the Saint for us the sinners in front of God, that God has mercy and forgies our trespasses and have mercy on us.
Each eart Saint George takes place here in Bulgaria on Sixth of May (06.05) and is one of the greatest christian as well as secular celebrations in Bulgaria. The 6th of Many is also an official holidays within the country and the official feast of the bulgarian army.
St. George is considered one of the greatest Christian saints in the Orthodox, world and even more venerated in Bulgaria.
It’s a common practice in Bulgaria as well as in Greece that spiritual abbeys or Churches are named after St. George.
What makes St. George even more special for us the fact that the Bulgarian Orthodox Church possesses a whole monastery named after st. George located in Mounth Athos. You can read more about St. George (Zograf) Bulgarian Monastery located in Mount Athos here
St. George’s monastery in Mount Athos is named Zograf after a miraculous icon located in the Monastery which dates back to the 14th century.
St. George Fanuilska miraculous icon Zograf Monastery
Saint George Zograf Miraculous icon owned by Zograf’s Monastery – Holy Mount Athos

The Holy Lord has completed many miracles through the icon of st. George Zograf. Many people received physical or spiritual healing, whilst others has received according to their prayers when they approached the icon as great relic to honour God.An interesting fact is that Pomorie Monastery also possesses a great relic a miraculous icon of st. George

Saint George Pomorie Monastery miraculous icon, relic
a miraculous icon of St. George Pomorie

In the center of Pomorie Monastery is located a Holy Fountain St. George. The wholy history of the monastery is related to the Holy Fountain.
According to bulgarian orthodox church tradition the Monastery was established by a Turkish bey in the 13th or 14th century. In that time the bay got an incurable sickness. He tried everything a man could try to get a relief or a cure. He went to a physician, he tried herbs, he tried going and praying to Allah in the mosque an imam praied over him to beg for Allah’s blessing and healing etc …
But nothing helped. One night the bey had a dream that a holy fountain is springing out of his house yard.
When he awoke from his sleep he went and digged on the same place where the holy spring has emerged. And oh miracle ! A spring emerged. He drinked from the water and got healed from his illness.
In the pit where he digged he found an image of St. George which is currently embedded in one of the walls of the tower of the holy spring in Pomorie’s monastery.
Afterwards the bey according to God’s revelation to establish Pomorie’s Monastery while all his family members accepted monastic life.
The bey has give away all hid possesseions (land, money) etc. to the monastery just established.
In the years since then the monastery has been destroyed once and rebuilt, many people has received a relief or healing while drinking from the Holy Fountain.
This year God has blessed the monstery even more abundantly. The abbot and one of the monks has temporary taken some Saint remains and brough them for pilgrimage in the monastery.
The saint remains are of St. John Chrysostom , St. Gregory the Theologian , st. Gregory Palamas
Some of the other relics which are available for pilgrimage in Pomorie monastery until 10th of May are st. remains of Glinsk elder hermits as well as

Saint Martyr Dasius icon
St. martyr Dasius who was tortured and slained here in Bulgaria for his faith in our Lord and Saviour Jesus Christ.

The Living of Saint Peter and Saint Fevronia – a fascinating Russian cartoon retelling the saints story

Monday, March 26th, 2012

Reading Time: 4minutes

Saint Peter and Fevronia Orthodox Christian saints protector of happy family, love and blessed marriage

The anime Living Story of Saint Peter and Saint Fevronia is a modern story remake of an ancient Church living of two bright Russian saints.
The official movie genre is orthodox christian kids animation movie.
The 14 minutes cartoon re-tolds the living story of St. Peter and st, Fevronia in a playful and entertaining way like for kids or youngsters.
Nomatter the movie primary target audience is children, the cartoon is great to see for adults people as well :).
The movie genre is orthodox christian kids animation movie.
The plot is based on a true (historic record) story of two saints venerated each year across Orthodox Churches around the world.

The original story I watched was a Bulgarian translation from Russian. But since I found it to be so valuable, I look for a translated video and got one in youtube.Take 14 minutes break and watch it, I'm sure you will like it so much, that probably give it a second time glimpse alone or with your wife, children or girlfriend.

Unfortunately, the english title is mis-translated as it says "Tale" and not "Story" and there is difference in meaning between this two words.
It is not tale as tale is made up story and this is not a made story but a story based on the two saints who lived in the end of 12 and beginning of the thirteen century.
Here is in short the real Church living:
The two saints were living in Murom Russia. Peter was prince and Fevronia a poor maid a daughter of a beekeeper who made his family living by collecting wild honey in the forest.

Prince Peter was striken by a severe sickness and in a vision it was revealed to him, that the only one that can cure him is Fevronia (a village maid living in the village of Laskovo Russia. The prince went to her and since he saw she is a pious, good and wisdom rich maid promised her, if she manage to heal him to take her as a bride to his place.
By her warm prayers to God and herbs, Fevronia succeeded in healing the prince sickness. Being fully restored st.Peter changed his mind and wanted to break his promise to marry her, cause the young made not part of the aristocratic Russian society. He didn't yet reached his home and the sickness, came back. This time with a deep repentance, he came back to Fevronia and she cured him again. Then the prince merried her and made her a princess of Murom
 

St. Peter and Fevronia Orthodox Church saints protector of marriage

However there love in Christ had to went through high temptations. Once the couple married, the prince proud boyars requested the prince to leave his new bride, as they didn't wanted to accept a simple girl as Fevronia will be governing them. Being in uneasy situation prince Peter prefered to leave his governing power and castle but to stay with his life. Together by boat by the near river Oka they left the kingdom. Soon after Gods wrath came Murom because of people's rebellion and the people requested the chased prince family to be restored to power.
Pushed by the peasents, the boyars bringed back the couple to power. The two saints governed their kingdom with great wisdom, love and mercy to the people.

In their old age the decided to become monks in separate monasteries. St. Peter received his new monk name David and princess Fevronia took the nun name Evphrosia (Evfrosia). Even living a sepate monk / nun life the couple continued having a deep love to each other and asked God to take them from this earthly life on the same date. God answered their prayer granting them to depart this earth on the same day in the same hour!
Like in life even in death people tried to separate them.
Fevronia was put in a coffin in the nun monastery, where Peter was prepared for a monk funeral in the man monastery and they put them in separate graves.
In the morning the graves were empty and their bodies were found buried together in one grave. People realized it is Gods will they are buried together and left them buried together. Today the incorruptable bodies of the two saints can be seen and venerated in Holy Trinity's nun monastery in the town of Murom Russia. St. Peter and St. Fevronia are considered patron of the Christian marriage, couple's love and family happiness.
It is common that many young people are, asking for the two saints prayer intercession in front of God for getting a good spouse in life and good marriage.

Also the two saints are oftenly asked for prayer for improving a marriege bindings.
The two saints feast day is like the Orthodox Antipode of the Roman Catholic feast of couples in love – St. Valentine.

As you see, St. Peter and St. Fevronia living is full of wisdom and true spirituality, and there is plenty we the modern disbelieving people can learn from it.
Let God by the two saints holy prayers have mercy on us.

Weekend with a lot of friends

Monday, April 7th, 2008

Reading Time: 2minutes
I spend the weekend with a lot of friends. God has prepared another great time for me. In Friday I was out with Alex, and we had a long discussion for who is God, because he thinks he can know the Lord without our Lord and Saviour Jesus Christ which is completely wrong and impossible like it’s written how our Lord says “I and the Father are one” and again “Noone comes to the Father except through me“. At friday I was out with Static a friend and brother in Christ 🙂 We went to the school where he lives “Dimitar Talev”. This is the second time when I visit this school but it’s so colourful and I like it so much that I wished for a second to be pupil again :). In Sunday I went to Liturgy as often I went to bed very late in Saturday so I wasn’t able to wake in the morning and I went to the Church service somewhere around 9:30. The Priest’s preach was about the gospel’s where our Lord Jesus freed and healed a boy who was very often abused by wicked spirits, he also mentioned how some of the ppl fasts only abstaining from food and how this ain’t fast at all. He gave us interesting example how the animals who ate meet are cruel and animals who eat grass are very meek and pure. For example he said look into the fox, wolf they eat meet and they’re fierce animals whether animals like sheeps and rabbits eat only grass and are mostly good. He also mentioned that the st. Ioan Zlatoust(John / James the Goldenmouth) said that the fasting gives wings to the soul. In the church I saw Stelio the man who introduced me to the Orthodox Faith and after the Service we spend some great time in the park talking about the past during the communist regime here in Bulgaria, also the current state in the country etc. etc. In 12:30 we had previously arranged meeting with two of my colleagues from the College. We had to make the SPSS project which we have to complete and pass to the teacher today. Thanks God everything just went smoothly. And we had to make just a little text to the project in order to complete it. In 09:00 we went to the Fountain with Vlado (hellpain), because he was here for the weekend. Now just after I came home I prayed I take a shower. And I intend to read a bit from the bible and go to sleep.

END—–

Opera!

Saturday, May 10th, 2008

Reading Time: < 1minute
Yesterday I and Alex went to an Opera. We watched Opera by Giuseppe Verdi called “La Traviata”.It’s the second time I see this opera, the first time was when I was still a puppil.Pretty awesome opera the actors were really good. And this classic music! Awesome!After the Opera performance I phoned to Ivo a.k.a. (Order)we went to a sort of pub which wasextremely cheap considering the fact it was near the center and the food and drinks were really good.After all this we had a walk around in Varna and hitted the road back to Dobrich.Kimmo My finish colleague is leaving for Finland and was organizing farewell party.Unfortunately because of the Opera I was unable to go to the party. Yesterday was a really niceday considering that nice opera show. The bad news for me yesterday was that I didn’t passedthe written exam in German and I have to go to the re-sit at Tuesday. With the Lord’s helpI hope I would pass it 🙂 Also I’m going to start for the Accounting exam because It’s at Monday.END—–