Posts Tagged ‘binary files’

How to move transfer binary files encoded with base64 on Linux with Copy Paste of text ASCII encoded string

Monday, October 25th, 2021

base64-encode-decode-binary-files-to-transfer-between-servers-base64-artistic-logo

If you have to work on servers in a protected environments that are accessed via multiple VPNs, Jump hosts or Web Citrix and you have no mean to copy binary files to your computer or from your computer because you have all kind of FTP / SFTP or whatever Data Copy clients disabled on remote jump host side or CITRIX server and you still are looking for a way to copy files between your PC and the Remote server Side.
Or for example if you have 2 or more servers that are in a special Demilitarized Network Zones ( DMZ ) and the machines does not have SFTP / FTP / WebServer or other kind of copy protocol service that can be used to copy files between the hosts and you still need to copy some files between the 2 or more machines in a slow but still functional way, then you might not know of one old school hackers trick you can employee to complete the copy of files between DMZ-ed Server Host A lets say with IP address (192.168.50.5) -> Server Host B (192.168.30.7). The way to complete the binary file copy is to Encode the binary on Server Host A and then, use cat  command to display the encoded string and copy whole encoded cat command output  to your (local PC buffer from where you access the remote side via SSH via the CITRIX or Jump host.). Then decode the encoded file with an encoding tool such as base64 or uuencode. In this article, I'll show how this is done with base64 and uuencode. Base64 binary is pretty standard in most Linux / Unix OS-es today on most Linux distributions it is part of the coreutils package.
The main use of base64 encoding to encode non-text Attachment files to Electronic Mail, but for our case it fits perfectly.
Keep in mind, that this hack to copy the binary from Machine A to Machine B of course depends on the Copy / Paste buffer being enabled both on remote Jump host or Citrix from where you reach the servers as well as your own PC laptop from where you access the remote side.

base64-character-encoding-string-table

Base64 Encoding and Decoding text strings legend

The file copy process to the highly secured PCI host goes like this:
 

1. On Server Host A encode with md5sum command

[root@serverA ~]:# md5sum -b /tmp/inputbinfile-to-encode
66c4d7b03ed6df9df5305ae535e40b7d *inputbinfile-to-encode

 

As you see one good location to encode the file would be /tmp as this is a temporary home or you can use alternatively your HOME dir

but you have to be quite careful to not run out of space if you produce it anywhere 🙂

 

2. Encode the binary file with base64 encoding

 [root@serverB ~]:# base64 -w0 inputbinfile-to-encode > outputbin-file.base64

The -w0 option is given to disable line wrapping. Line wrapping is perhaps not needed if you will copy paste the data.

base64-encoded-binary-file-text-string-linux-screenshot

Base64 Encoded string chunk with line wrapping

For a complete list of possible accepted arguments check here.

3. Cat the inputbinfile-to-encode just generated to display the text encoded file in your SecureCRT / Putty / SuperPutty etc. remote ssh access client

[root@serverA ~]:# cat /tmp/inputbinfile-to-encode
f0VMRgIBAQAAAAAAAAAAAAMAPgABAAAAMGEAAAAAAABAAAAAAAAAACgXAgAAAAAAAAAAA
EAAOAALAEAAHQAcAAYAAAAEAAA ……………………………………………………………… cTD6lC+ViQfUCPn9bs

 

4. Select the cat-ted string and copy it to your PC Copy / Paste buffer


If the bin file is not few kilobytes, but few megabytes copying the file might be tricky as the string produced from cat command would be really long, so make sure the SSH client you're using is configured to have a large buffer to scroll up enough and be able to select the whole encoded string until the end of the cat command and copy it to Copy / Paste buffer.

 

5. On Server Host B paste the bas64 encoded binary inside a newly created file

Open with a text editor vim / mc or whatever is available

[root@serverB ~]:# vi inputbinfile-to-encode

Some very paranoid Linux / UNIX systems might not have even a normal text editor like 'vi' if you happen to need to copy files on such one a useful thing is to use a simple cat on the remote side to open a new File Descriptor buffer, like this:

[root@server2 ~]:# cat >> inputbinfile-to-encode <<'EOF'
Paste the string here

 

6. Decode the encoded binary with base64 cmd again

[root@serverB ~]:# base64 –decode outputbin-file.base64 > inputbinfile-to-encode

 

7. Set proper file permissions (the same as on Host A)

[root@serverB ~]:#  chmod +x inputbinfile-to-encode

 

8. Check again the binary file checksum on Host B is identical as on Host A

[root@serverB ~]:# md5sum -b inputbinfile-to-encode
66c4d7b03ed6df9df5305ae535e40b7d *inputbinfile-to-encode

As you can md5sum match on both sides so file should be OK.

 

9. Encoding and decoding files with uuencode


If you are lucky and you have uuencode installed (sharutils) package is present on remote machine to encode lets say an archived set of binary files in .tar.gz format do:

Prepare the archive of all the files you want to copy with tar on Host A:

[root@Machine1 ~]:#  tar -czvf /bin/whatever /usr/local/bin/htop /usr/local/bin/samhain /etc/hosts archived-binaries-and-configs.tar.gz

[root@Machine1 ~]:# uuencode archived-binaries-and-configs.tar.gz archived-binaries-and-configs.uu

Cat / Copy / paste the encoded content as usual to a file on Host B:

Then on Machine 2 decode:

[root@Machine2 ~]:# uuencode -c < archived-binaries-and-configs.tar.gz.uu

 

Conclusion


In this short method I've shown you a hack that is used often by script kiddies to copy over files between pwn3d machines, a method which however is very precious and useful for sysadmins like me who has to admin a paranoid secured servers that are placed in a very hard to access environments.

With the same method you can encode or decode not only binary file but also any standard input/output file content. base64 encoding is quite useful stuff to use also in bash scripts or perl where you want to have the script copy file in a plain text format . Datas are encoded and decoded to make the data transmission and storing process easier. You have to keep in mind always that Encoding and Decoding are not similar to encryption and decryption as encr. deprytion gives a special security layers to the encoded that. Encoded data can be easily revealed by decoding, so if you need to copy between the servers very sensitive data like SSL certificates Private RSA / DSA key, this command line utility tool better to be not used for sesitive data copying.

 

 

Editting binary files in console and GUI on FreeBSD and Linux

Thursday, April 26th, 2012

I’ve recently wanted to edit one binary file because there was compiled in the binary a text string with a word I didn’t liked and therefore I wanted to delete. I know I can dig in the source of the proggie with grep and directly substitute my “unwatned text” there but I wanted to experiment, and see what kind of hex binary text editors are for Free OSes.
All those who lived the DOS OS computer era should certainly remember the DOS hex editors was very enjoyable. It was not rare case, where in this good old days, one could simply use the hex editor to “hack” the game and add extra player lives or modify some vital game parameter like put himself first in the top scores list. I even remember some DOS programs and games was possible to be cracked with a text editor … Well it was times, now back to current situation as a Free Software user for the last 12 years it was interesting to see what is the DOS hexeditor like alternatives for FreeBSD and Linux and hence in this article I will present my findings:

A quick search in FreeBSD ports tree and Debian installable packages list, I’ve found a number of programs allowing one to edit in console and GUI binary files.

Here is a list of the hex editors I will in short review in this article:

  • hexedit
  • dhex
  • chexedit
  • hte
  • hexer
  • hexcurse
  • ghex
  • shed
  • okteta
  • bless
  • lfhex

1. hexedit on Linux and BSD – basic hex editor

I’ve used hexedit already on Linux so I’ve used it some long time ago.

My previou experience in using hexedit is not too pinky, I found it difficult to use on Redhat and Debian Linux back in the day. hexedit is definitely not a choice of people who are not “initiated” with hex editting.
Anyways if you want to give it a try you can install it on FreeBSD with:

freebsd# cd /usr/ports/editors/hexedit
freebsd# make install clean

On Debian the hexedit, install package is named the same so installation is with apt:

debian:~# apt-get –yes install hexedit

hexedit screenshot Debian Linux Squeeze

2. Hex editting with chexedit

I’ve installed chexedit the usual way from ports:

freebsd# cd /usr/ports/editors/chexedit
freebsd# make install clean

chexedit is using the ncurses text console library, so the interface is very similar to midnight commander (mc) as you see from below’s screenshot:

Chexeditor FreeBSD 7.2 OS Screenshot

Editting the binary compiled in string was an easy task with chexedit as most of the commands are clearly visible, anyways changing a certain text string contained within the binary file with some other is not easy with chexedit as you need to know the corresponding binary binary value representing each text string character.
I’m not a low level programmer, so I don’t know the binary values of each keyboard character and hence my competence came to the point where I can substitute the text string I wanted with some unreadable characters by simply filling all my text string with AA AA AA AA values…

chexedit on Debian is packaged under a deb ncurses-hexedit. Hence to install it on Deb run:

debian:~# apt-get –yes install ncurses-hexedit

Further on the binary to run chexedit on binary contained within ncurses-hexedit is:

debian:~# hexeeditor

3. Hex Editting on BSD and Linux with hte

Just after trying out chexedit, I’ve found about the existence of one even more sophisticated hexeditor console program available across both FreeBSD and Linux.
The program is called hte (sounds to me a bit like the Indian word for Elephant “Hatti” :))

hte is installable on Debian with cmd:

debian:~# apt-get install ht

On FreeBSD the port name is identical, so to install it I execed:

freebsd# cd /usr/ports/editors/hte
freebsd# make install clean

hte is started on Debian Linux (and presumably other Linux distros) with:

$ hte

On FreeBSD you need to run it with ht command:

freebsd# ht

You see how hte looks like in below screenshot:

ht has the look & feel like midnight commander and I found it easier to use than chexedit and hexeditor
4. hexer VI like interface for Linux

As I was looking through the available packages ready to install, I’ve tried hexer

debian:~# apt-get install –yes hexer

hexer does follow the same standard commands like VIM, e.g. i for insert, a for append etc.

Hexer Debian Linux vim like binary editor screenshot

It was interesting to find out hexer was written by a Bulgarian fellow Petar Penchev 🙂
(Proud to be Bulgarian)

http://people.freebsd.org/~roam/ – Petar Penchev has his own page on FreeBSD.org

As a vim user I really liked the idea, the only thing I didn’t liked is there is no easy way to just substitute a string within the binary with another string.

5. hexcurse another ncurses library based hex editor

On Deb install and run via:

debian:~# apt-get –yes install hexcurse
debian:~# hexcurse /usr/bin/mc

Hexcurse Debian Linux text binary editor screenshot

hexcurse is also available on FreeBSD to install it use cmd:

freebsd# cd /usr/ports/editors/hexcurse
freebsd# make install clean
….

To access the editor functions press CTRL+the first letter of the word in the bottom menu, CTRL+H, CTRL+S etc.
Something I disliked about it is the program search is always in hex, so I cannot look for a text string within the binaries with it.

6. ghex – Editting binary files in graphical environment

If you’re running a graphical environment, take a look at ghex. ghex is a gnome (graphical hex) editor.Installing ghex on Debian is with:

debian:~# apt-get –yes install ghex
….

To run ghex from terminal type:

debian:~# ghex2

GHex2 GNOME hex binary editor screenshot

To install ghex on FreeBSD (and I assume other BSDs), install via port:

freebsd# cd /usr/ports/editors/ghex
freebsd# make install clean

Gnome hex editor have plenty of tools, useful for developers to debug binary files.

Some nice tools one can find are under the the menus:

Windows -> Character Table

This will show a complete list of each keyboard sent character in ASCII, Hex, Decimal, Octal and Binary

Screenshot ghex Character table Debian Linux

Another useful embedded tool in ghex is:

Windows -> Type Convertion Dialog

Ghex type convertion dialog screenshot

Note that if you want to use the Type Convertion Dialog tool to find the representing binary values of a text string you will have to type in the letters one by one and save the output within a text file and later you can go and use the same editor to edit the text string within the binary file you like.

I’m not a programmer but surely for programmers or people who want to learn some binary counting, this 2 ghex edmebbed tools are surely valuable.

To conclude even though there are plenty of softwares for hex editting in Linux and BSD, none of them is not so easy to use as the old DOS hexdedit tool, maybe it will be a nice idea if someone actually rewrites the DOS tool and they package it for various free operating systems, I’m sure many people will find it helpful to have a 1:1 equivalent to the DOS tool.

7. Shed pico like interfaced hex editor

For people, who use pico / nano as a default text editor in Linux shed will probably be the editor of choice as it follows the command shortcuts of picoOn Deb based distros to install it run:

debian:~# apt-get install –yes shed

shed pico like hex binary editor Linux

Shed has no BSD port as of time of writting.8. Okteta a KDE GUI hex editor

For KDE users, I found a program called okteta. It is available for Deb based Linuxes as deb to install it:

debian:~# apt-get –yes install okteta

Screenshot Okteta Debian GNU / Linux Squeeze

As of time of writting this article there is no okteta port for BSDs.
Okteta has plenty of functions and even has more of a functions than ghexedit. Something distinctive for it is it supports opening multiple files in tabs.

9. lfhex a large file text editor

lfhex is said to be a large (binary) file text editor, I have not tested it myself but just run it to see how it looks like. I don’t have a need to edit large binary files too, but I guess there are people with such requirements too 🙂

lfhex - Linux The Large file hex editor

To install lfhex on Debian:

debian:~# apt-get install –yes lfhex

lfhex has also a FreeBSD port installable via:

freebsd# cd /usr/ports/editors/lfhex
freebsd# make install clean

10. Bless a GUI tool for editting large hex (binary) files

Here is the description directly taken from the BSD port /usr/ports/editors/bless

Bless is a binary (hex) editor, a program that enables you to edit files asa sequence of bytes. It is written in C# and uses the Gtk# bindings for theGTK+ toolkit.

To install and use ot on deb based Linuxes:

debian:~# apt-get install –yes bless
….

On BSD installation is again from port:

freebsd# cd /usr/ports/editors/bless
freebsd# make install clean
….

Something that makes bless, maybe more desirable choice for GUI users than ghex is its availability of tabs. Opening multiple binaries in tabs will be useful only to few heavy debuggers.

Bless GUI hex editor Debian Linux tabs opened screenshot

11. Ghextris – an ultra hard hacker tetris game 🙂

For absolute, hacker / (geeks), there is a tetris game called ghextris. The game is the hardest tetris game I ever played in my life. It requires more than regular IQ and a lot of practice if you want to become really good in this game.

To enjoy it:

debian:~# apt-get –yes install ghextris

Ultra hrad hardcore hackers game ghextris screenshot

Unfortunately there is no native port of ghextris for BSD (yet). Anyhow, it can be probably run using the Linux emulation or even compiled from source.
Well that’s all I found for hexedit-ing, I’ll be happy to hear if someone can give me some feedback on his favourite editor.

How to set up Path to .exe GNUWin32 binary files in Windows XP / Vista / 2003 / 2008 (Setting PATH to executables on Windows)

Tuesday, August 23rd, 2011

I’ve been working on a servers running Windows 2003 and Windows 2008 these days.
As I wanted to be more flexible on what I can do from the command line I decided to install GNUwin (provides port of GNU tools), most of which are common part of any Linux distribution).
Having most of the command line flexibility on a Windows server is a great thing, so I would strongly recommend GNUWin to any Windows server adminsitrator out there.

Actually it’s a wonderful thing that most of the popular Linux tools can easily be installed and used on Windows for more check GnuWin32 on sourceforge

One of the reasons I installed Gnuwin was my intention to use the good old Linux tail command to keep an eye interactive on the IIS server access log files, which by the way for IIS webserver are stored by default in C:WindowsSystem32LogFilesW3SVC1*.log

I’ve managed to install the GNUWin following the install instructions, not with too much difficulties. The install takes a bit of time, cause many packs containing different parts of the GNUWin has to be fetched.

To install I downloaded the GNUWin installer available from GNUWin32’s website and instructed to extracted the files into C:Program FilesGnuwin
Then I followed the install instructions suggestions, e.g.:

C:> cd c:Program FilesGnuWin
C:Program FilesGnuWin> download.bat
...
C:Program FilesGnuWin> install c:gnuwin32
...

After the installation was succesfully completed on the two Windows machines, both of which by the way are running 64 bit Windows, it was necessery to add the newly installed GNU .exe files to my regular cmd.exe PATH variable in order to be able to access the sed, tail and the rest of the gnuwin32 command line tools.

In order to add C:GnuWin32bin directory to the windows defined Command line Path , I had to do the following:

a. Select (Properties) for My Computer

Start (button) -> My Computer (choose properties)

b. Select the My Computer Advanced (tab)

Then, from the My Computer pane press on Advanced tab

c. Next press on Environment Variables

Windows environment variables screenshot

You see in above’s screenshot the Environment Variables config dialog, to add the new path location in System Variables sectiom, between the list I had to add the c:GNUwin32bin path locatiion. To find I pressed on Edit button scrolled down to find the Variable and hence added at the end of the long list defined paths.
After adding in GNUwin, the Windows path looks like this:

C:Program Files (x86)EWANAPI;C:WINDOWSsystem32;C:WINDOWS;C:WINDOWSSystem32Wbem;C:Program Files (x86)IntelNGSMSMPFiles;C:Program Files (x86)Microsoft SQL Server100ToolsBinn;C:Program FilesMicrosoft SQL Server100ToolsBinn;C:Program FilesMicrosoft SQL Server100DTSBinn;C:Program Files (x86)Microsoft SQL Server100ToolsBinnVSShellCommon7IDE;C:Program Files (x86)Microsoft Visual Studio 9.0Common7IDEPrivateAssemblies;C:Program Files (x86)Microsoft SQL Server100DTSBinn;C:WINDOWSsystem32WindowsPowerShellv1.0;C:gnuwin32bin

Further on, I launched the tail command to intercatively take an eye on who is accessing the IIS webserver.
Sadly this worked not, trying to use tail with the IIS ex10116.log log;

C:WindowsSystem32LogfilesW3SVC1> tail -f ex10116.log

Spit an error tail: ex10116.log: Bad file descriptor

Since I couldn’t use tail -f I looked for alternative and a quick search led me to Tail 4 Win32 . Lest the name suggests it is supposed to work on 32 bit arch Windows the version on tailforwin32’s website is working perfectly fine on 64 bit Windows as well.
What it does is to simulate a normal tail -f command inside a very simplistic window interface. You see it in action with opened IIS log on below’s screenshot:

GUI Tail for Windows screenshot

Finally my goal is achieved and I can take an eye interactively on IIS logs. End of the article, hope it wasn’t too boring 😉