Posts Tagged ‘prints’

Show directory structure bash script on Linux howto – See hierarchical directory tree structure one liner shell script

Friday, February 24th, 2017

show-directory-structure-see-hierarchical-directory-tree-structure-on-linux-with-tree-command-and-with-bash-shell-scripts

If you have Sys Adminned Linux or *Nix OS like, whether for some shell scripting purpose or just for sake of keeping a backup you should have definitely come
into some need to list a tree of a directories content in a hierarchical order.

The most obvious way to do that on Linux is by simply using:

1.  "tree" command (not installed by default on most Linux distributions so in order to have it on Deb / Debian based Linux do:
 

# apt-get install –yes tree


On Fedora / CentOS Redhat Linux (RHEL) etc. install with:

# yum –yes install tree

 

By the way for those that needs tree on FreeBSD / BSD UNIX, tree is also available on that platform you can install it with:
 

pkg_add -vr tree


Then simply check man tree to get idea on how to use it, the easiest way to use the command tree once package is installed is to run tree inside directory of choice, i.e.

 

$ cd /somedir
$ tree -a

.
├── acpi
│   ├── events
│   │   └── powerbtn-acpi-support
│   └── powerbtn-acpi-support.sh
├── adduser.conf
├── adjtime
├── aliases
├── alternatives
│   ├── ABORT.7.gz -> /usr/share/postgresql/9.5/man/man7/ABORT.7.gz
│   ├── aclocal -> /usr/bin/aclocal-1.11
│   ├── aclocal.1.gz -> /usr/share/man/man1/aclocal-1.11.1.gz
│   ├── ALTER_AGGREGATE.7.gz -> /usr/share/postgresql/9.5/man/man7/ALTER_AGGREGATE.7.gz
│   ├── ALTER_COLLATION.7.gz -> /usr/share/postgresql/9.5/man/man7/ALTER_COLLATION.7.gz
│   ├── ALTER_CONVERSION.7.gz -> /usr/share/postgresql/9.5/man/man7/ALTER_CONVERSION.7.gz
 

 

To get a list of only directories with tree use:
 

$ tree -d /

 

  │   ├── bin
│   │   ├── boot
│   │   │   └── grub
│   │   │       └── locale
│   │   ├── disk
│   │   │   ├── Books
│   │   │   │   ├── 200 E-BOOKS
│   │   │   │   │   ├── McGraw-Hill – Windows Server 2003
│   │   │   │   │   ├── Oreilly.Access.Cookbook.2nd.Edition-LiB
│   │   │   │   │   ├── Oreilly.ActionScript.Cookbook.eBook-LiB
│   │   │   │   │   ├── OReilly.ActionScript.The.Definative.Guide.WinAll.Retail-EAT
│   │   │   │   │   ├── Oreilly.Active.Directory.2nd.Edition.eBook-LiB
│   │   │   │   │   ├── Oreilly.Active.Directory.Cookbook.eBook-LiB
│   │   │   │   │   ├── Oreilly.ADO.Dot.NET.Cookbook.eBook-LiB
│   │   │   │   │   ├── Oreilly.Amazon.Hacks.eBook-LiB
│   │   │   │   │   ├── OREILLY.ANT.THE.DEFINITIVE.GUIDE-JGT
│   │   │   │   │   ├── Oreilly.Apache.Cookbook.eBook-LiB
│   │   │   │   │   ├── Oreilly.AppleScript.The.Definitive.Guide.eBook-LiB
│   │   │   │   │   ├── Oreilly.ASP.Dot.NET.In.A.Nutshell.2nd.Edition.eBook-LiB
│   │   │   │   │   ├── OReilly.Better.Faster.Lighter.Java.Jun.2004.eBook-DDU
│   │   │   │   │   ├── Oreilly.BLAST.eBook-LiB
│   │   │   │   │   ├── OReilly.BSD.Hacks.May.2004.eBook-DDU
│   │   │   │   │   ├── Oreilly.Building.Embedded.Linux.Systems.eBook-LiB

If you have a colorful terminal and you like colors for readability the -C option is quite handy

 

$ tree -C /

 

tree-command-linux-hierarchical-structure-directory-tree
 

To list the directory tree with permissions included use tree cmd like so:

 

$ tree -L 2 -p /usr

/usr/
├── [drwxr-xr-x]  bin
│   ├── [-rwxr-xr-x]  [
│   ├── [lrwxrwxrwx]  2to3 -> 2to3-2.6
│   ├── [-rwxr-xr-x]  2to3-2.6
│   ├── [-rwxr-xr-x]  411toppm
│   ├── [-rwxr-xr-x]  7z
│   ├── [-rwxr-xr-x]  7za
│   ├── [-rwxr-xr-x]  a2p
│   ├── [-rwxr-xr-x]  ab
│   ├── [-rwxr-xr-x]  ac
│   ├── [lrwxrwxrwx]  aclocal -> /etc/alternatives/aclocal
│   ├── [-rwxr-xr-x]  aclocal-1.11
│   ├── [-rwxr-xr-x]  acpi


Another truly handy option of tree is to list the directory structure index with included file sizes information

 

$ tree -L 2 -sh /bin

/bin
├── [903K]  bash
├── [147K]  bsd-csh
├── [ 30K]  bunzip2
├── [681K]  busybox
├── [ 30K]  bzcat
├── [   6]  bzcmp -> bzdiff
├── [2.1K]  bzdiff
├── [   6]  bzegrep -> bzgrep
├── [4.8K]  bzexe
├── [   6]  bzfgrep -> bzgrep
├── [3.6K]  bzgrep
├── [ 30K]  bzip2
├── [ 14K]  bzip2recover
├── [   6]  bzless -> bzmore
├── [1.3K]  bzmore
├── [ 51K]  cat
├── [ 59K]  chgrp
├── [ 55K]  chmod
├── [ 63K]  chown
├── [ 10K]  chvt
├── [127K]  cp
├── [134K]  cpio
├── [  21]  csh -> /etc/alternatives/csh
├── [104K]  dash


To list a directory tree of a search pattern, lets say all files with .conf extensions use:
 

$ tree -P *.conf

/etc/ca-certificates.conf [error opening dir]
/etc/dante.conf [error opening dir]
/etc/debconf.conf [error opening dir]
/etc/deluser.conf [error opening dir]
/etc/discover-modprobe.conf [error opening dir]
/etc/fuse.conf [error opening dir]
/etc/gai.conf [error opening dir]
/etc/gpm.conf [error opening dir]
/etc/gssapi_mech.conf [error opening dir]
/etc/hdparm.conf [error opening dir]
/etc/host.conf [error opening dir]
/etc/idmapd.conf [error opening dir]
/etc/inetd.conf [error opening dir]
/etc/insserv.conf [error opening dir]
/etc/irssi.conf [error opening dir]
/etc/kernel-img.conf [error opening dir]
/etc/ld.so.conf [error opening dir]
/etc/libao.conf [error opening dir]
/etc/libaudit.conf [error opening dir]
/etc/logrotate.conf [error opening dir]
/etc/memcached.conf [error opening dir]
/etc/mke2fs.conf [error opening dir]
/etc/mongodb.conf [error opening dir]
/etc/mtools.conf [error opening dir]
/etc/multitail.conf [error opening dir]
/etc/nsswitch.conf [error opening dir]
/etc/ntp.conf [error opening dir]
/etc/ocamlfind.conf [error opening dir]

 

 

tree -I option does exclude all petterns you don't want tree to list

Here are few other tree useful options:

  • tree -u /path/to/file – displays the users owning the files
  • tree -g /path/to/file – display the groups owning the files
  • tree -a /path/to/file – display the hidden files/folders
  • tree -d /path/to/file – display only the directories in the hierarchy


However there might be some cases where you have to support a Linux server or you just have to write a script for a non-root user and you might not have the permissins to install the tree command to make your life confortable. If that's the case then you can still use a couple of command line tools and tricks (assuming you have permissions) to list a log a directory / files and subdirectories tree structure in a hierarchical tree like command order

2. Print a list of all sub-directories and files within a directory tree

To print all directories within any path of choise on a server use
 

$ find /path/ -type d -print

 

To print all files within a root filesystem hierarchically with find command

Another way to do it in a more beautiful output is by using find in conjunction with awk
 

$ find . -type d -print 2>/dev/null|awk '!/\.$/ {for (i=1;i<NF;i++){d=length($i);if ( d < 5  && i != 1 )d=5;printf("%"d"s","|")}print "—"$NF}'  FS='/'

 

|—bashscripts
|          |—not-mine
|          |—various
|          |—output
|          |—examples
|          |—fun
|          |—educational
|          |—backdoor-cgi
|          |—fork-bombs
|          |—tmp
|          |    |—old
|          |—bullshits
|—packages
|       |—ucspi-ssl-0.70.2
|       |               |—package
|       |               |—compile
|       |               |      |—rts-tmp
|       |               |—command
|       |               |—src
|—bin
|—package
|      |—host
|      |    |—superscript.com
|      |    |              |—command
|—mnt
|    |—tmpfs
|    |—disk
|    |—flash_drive
|    |—ramfs
|    |—cdrom

3. Get a list of the directories on filesystem structure with one-liner ls + sed script
 

$ ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//–/g' -e 's/^/ /' -e 's/-/|/'

 

 |-bin
 |-boot
 |-dev
 |—net
 |—pts
 |-downloads
 |—autorespond-2.0.5
 |—–debian
 |—deb-packages
 |—–IP-Country-2.28
 |——-bin
 |——-blib
 |———arch
 |———–auto

….
 

4. Print all files within root filesystem and issue any command on each of the files
 

ls -R1 / |    while read l; do case $l in *:) d=${l%:};; "") d=;; *) echo "$d/$l";; esac; done


Above command just prints all the found files with full-path if you want to check the file size or file type you just check echo command with any command you need to execute on each of the listed file

5. Get a list hierarchical directory Linux tree with bash shell scripts: Assuming that the server where you need to have a list of the directory filesystem structure in a tree fashion has bash you could use this little script called tree.sh to do the job or for a full filesystem hierarchical tree like directory structure use fulltree.sh

How to count how many files are in a directory with find on Linux

Tuesday, February 21st, 2012

how to count how many directories are on your linux server

Did you ever needed to count, how many files in a directory are there?
Having the concrete number of files in a directory is not a seldom task but still very useful especially for scripts or simply for the sake of learning

The quickest and maybe the easiest way to count all files in a directory in Linux is with a combination of find and wc commands:

Here is how;

linux:~# cd ascii
linux:~/ascii# find . -type f -iname '*' -print |wc -l
407

This will find and list all matched files in any directory and subdirectories, print them out and count them with wc command.
The -type f argument instructs find to look only for files.

Other helpful variance of finding and listing all files in a directory and subdirectories is to list and count all the files with a certain file extension under a directory. For example, lets list all text files (.txt) contained in a directory and all level sub-directories:

linux:~/ascii# find . -type f -iname '*.txt' -print |wc -l
401

If you need to check the number of files in a directory for multiple directories on a server and you're aiming at doing it efficienly, issung above find .. | wc code will definitely be not a good choice. If used it will generate heavy load for the system and along with that will complete the execution in ages if issued on a large number of files containing dirs.

Thanksfully if efficiency is targetted, there is a command written in C called tree which is more efficient than find.
To count the number of files in dir but using tree :

linux:~# cd ascii
linux:/ascii# tree | tail -n 1
32 directories, 407 files

By default tree prints info for both the number of found files and directories.
To print out only the files matched, awk comes handy, e.g.:

linux:/ascii# tree |tail -n 1| awk '{ print $3 }'407

To list only the number of files in a directory without its existing sub-directories ls + wc use is also possible:

linux:~/ascii# ls -l | grep ^- | wc -l68

This result the above command would produce is +1 more than the real number of files, as it counts the directory ".." as one file (in UNIX / LINUX everything is file).

A short one liner script that can calculate all files correctly by substracting 1 is and hence present correct result on number of files is like so:

linux:~/ascii# var=$(ls -l | grep ^- | wc -l); var=$(($var - 1)); echo $var

ls can be used to calculate the number of 1-st level sub-directories under certain directory for instance:

linux:~/ascii# ls -l |grep ^d|wc -l
25

You see the ascii directory has 25 subdirectories in its 1st level.

To check symlinks under a directory with ls the command would be:

linux:~/ascii# ls -l | grep ^l | wc -l
0

Note above 3 ls | grep … examples, will not work properly if the directory contains files with SUID or some special properties set.
Hence to get the same 3 results for active files, directories and symbolic links, a one liner similar to the one below can be used instead:

linux:~/ascii# for t in files links directories; do echo `find . -type ${t:0:1} | wc -l` $t; done 2> /dev/null
407 files
0 links
33 directories

This will show statistics about all files, links and directories for all directory sub-levels.
Just in case if there is need to only count files, links and directories without directory recursion enabled, use:

linux:~/ascii# for t in files links directories; do echo `find . -maxdepth 1 -type ${t:0:1} | wc -l` $t; done 2> /dev/null
68 files
0 links
26 directories

Anyways the above bash loop will be slow, for directories containing thousands of files. For better performance the equivallent of above bash loop rewritten in perl would be:

linux:~/ascii# ls -l |perl -e 'while(<>){$h{substr($_,0,1)}+=1;} END {foreach(keys %h) {print "$_ $h{$_}\n";}}'
- 68
d 25
t 1
linux:~/ascii#
In any case the most preferrable and efficient way to count files en directories is by using tree command.
In my view using always tree command instead of code "hacks" is smart idea.

In Slackware tree command is part of the base install, on Debian and CentOS Linux, tree cmd is not part of the base system and requires install via apt / yum e.g.:

debian:~# apt-get --yes install tree
...

[root@centos:~ ]# yum --yes install tree

Happy counting 😉

Tiny PHP script to dump your browser set HTTP headers (useful in debugging)

Friday, March 30th, 2012

While browsing I stumbled upon a nice blog article

Dumping HTTP headers

The arcitle, points at few ways to DUMP the HTTP headers obtained from user browser.
As I'm not proficient with Ruby, Java and AOL Server what catched my attention is a tiny php for loop, which loops through all the HTTP_* browser set variables and prints them out. Here is the PHP script code:

<?php<br />
foreach($_SERVER as $h=>$v)<br />
if(ereg('HTTP_(.+)',$h,$hp))<br />
echo "<li>$h = $v</li>\n";<br />
header('Content-type: text/html');<br />
?>

The script is pretty easy to use, just place it in a directory on a WebServer capable of executing php and save it under a name like:
show_HTTP_headers.php

If you don't want to bother copy pasting above code, you can also download the dump_HTTP_headers.php script here , rename the dump_HTTP_headers.php.txt to dump_HTTP_headers.php and you're ready to go.

Follow to the respective url to exec the script. I've installed the script on my webserver, so if you are curious of the output the script will be returning check your own browser HTTP set values by clicking here.
PHP will produce output like the one in the screenshot you see below, the shot is taken from my Opera browser:

Screenshot show HTTP headers.php script Opera Debian Linux

Another sample of the text output the script produce whilst invoked in my Epiphany GNOME browser is:

HTTP_HOST = www.pc-freak.net
HTTP_USER_AGENT = Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Version/5.0 Safari/531.2+ Debian/squeeze (2.30.6-1) Epiphany/2.30.6
HTTP_ACCEPT = application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
HTTP_ACCEPT_ENCODING = gzip
HTTP_ACCEPT_LANGUAGE = en-us, en;q=0.90
HTTP_COOKIE = __qca=P0-2141911651-1294433424320;
__utma_a2a=8614995036.1305562814.1274005888.1319809825.1320152237.2021;wooMeta=MzMxJjMyOCY1NTcmODU1MDMmMTMwODQyNDA1MDUyNCYxMzI4MjcwNjk0ODc0JiYxMDAmJjImJiYm; 3ec0a0ded7adebfeauth=22770a75911b9fb92360ec8b9cf586c9;
__unam=56cea60-12ed86f16c4-3ee02a99-3019;
__utma=238407297.1677217909.1260789806.1333014220.1333023753.1606;
__utmb=238407297.1.10.1333023754; __utmc=238407297;
__utmz=238407297.1332444980.1586.413.utmcsr=www.pc-freak.net|utmccn=(referral)|utmcmd=referral|utmcct=/blog/

You see the script returns, plenty of useful information for debugging purposes:
HTTP_HOST – Virtual Host Webserver name
HTTP_USER_AGENT – The browser exact type useragent returnedHTTP_ACCEPT – the type of MIME applications accepted by the WebServerHTTP_ACCEPT_LANGUAGE – The language types the browser has support for
HTTP_ACCEPT_ENCODING – This PHP variable is usually set to gzip or deflate by the browser if the browser has support for webserver returned content gzipping.
If HTTP_ACCEPT_ENCODING is there, then this means remote webserver is configured to return its HTML and static files in gzipped form.
HTTP_COOKIE – Information about browser cookies, this info can be used for XSS attacks etc. 🙂
HTTP_COOKIE also contains the referrar which in the above case is:
__utmz=238407297.1332444980.1586.413.utmcsr=www.pc-freak.net|utmccn=(referral)
The Cookie information HTTP var also contains information of the exact link referrar:
|utmcmd=referral|utmcct=/blog/

For the sake of comparison show_HTTP_headers.php script output from elinks text browser is like so:

* HTTP_HOST = www.pc-freak.net
* HTTP_USER_AGENT = Links (2.3pre1; Linux 2.6.32-5-amd64 x86_64; 143x42)
* HTTP_ACCEPT = */*
* HTTP_ACCEPT_ENCODING = gzip,deflate * HTTP_ACCEPT_CHARSET = us-ascii, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10, ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16, windows-1250, windows-1251, windows-1252, windows-1256,
windows-1257, cp437, cp737, cp850, cp852, cp866, x-cp866-u, x-mac, x-mac-ce, x-kam-cs, koi8-r, koi8-u, koi8-ru, TCVN-5712, VISCII,utf-8 * HTTP_ACCEPT_LANGUAGE = en,*;q=0.1
* HTTP_CONNECTION = keep-alive
One good reason, why it is good to give this script a run is cause it can help you reveal problems with HTTP headers impoperly set cookies, language encoding problems, security holes etc. Also the script is a good example, for starters in learning PHP programming.