Posts Tagged ‘fuse’

Show directory structure bash script on Linux howto – See hierarchical directory tree structure one liner shell script

Friday, February 24th, 2017

Reading Time: 5 minutes

show-directory-structure-see-hierarchical-directory-tree-structure-on-linux-with-tree-command-and-with-bash-shell-scripts

If you have Sys Adminned Linux or *Nix OS like, whether for some shell scripting purpose or just for sake of keeping a backup you should have definitely come
into some need to list a tree of a directories content in a hierarchical order.

The most obvious way to do that on Linux is by simply using:

1.  "tree" command (not installed by default on most Linux distributions so in order to have it on Deb / Debian based Linux do:
 

# apt-get install –yes tree


On Fedora / CentOS Redhat Linux (RHEL) etc. install with:

# yum –yes install tree

 

By the way for those that needs tree on FreeBSD / BSD UNIX, tree is also available on that platform you can install it with:
 

pkg_add -vr tree


Then simply check man tree to get idea on how to use it, the easiest way to use the command tree once package is installed is to run tree inside directory of choice, i.e.

 

$ cd /somedir
$ tree -a

.
├── acpi
│   ├── events
│   │   └── powerbtn-acpi-support
│   └── powerbtn-acpi-support.sh
├── adduser.conf
├── adjtime
├── aliases
├── alternatives
│   ├── ABORT.7.gz -> /usr/share/postgresql/9.5/man/man7/ABORT.7.gz
│   ├── aclocal -> /usr/bin/aclocal-1.11
│   ├── aclocal.1.gz -> /usr/share/man/man1/aclocal-1.11.1.gz
│   ├── ALTER_AGGREGATE.7.gz -> /usr/share/postgresql/9.5/man/man7/ALTER_AGGREGATE.7.gz
│   ├── ALTER_COLLATION.7.gz -> /usr/share/postgresql/9.5/man/man7/ALTER_COLLATION.7.gz
│   ├── ALTER_CONVERSION.7.gz -> /usr/share/postgresql/9.5/man/man7/ALTER_CONVERSION.7.gz
 

 

To get a list of only directories with tree use:
 

$ tree -d /

 

  │   ├── bin
│   │   ├── boot
│   │   │   └── grub
│   │   │       └── locale
│   │   ├── disk
│   │   │   ├── Books
│   │   │   │   ├── 200 E-BOOKS
│   │   │   │   │   ├── McGraw-Hill – Windows Server 2003
│   │   │   │   │   ├── Oreilly.Access.Cookbook.2nd.Edition-LiB
│   │   │   │   │   ├── Oreilly.ActionScript.Cookbook.eBook-LiB
│   │   │   │   │   ├── OReilly.ActionScript.The.Definative.Guide.WinAll.Retail-EAT
│   │   │   │   │   ├── Oreilly.Active.Directory.2nd.Edition.eBook-LiB
│   │   │   │   │   ├── Oreilly.Active.Directory.Cookbook.eBook-LiB
│   │   │   │   │   ├── Oreilly.ADO.Dot.NET.Cookbook.eBook-LiB
│   │   │   │   │   ├── Oreilly.Amazon.Hacks.eBook-LiB
│   │   │   │   │   ├── OREILLY.ANT.THE.DEFINITIVE.GUIDE-JGT
│   │   │   │   │   ├── Oreilly.Apache.Cookbook.eBook-LiB
│   │   │   │   │   ├── Oreilly.AppleScript.The.Definitive.Guide.eBook-LiB
│   │   │   │   │   ├── Oreilly.ASP.Dot.NET.In.A.Nutshell.2nd.Edition.eBook-LiB
│   │   │   │   │   ├── OReilly.Better.Faster.Lighter.Java.Jun.2004.eBook-DDU
│   │   │   │   │   ├── Oreilly.BLAST.eBook-LiB
│   │   │   │   │   ├── OReilly.BSD.Hacks.May.2004.eBook-DDU
│   │   │   │   │   ├── Oreilly.Building.Embedded.Linux.Systems.eBook-LiB

If you have a colorful terminal and you like colors for readability the -C option is quite handy

 

$ tree -C /

 

tree-command-linux-hierarchical-structure-directory-tree
 

To list the directory tree with permissions included use tree cmd like so:

 

$ tree -L 2 -p /usr

/usr/
├── [drwxr-xr-x]  bin
│   ├── [-rwxr-xr-x]  [
│   ├── [lrwxrwxrwx]  2to3 -> 2to3-2.6
│   ├── [-rwxr-xr-x]  2to3-2.6
│   ├── [-rwxr-xr-x]  411toppm
│   ├── [-rwxr-xr-x]  7z
│   ├── [-rwxr-xr-x]  7za
│   ├── [-rwxr-xr-x]  a2p
│   ├── [-rwxr-xr-x]  ab
│   ├── [-rwxr-xr-x]  ac
│   ├── [lrwxrwxrwx]  aclocal -> /etc/alternatives/aclocal
│   ├── [-rwxr-xr-x]  aclocal-1.11
│   ├── [-rwxr-xr-x]  acpi


Another truly handy option of tree is to list the directory structure index with included file sizes information

 

$ tree -L 2 -sh /bin

/bin
├── [903K]  bash
├── [147K]  bsd-csh
├── [ 30K]  bunzip2
├── [681K]  busybox
├── [ 30K]  bzcat
├── [   6]  bzcmp -> bzdiff
├── [2.1K]  bzdiff
├── [   6]  bzegrep -> bzgrep
├── [4.8K]  bzexe
├── [   6]  bzfgrep -> bzgrep
├── [3.6K]  bzgrep
├── [ 30K]  bzip2
├── [ 14K]  bzip2recover
├── [   6]  bzless -> bzmore
├── [1.3K]  bzmore
├── [ 51K]  cat
├── [ 59K]  chgrp
├── [ 55K]  chmod
├── [ 63K]  chown
├── [ 10K]  chvt
├── [127K]  cp
├── [134K]  cpio
├── [  21]  csh -> /etc/alternatives/csh
├── [104K]  dash


To list a directory tree of a search pattern, lets say all files with .conf extensions use:
 

$ tree -P *.conf

/etc/ca-certificates.conf [error opening dir]
/etc/dante.conf [error opening dir]
/etc/debconf.conf [error opening dir]
/etc/deluser.conf [error opening dir]
/etc/discover-modprobe.conf [error opening dir]
/etc/fuse.conf [error opening dir]
/etc/gai.conf [error opening dir]
/etc/gpm.conf [error opening dir]
/etc/gssapi_mech.conf [error opening dir]
/etc/hdparm.conf [error opening dir]
/etc/host.conf [error opening dir]
/etc/idmapd.conf [error opening dir]
/etc/inetd.conf [error opening dir]
/etc/insserv.conf [error opening dir]
/etc/irssi.conf [error opening dir]
/etc/kernel-img.conf [error opening dir]
/etc/ld.so.conf [error opening dir]
/etc/libao.conf [error opening dir]
/etc/libaudit.conf [error opening dir]
/etc/logrotate.conf [error opening dir]
/etc/memcached.conf [error opening dir]
/etc/mke2fs.conf [error opening dir]
/etc/mongodb.conf [error opening dir]
/etc/mtools.conf [error opening dir]
/etc/multitail.conf [error opening dir]
/etc/nsswitch.conf [error opening dir]
/etc/ntp.conf [error opening dir]
/etc/ocamlfind.conf [error opening dir]

 

 

tree -I option does exclude all petterns you don't want tree to list

Here are few other tree useful options:

  • tree -u /path/to/file – displays the users owning the files
  • tree -g /path/to/file – display the groups owning the files
  • tree -a /path/to/file – display the hidden files/folders
  • tree -d /path/to/file – display only the directories in the hierarchy


However there might be some cases where you have to support a Linux server or you just have to write a script for a non-root user and you might not have the permissins to install the tree command to make your life confortable. If that's the case then you can still use a couple of command line tools and tricks (assuming you have permissions) to list a log a directory / files and subdirectories tree structure in a hierarchical tree like command order

2. Print a list of all sub-directories and files within a directory tree

To print all directories within any path of choise on a server use
 

$ find /path/ -type d -print

 

To print all files within a root filesystem hierarchically with find command

Another way to do it in a more beautiful output is by using find in conjunction with awk
 

$ find . -type d -print 2>/dev/null|awk '!/\.$/ {for (i=1;i<NF;i++){d=length($i);if ( d < 5  && i != 1 )d=5;printf("%"d"s","|")}print "—"$NF}'  FS='/'

 

|—bashscripts
|          |—not-mine
|          |—various
|          |—output
|          |—examples
|          |—fun
|          |—educational
|          |—backdoor-cgi
|          |—fork-bombs
|          |—tmp
|          |    |—old
|          |—bullshits
|—packages
|       |—ucspi-ssl-0.70.2
|       |               |—package
|       |               |—compile
|       |               |      |—rts-tmp
|       |               |—command
|       |               |—src
|—bin
|—package
|      |—host
|      |    |—superscript.com
|      |    |              |—command
|—mnt
|    |—tmpfs
|    |—disk
|    |—flash_drive
|    |—ramfs
|    |—cdrom

3. Get a list of the directories on filesystem structure with one-liner ls + sed script
 

$ ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//–/g' -e 's/^/ /' -e 's/-/|/'

 

 |-bin
 |-boot
 |-dev
 |—net
 |—pts
 |-downloads
 |—autorespond-2.0.5
 |—–debian
 |—deb-packages
 |—–IP-Country-2.28
 |——-bin
 |——-blib
 |———arch
 |———–auto

….
 

4. Print all files within root filesystem and issue any command on each of the files
 

ls -R1 / |    while read l; do case $l in *:) d=${l%:};; "") d=;; *) echo "$d/$l";; esac; done


Above command just prints all the found files with full-path if you want to check the file size or file type you just check echo command with any command you need to execute on each of the listed file

5. Get a list hierarchical directory Linux tree with bash shell scripts: Assuming that the server where you need to have a list of the directory filesystem structure in a tree fashion has bash you could use this little script called tree.sh to do the job or for a full filesystem hierarchical tree like directory structure use fulltree.sh

How to mount NTFS Windows XP filesystem on FreeBSD, NetBSD, OpenBSD

Friday, May 11th, 2012

Reading Time: 4 minutes

Mounting NTFS hdd partitions on FreeBSD logo picture

A friend of mine bring home a Seagate External Hard Disk Drive using USB 3 as a communication media. I needed to attach the hard disk to my FreeBSD router to transfer him some data, the External HDD is formatted to use NTFS as a main file partition and hence to make the file transfers I had to somehow mount the NTFS partition on the HDD.

FreeBSD and other BSDs, just like Linux does not have embedded NTFS file system mount support.
In order to add an external write support for the plugged hdd NTFS I looked in the ports tree:

freebsd# cd /usr/ports
freebsd# make search name='ntfs'
Port: fusefs-ntfs-2010.10.2
Path: /usr/ports/sysutils/fusefs-ntfs
Info: Mount NTFS partitions (read/write) and disk images
Maint: ports@FreeBSD.org
B-deps: fusefs-libs-2.7.4 libiconv-1.13.1_1 libtool-2.4 libublio-20070103 pkg-config-0.25_1
R-deps: fusefs-kmod-0.3.9.p1.20080208_7 fusefs-libs-2.7.4 libiconv-1.13.1_1 libublio-20070103 pkg-config-0.25_1
WWW: http://www.tuxera.com/community/

Port: ntfsprogs-2.0.0_1
Path: /usr/ports/sysutils/ntfsprogs
Info: Utilities and library to manipulate NTFS partitions
Maint: ports@FreeBSD.org
B-deps: fusefs-libs-2.7.4 libiconv-1.13.1_1 libublio-20070103 pkg-config-0.25_1
R-deps: libublio-20070103 pkg-config-0.25_1
WWW: http://www.linux-ntfs.org/
freebs# cd sysutils/fusefs-ntfs/
freebsd# ls
Makefile distinfo files/ pkg-descr pkg-plist
freebsd# cat pkg-descr
The ntfs-3g driver is an open source, freely available read/write NTFS
driver, which provides safe and fast handling of the Windows XP, Windows
Server 2003 and Windows 2000 filesystems. Almost the full POSIX filesystem
functionality is supported, the major exceptions are changing the file
ownerships and the access rights.
WWW: http://www.tuxera.com/community/

Using ntfs-3g I managed to succeed mounting the NTFS on my old PC running FreeBSD ver. 7_2

1. Installing fuserfs-ntfs support on BSD

Before I can use ntfs-3g, to mount the paritition, I had to install fuserfs-ntfs bsd port, with:

freebsd# cd /usr/ports/sysutils/fusefs-ntfs
freebsd# make install clean
.....

I was curious if ntfsprogs provides other utilities to do the ntfs mount but whilst trying to install it I realized it is already installed as a dependency package to fusefs-ntfs.

fusefs-ntfs package provides a number of utilities for creating, mounting, fixing and doing various manipulations with Microsoft NTFS filesystems.

Here is a list of all the executable utilities helpful in NTFS fs management:

freebsd# pkg_info -L fusefs-ntfs\* | grep -E "/bin/|/sbin|README"
/usr/local/bin/lowntfs-3g
/usr/local/bin/ntfs-3g
/usr/local/bin/ntfs-3g.probe
/usr/local/bin/ntfs-3g.secaudit
/usr/local/bin/ntfs-3g.usermap
/usr/local/bin/ntfscat
/usr/local/bin/ntfscluster
/usr/local/bin/ntfscmp
/usr/local/bin/ntfsfix
/usr/local/bin/ntfsinfo
/usr/local/bin/ntfsls
/usr/local/sbin/mkntfs
/usr/local/sbin/ntfsclone
/usr/local/sbin/ntfscp
/usr/local/sbin/ntfslabel
/usr/local/sbin/ntfsresize
/usr/local/sbin/ntfsundelete
/usr/local/share/doc/ntfs-3g/README
/usr/local/share/doc/ntfs-3g/README.FreeBSD

The README and README.FreeBSD are wonderful, reading for those who want to get more in depth knowledge on using the up-listed utilities.

One utility, worthy to mention, I have used in the past is ntfsfix. ntfsfix resolve issues with NTFS partitions which were not unmounted on system shutdown (electricity outage), system hang up etc.

2. Start fusefs (ntfs) and configure it to auto load on system boot

Once fuserfs-ntfs is installed, if its necessery ntfs fs mounts to be permanently supported on the BSD system add fusefs_enable="YES" to /etc/rc.conf(the FreeBSD services auto load conf).

freebsd# echo 'fusefs_enable="YES"' >> /etc/rc.conf

One note to make here is that you need to have also dbus_enable="YES" and hald_enable="YES" in /etc/rc.conf, not having this two in rc.conf will prevent fusefs to start properly. Do a quick grep to make sure this two variables are enabled:

Afterwards fsusefs load up script should be run:

freebsd# /usr/local/etc/rc.d/fusefs start
Starting fusefs.

Another alternative way to load ntfs support on the BSD host is to directly load fuse.ko kernel module:

freebsd# /sbin/kldload fuse.ko

3. Mounting the NTFS partition

In my case, the Seagate hard drive was detected as da0, where the NTFS partition was detected as s1 (da0s1):

freebsd# dmesg|grep -i da0
da0 at umass-sim0 bus 0 target 0 lun 0
da0: Fixed Direct Access SCSI-4 device
da0: 40.000MB/s transfers
da0: 953869MB (1953525164 512 byte sectors: 255H 63S/T 121601C)br />GEOM_LABEL: Label for provider da0s1 is ntfs/Expansion Drive.
GEOM_LABEL: Label for provider da0s1 is ntfs/Expansion Drive.

Therefore further to mount it one can use mount_ntfs (to quickly mount in read only mode) or ntfs-3g for (read / write mode):

If you need to just quickly mount a disk drive to copy some data and umount it with no need for writting to the NTFS partition do;

freebsd# /sbin/mount_ntfs /dev/ad0s1 /mnt/disk

Note that mount_ntfs command is a native BSD command and have nothing to do with ntfs-3g. Therefore using it to mount NTFS is not the same as mounting it via ntfs-3g cmd

freebsd# /usr/local/bin/ntfs-3g -o rw /dev/da0s1 /mnt/disk/

Something, I've noticed while using ntfs-3g is, it fails to properly exit even when the ntfs-3g shell execution is over:

freebsd# ps ax |grep -i ntfs|grep -v grep
18892 ?? Is 0:00.00 /usr/local/bin/ntfs-3g -o rw /dev/da0s1 /mnt/disk/

I dunno if this is some kind of ntfs-3g bug or feature specific to all versions of FreeBSD or it is something local to FBSD 7.2

Thought ntfs-3g, keeps appearing in process list, praise God as of time of writting NTFS support on FreeBSD prooved to be stable.
Read / Write disk operations to the NTFS I tested it with works great. Just about 5 years ago I still remember write mode was still experimental. Now it seems NTFS mounts can be used with no hassle even on production machines.

4. Auto mounting NTFS partition on FreeBSD system boot

There are two approaches towards 'the problem' I can think of.
The better way to auto mount on boot (in my view) is through /etc/fstab use

If /etc/fstab + ntfs-3g is to be used, you will however change the default /sbin/mount_ntfs command to point to /usr/local/bin/ntfs-3g, i.e.:

freebsd# mv /sbin/mount_ntfs /sbin/mount_ntfs.orig
freebsd# ln -s /usr/local/bin/ntfs-3g /sbin/mount_ntfs

Then to mount /dev/da0s1 via /etc/fstab add line:

/dev/ad0s1 /mnt/disk ntfs rw,late 0 0

To not bother with text editor run:

freebsd# echo '/dev/ad0s1 /mnt/disk ntfs rw,late 0 0' >> /etc/fstab

I've red in posts in freebsd forums, there is also a way to use ntfs-3g for mounting partitions, without substituting the original bsd /sbin/mount_ntfs, the exact commands suggested to be used with no need to prior mv /sbin/mount_ntfs to /sbin/mount_ntfs.orig and link it to ntfs is:

/dev/ad0s1 /disk ntfs rw,mountprog=/usr/local/bin/ntfs-3g,late 0 0

For any other NTFS partitions, for instance /dev/ad0s2, /dev/ad2s1 etc. simply change the parititon name and mount points.

The second alternative to adding the NTFS to auto mount is through /etc/rc.local. /etc/rc.local content will be executed very late in system boot. :

echo '/usr/local/bin/ntfs-3g -o rw /dev/da0s1' >> /etc/rc.local

One disadvanage of using /etc/rc.local for mounting the partition is the hanging ntfs-3g in proc list:

freebsd# ps ax |grep -i ntfs|grep -v grep
18892 ?? Is 0:00.00 /usr/local/bin/ntfs-3g -o rw /dev/da0s1 /mnt/disk/

Though, I haven't tested it yet. Using the same methodology should be perfectly working on PC-BSD, DragonFlyBSD, NetBSD and OpenBSD.
I will be glad if someone who runs any of the other BSDs can confirm, following this instructions works fine on these BSDs too.