Linux Shell Script: Tips and Tricks

Exclusive offer: get 50% off this eBook here
Linux Shell Scripting Cookbook

Linux Shell Scripting Cookbook — Save 50%

Solve real-world shell scripting problems with over 110 simple but incredibly effective recipes

$26.99    $13.50
by Sarath Lakshman | April 2011 | Linux Servers Open Source

GNU/Linux is a remarkable operating system that comes with a complete development environment that is stable, reliable, and extremely powerful. The shell, being the native interface to communicate with the operating system, is capable of controlling the entire operating system. An understanding of shell scripting helps you to have better awareness of the operating system and helps you to automate most of the manual tasks with a few lines of script, saving you an enormous amount of time. Shell scripts can work with many external command-line utilities for tasks such as querying information, easy text manipulation, scheduling task running times, preparing reports, sending mails, and so on. There are numerous commands on the GNU/Linux shell, which are documented but hard to understand.

In this article we will take a look at some of the tips and tricks on working with Linux Shell Script.

 

Linux Shell Scripting Cookbook

Linux Shell Scripting Cookbook

Solve real-world shell scripting problems with over 110 simple but incredibly effective recipes

        Read more about this book      

(For more resources on Linux, see here.)

Successful and unsuccessful command
Tip: When a command returns after error, it returns a non-zero exit status. The command returns zero when it terminates after successful completion. Return status can be read from special variable $? (run echo $? immediately after the command execution statement to print the exit status).

 

Fork bomb
:(){ :|:& };:
Tip: This recursive function is a function that calls itself. It infinitely spawns processes and ends up in a denial of service attack. & is postfixed with the function call to bring the subprocess into the background. This is a dangerous code as it forks processes and, therefore, it is called a fork bomb.
You may find it difficult to interpret the above code. See Wikipedia page http://en.wikipedia.org/wiki/Fork_bomb for more details and interpretation of the fork bomb. It can be prevented by restricting the maximum number of processes that can be spawned from the config file /etc/security/limits.conf.

 

Specify -maxdepth and –mindepth as the third argument
Tip: -maxdepth and –mindepth should be specified as the third argument to the find. If they are specified as the fourth or further arguments, it may affect the efficiency of the find as it has to do unnecessary checks (for example, if –maxdepth is specified as the fourth argument and –type as the third argument, the find command first finds out all the files having the specified –type and then finds all of the matched files having the specified depth. However, if the depth were specified as the third argument and –type as the fourth, find could collect all the files having at most the specified depth and then check for the file type, which is the most efficient way of searching.

 

-exec with multiple commands
Tip: We cannot use multiple commands along with the –exec parameter. It accepts only a single command, but we can use a trick. Write multiple commands in a shell script (for example, commands.sh) and use it with –exec as follows:

-exec ./commands.sh {} \;

 

-n option for numeric sort
Tip: Always be careful about the -n option for numeric sort. The sort command treats alphabetical sort and numeric sort differently. Hence, in order to specify numeric sort the –n option should be provided.

 

The ## operator
Tip: The ## operator is more preferred over the # operator to extract an extension from a filename since the filename may contain multiple '.' characters. Since ## makes greedy match, it always extract extensions only.

 

Recursively search many files
Tip: To recursively search for a text over many directories of descendants use:

$ grep "text" . -R -n

This is one of the most frequently used commands by developers. It is used to find the file of source code in which a certain text exists.

 

Placing variable assignments
Tip: Usually, we place initial variable assignments, such as var=0; and statements to print the file header in the BEGIN block. In the END{} block, we place statements such as printing results and so on.

 

-d argument
Tip: The -d argument should always be given in quotes. If quotes are not used, & is interpreted by the shell to indicate this should be a background process.

 

Excluding a set of files from archiving
Tip: It is possible to exclude a set of files from archiving by specifying patterns. Use --exclude [PATTERN] for excluding files matched by wildcard patterns. For example, to exclude all .txt files from archiving use:

$ tar -cf arch.tar * --exclude "*.txt"

Note that the pattern should be enclosed in double quotes.

 

Using cpio command for absolute paths
Tip: By using cpio, we can also archive using files as absolute paths. /usr/somedir is an absolute path as it contains the full path starting from root (/).
A relative path will not start with / but it starts the path from the current directory. For example, test/file means that there is a directory test and the file is inside the test directory.
While extracting, cpio extracts to the absolute path itself. But incase of tar it removes the / in the absolute path and converts it as relative path.

 

PATH format
Tip: For the PATH format, if we use / at the end of the source, rsync will copy contents of that end directory specified in the source_path to the destination.
If / not at the end of the source, rsync will copy that end directory itself to the destination.

 

/ at the end of destination_path
Tip: If / is at the end of destination_path, rsync will copy the source to the destination directory. If / is not used at the end of the destination path, rsync will create a folder, named similar to the source directory, at the end of the destination path and copy the source into that directory.

 

wait command
Tip: The wait command enables a script to be terminated only after all its child process or background processes terminate or complete.

 

First argument of the sftp command
Tip: -oPort should be the first argument of the sftp command.

 

Running du DIRECTORY
Tip: Running du DIRECTORY will output a similar result, but it will show only the size consumed by subdirectories. However, they do not show the disk usage for each of the files. For printing the disk usage by files, -a is mandatory.

 

du DIRECTORY commands traversal
Tip: du can be restricted to traverse only a single file system by using the –x argument. Suppose du DIRECTORY is run, it will traverse through every possible subdirectory of DIRECTORY recursively. A subdirectory in the directory hierarchy may be a mount point (for example, /mnt/sda1 is a subdirectory of /mnt and it is a mount point for the device /dev/sda1). du will traverse that mount point and calculate the sum of disk usage for that device filesystem also. In order to prevent du from traversing and to calculate from other mount points or filesystems, use the -x flag along with other du options. du –x / will exclude all mount points in /mnt/ for disk usage calculation.

 

Use an absolute path for the executable
Tip: An executable binary of the time command is available at /usr/bin/time as well as a shell built-in named time exists. When we run time, it calls the shell built-in by default. The shell built-in time has limited options. Hence, we should use an absolute path for the executable (/usr/bin/time) for performing additional functionalities.

 

-x argument
Tip: The -x argument along with -a specifies to remove the TTY restriction imparted, by default, by ps. Usually, using ps without arguments prints processes that are attached to terminal only.

 

Parameters for -o
Tip: Parameters for -o are delimited by using the comma (,) operator. It should be noted that there is no space in between the comma operator and next parameter. Mostly, the -o option is combined with the -e (every) option (-oe) since it should list every process running in the system. However, when certain filters are used along with –o, such as those used for listing the processes owned by specified users, -e is not used along with –o. Usage of -e with a filter will nullify the filter and it will show all process entries.

 

pgrep command
Tip: pgrep requires only a portion of the command name as its input argument to extract a Bash command, for example, pgrep ash or pgrep bas will also work. But ps requires you to type the exact command.

 

apropos
Tip: Sometimes we need to search if some command related to a word exists. Then we can search the manpages for strings in the command. For this we can use:

apropos COMMAND

 

Summary

In this article we took a look at some tips and tricks on working with Linux shell script.


Further resources on this subject:


Linux Shell Scripting Cookbook Solve real-world shell scripting problems with over 110 simple but incredibly effective recipes
Published: January 2011
eBook Price: $26.99
Book Price: $44.99
See more
Select your format and quantity:

About the Author :


Sarath Lakshman

Sarath Lakshman is a 23 year old who was bitten by the Linux bug during his teenage years. He is a software engineer working in ZCloud engineering group at Zynga, India. He is a life hacker who loves to explore innovations. He is a GNU/Linux enthusiast and hactivist of free and open source software. He spends most of his time hacking with computers and having fun with his great friends. Sarath is well known as the developer of SLYNUX (2005)—a user friendly GNU/Linux distribution for Linux newbies. The free and open source software projects he has contributed to are PiTiVi Video editor, SLYNUX GNU/Linux distro, Swathantra Malayalam Computing, School-Admin, Istanbul, and the Pardus Project. He has authored many articles for the Linux For You magazine on various domains of FOSS technologies. He had made a contribution to several different open source projects during his multiple Google Summer of Code projects. Currently, he is exploring his passion about scalable distributed systems in his spare time. Sarath can be reached via his website http://www.sarathlakshman.com.

Books From Packt


Cacti 0.8 Beginner's Guide
Cacti 0.8 Beginner's Guide

Koha 3 Library Management System
Koha 3 Library Management System

Scalix: Linux Administrator's Guide
Scalix: Linux Administrator's Guide

Asterisk 1.4 – the Professional’s Guide
Asterisk 1.4 – the Professional’s Guide

OpenVPN 2 Cookbook
OpenVPN 2 Cookbook

iPhone Applications Tune-Up
iPhone Applications Tune-Up

Getting started with Audacity 1.3
Getting started with Audacity 1.3

Hacking Vim 7.2
Hacking Vim 7.2


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software