Linux Shell Scripting Cookbook, Second Edition

Don’t neglect the shell – this book will empower you to use simple commands to perform complex tasks. Whether you’re a casual or advanced Linux user, the cookbook approach makes it all so brilliantly accessible and, above all, useful.

Linux Shell Scripting Cookbook, Second Edition

Cookbook
Shantanu Tushar, Sarath Lakshman

Don’t neglect the shell – this book will empower you to use simple commands to perform complex tasks. Whether you’re a casual or advanced Linux user, the cookbook approach makes it all so brilliantly accessible and, above all, useful.
$10.00
$44.99
RRP $26.99
RRP $44.99
eBook
Print + eBook
$12.99 p/month

Get Access

Get Unlimited Access to every Packt eBook and Video course

Enjoy full and instant access to over 3000 books and videos – you’ll find everything you need to stay ahead of the curve and make sure you can always get the job done.

+ Collection
Free Sample

Book Details

ISBN 139781782162742
Paperback384 pages

About This Book

  • Master the art of crafting one-liner command sequence to perform text processing, digging data from files, backups to sysadmin tools, and a lot more
  • And if powerful text processing isn't enough, see how to make your scripts interact with the web-services like Twitter, Gmail
  • Explores the possibilities with the shell in a simple and elegant way - you will see how to effectively solve problems in your day to day life

Who This Book Is For

This book is both for the casual GNU/Linux users who want to do amazing things with the shell, and for advanced users looking for ways to make their lives with the shell more productive.

You can start writing scripts and one-liners by simply looking at the similar recipe and its descriptions without any working knowledge of shell scripting or Linux. Intermediate/advanced users as well as system administrators/ developers and programmers can use this book as a reference when they face problems while coding.

Table of Contents

Chapter 1: Shell Something Out
Introduction
Printing in the terminal
Playing with variables and environment variables
Function to prepend to environment variables
Math with the shell
Playing with file descriptors and redirection
Arrays and associative arrays
Visiting aliases
Grabbing information about the terminal
Getting and setting dates and delays
Debugging the script
Functions and arguments
Reading the output of a sequence of commands in a variable
Reading n characters without pressing the return key
Running a command until it succeeds
Field separators and iterators
Comparisons and tests
Chapter 2: Have a Good Command
Introduction
Concatenating with cat
Recording and playing back of terminal sessions
Finding files and file listing
Playing with xargs
Translating with tr
Checksum and verification
Cryptographic tools and hashes
Sorting unique and duplicates
Temporary file naming and random numbers
Splitting files and data
Slicing filenames based on extension
Renaming and moving files in bulk
Spell checking and dictionary manipulation
Automating interactive input
Making commands quicker by running parallel processes
Chapter 3: File In, File Out
Introduction
Generating files of any size
The intersection and set difference (A-B) on text files
Finding and deleting duplicate files
Working with file permissions, ownership, and the sticky bit
Making files immutable
Generating blank files in bulk
Finding symbolic links and their targets
Enumerating file type statistics
Using loopback files
Creating ISO files and hybrid ISO
Finding the difference between files, patching
Using head and tail for printing the last or first 10 lines
Listing only directories – alternative methods
Fast command-line navigation using pushd and popd
Counting the number of lines, words, and characters in a file
Printing the directory tree
Chapter 4: Texting and Driving
Introduction
Using regular expressions
Searching and mining a text inside a file with grep
Cutting a file column-wise with cut
Using sed to perform text replacement
Using awk for advanced text processing
Finding the frequency of words used in a given file
Compressing or decompressing JavaScript
Merging multiple files as columns
Printing the nth word or column in a file or line
Printing text between line numbers or patterns
Printing lines in the reverse order
Parsing e-mail addresses and URLs from text
Removing a sentence in a file containing a word
Replacing a pattern with text in all the files in a directory
Text slicing and parameter operations
Chapter 5: Tangled Web? Not At All!
Introduction
Downloading from a web page
Downloading a web page as plain text
A primer on cURL
Accessing Gmail e-mails from the command line
Parsing data from a website
Image crawler and downloader
Web photo album generator
Twitter command-line client
Creating a "define" utility by using the Web backend
Finding broken links in a website
Tracking changes to a website
Posting to a web page and reading the response
Chapter 6: The Backup Plan
Introduction
Archiving with tar
Archiving with cpio
Compressing data with gzip
Archiving and compressing with zip
Faster archiving with pbzip2
Creating filesystems with compression
Backup snapshots with rsync
Version control-based backup with Git
Creating entire disk images using fsarchiver
Chapter 7: The Old-boy Network
Introduction
Setting up the network
Let us ping!
Listing all the machines alive on a network
Running commands on a remote host with SSH
Transferring files through the network
Connecting to a wireless network
Password-less auto-login with SSH
Port forwarding using SSH
Mounting a remote drive at a local mount point
Network traffic and port analysis
Creating arbitrary sockets
Sharing an Internet connection
Basic firewall using iptables
Chapter 8: Put on the Monitor's Cap
Introduction
Monitoring disk usage
Calculating the execution time for a command
Collecting information about logged in users, boot logs, and boot failures
Listing the top 10 CPU consuming processes in an hour
Monitoring command outputs with watch
Logging access to files and directories
Logfile management with logrotate
Logging with syslog
Monitoring user logins to find intruders
Remote disk usage health monitor
Finding out active user hours on a system
Measuring and optimizing power usage
Monitoring disk activity
Checking disks and filesystems for errors
Chapter 9: Administration Calls
Introduction
Gathering information about processes
Killing processes and send or respond to signals
Sending messages to user terminals
Gathering system information
Using /proc for gathering information
Scheduling with cron
Writing and reading the MySQL database from Bash
User administration script
Bulk image resizing and format conversion
Taking screenshots from the terminal
Managing multiple terminals from one

What You Will Learn

  • Explore a variety of regular usage tasks and how it can be made faster using shell command
  • Write shell scripts that can dig data from web and process it with few lines of code
  • Use different kinds of tools together to create solutions
  • Interact with simple web API from scripts
  • Perform and automate tasks such as automating backups and restore with archiving tools
  • Create and maintain file/folder archives, compression formats and encrypting techniques with shell
  • Set up Ethernet and Wireless LAN with the shell script
  • Monitor different activities on the network using logging techniques

In Detail

The shell remains one of the most powerful tools on a computer system — yet a large number of users are unaware of how much one can accomplish with it. Using a combination of simple commands, we will see how to solve complex problems in day to day computer usage.

Linux Shell Scripting Cookbook, Second Edition will take you through useful real-world recipes designed to make your daily life easy when working with the shell. The book shows the reader how to effectively use the shell to accomplish complex tasks with ease.

The book discusses basics of using the shell, general commands and proceeds to show the reader how to use them to perform complex tasks with ease.

Starting with the basics of the shell, we will learn simple commands with their usages allowing us to perform operations on files of different kind. The book then proceeds to explain text processing, web interaction and concludes with backups, monitoring and other sysadmin tasks.

Linux Shell Scripting Cookbook, Second Edition serves as an excellent guide to solving day to day problems using the shell and few powerful commands together to create solutions.

Authors

Table of Contents

Chapter 1: Shell Something Out
Introduction
Printing in the terminal
Playing with variables and environment variables
Function to prepend to environment variables
Math with the shell
Playing with file descriptors and redirection
Arrays and associative arrays
Visiting aliases
Grabbing information about the terminal
Getting and setting dates and delays
Debugging the script
Functions and arguments
Reading the output of a sequence of commands in a variable
Reading n characters without pressing the return key
Running a command until it succeeds
Field separators and iterators
Comparisons and tests
Chapter 2: Have a Good Command
Introduction
Concatenating with cat
Recording and playing back of terminal sessions
Finding files and file listing
Playing with xargs
Translating with tr
Checksum and verification
Cryptographic tools and hashes
Sorting unique and duplicates
Temporary file naming and random numbers
Splitting files and data
Slicing filenames based on extension
Renaming and moving files in bulk
Spell checking and dictionary manipulation
Automating interactive input
Making commands quicker by running parallel processes
Chapter 3: File In, File Out
Introduction
Generating files of any size
The intersection and set difference (A-B) on text files
Finding and deleting duplicate files
Working with file permissions, ownership, and the sticky bit
Making files immutable
Generating blank files in bulk
Finding symbolic links and their targets
Enumerating file type statistics
Using loopback files
Creating ISO files and hybrid ISO
Finding the difference between files, patching
Using head and tail for printing the last or first 10 lines
Listing only directories – alternative methods
Fast command-line navigation using pushd and popd
Counting the number of lines, words, and characters in a file
Printing the directory tree
Chapter 4: Texting and Driving
Introduction
Using regular expressions
Searching and mining a text inside a file with grep
Cutting a file column-wise with cut
Using sed to perform text replacement
Using awk for advanced text processing
Finding the frequency of words used in a given file
Compressing or decompressing JavaScript
Merging multiple files as columns
Printing the nth word or column in a file or line
Printing text between line numbers or patterns
Printing lines in the reverse order
Parsing e-mail addresses and URLs from text
Removing a sentence in a file containing a word
Replacing a pattern with text in all the files in a directory
Text slicing and parameter operations
Chapter 5: Tangled Web? Not At All!
Introduction
Downloading from a web page
Downloading a web page as plain text
A primer on cURL
Accessing Gmail e-mails from the command line
Parsing data from a website
Image crawler and downloader
Web photo album generator
Twitter command-line client
Creating a "define" utility by using the Web backend
Finding broken links in a website
Tracking changes to a website
Posting to a web page and reading the response
Chapter 6: The Backup Plan
Introduction
Archiving with tar
Archiving with cpio
Compressing data with gzip
Archiving and compressing with zip
Faster archiving with pbzip2
Creating filesystems with compression
Backup snapshots with rsync
Version control-based backup with Git
Creating entire disk images using fsarchiver
Chapter 7: The Old-boy Network
Introduction
Setting up the network
Let us ping!
Listing all the machines alive on a network
Running commands on a remote host with SSH
Transferring files through the network
Connecting to a wireless network
Password-less auto-login with SSH
Port forwarding using SSH
Mounting a remote drive at a local mount point
Network traffic and port analysis
Creating arbitrary sockets
Sharing an Internet connection
Basic firewall using iptables
Chapter 8: Put on the Monitor's Cap
Introduction
Monitoring disk usage
Calculating the execution time for a command
Collecting information about logged in users, boot logs, and boot failures
Listing the top 10 CPU consuming processes in an hour
Monitoring command outputs with watch
Logging access to files and directories
Logfile management with logrotate
Logging with syslog
Monitoring user logins to find intruders
Remote disk usage health monitor
Finding out active user hours on a system
Measuring and optimizing power usage
Monitoring disk activity
Checking disks and filesystems for errors
Chapter 9: Administration Calls
Introduction
Gathering information about processes
Killing processes and send or respond to signals
Sending messages to user terminals
Gathering system information
Using /proc for gathering information
Scheduling with cron
Writing and reading the MySQL database from Bash
User administration script
Bulk image resizing and format conversion
Taking screenshots from the terminal
Managing multiple terminals from one

Book Details

ISBN 139781782162742
Paperback384 pages
Read More