Linux Shell Scripting Cookbook - Third Edition

Do amazing things with the shell
Preview in Mapt

Linux Shell Scripting Cookbook - Third Edition

Clif Flynt, Sarath Lakshman, Shantanu Tushar

Do amazing things with the shell
Mapt Subscription
FREE
$29.99/m after trial
eBook
$28.00
RRP $39.99
Save 29%
Print + eBook
$49.99
RRP $49.99
What do I get with a Mapt Pro subscription?
  • Unlimited access to all Packt’s 5,000+ eBooks and Videos
  • Early Access content, Progress Tracking, and Assessments
  • 1 Free eBook or Video to download and keep every month after trial
What do I get with an eBook?
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with Print & eBook?
  • Get a paperback copy of the book delivered to you
  • Download this book in EPUB, PDF, MOBI formats
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
What do I get with a Video?
  • Download this Video course in MP4 format
  • DRM FREE - read and interact with your content when you want, where you want, and how you want
  • Access this title in the Mapt reader
$0.00
$28.00
$49.99
$29.99p/m after trial
RRP $39.99
RRP $49.99
Subscription
eBook
Print + eBook
Start 30 Day Trial

Frequently bought together


Linux Shell Scripting Cookbook - Third Edition Book Cover
Linux Shell Scripting Cookbook - Third Edition
$ 39.99
$ 28.00
Microsoft Exchange Server PowerShell Cookbook - Third Edition Book Cover
Microsoft Exchange Server PowerShell Cookbook - Third Edition
$ 47.99
$ 33.60
Buy 2 for $35.00
Save $52.98
Add to Cart
Subscribe and access every Packt eBook & Video.
 
  • 5,000+ eBooks & Videos
  • 50+ New titles a month
  • 1 Free eBook/Video to keep every month
Start Free Trial
 

Book Details

ISBN 139781785881985
Paperback552 pages

Book Description

The shell is the most powerful tool your computer provides. Despite having it at their fingertips, many users are unaware of how much the shell can accomplish.

Using the shell, you can generate databases and web pages from sets of files, automate monotonous admin tasks such as system backups, monitor your system's health and activity, identify network bottlenecks and system resource hogs, and more.

This book will show you how to do all this and much more.

This book, now in its third edition, describes the exciting new features in the newest Linux distributions to help you accomplish more than you imagine. It shows how to use simple commands to automate complex tasks, automate web interactions, download videos, set up containers and cloud servers, and even get free SSL certificates.

Starting with the basics of the shell, you will learn simple commands and how to apply them to real-world issues. From there, you'll learn text processing, web interactions, network and system monitoring, and system tuning.

Software engineers will learn how to examine system applications, how to use modern software management tools such as git and fossil for their own work, and how to submit patches to open-source projects.

Finally, you'll learn how to set up Linux Containers and Virtual machines and even run your own Cloud server with a free SSL Certificate from letsencrypt.org.

Table of Contents

Chapter 1: Shell Something Out
Introduction
Displaying output in a terminal
Using variables and environment variables
Function to prepend to environment variables
Math with the shell
Playing with file descriptors and redirection
Arrays and associative arrays
Visiting aliases
Grabbing information about the terminal
Getting and setting dates and delays
Debugging the script
Functions and arguments
Sending output from one command to another
Reading n characters without pressing the return key
Running a command until it succeeds
Field separators and iterators
Comparisons and tests
Customizing bash with configuration files
Chapter 2: Have a Good Command
Introduction
Concatenating with cat
Recording and playing back terminal sessions
Finding files and file listing
Playing with xargs
Translating with tr
Checksum and verification
Cryptographic tools and hashes
Sorting unique and duplicate lines
Temporary file naming and random numbers
Splitting files and data
Slicing filenames based on extensions
Renaming and moving files in bulk
Spell–checking and dictionary manipulation
Automating interactive input
Making commands quicker by running parallel processes
Examining a directory, files and subdirectories in it
Chapter 3: File In, File Out
Introduction
Generating files of any size
The intersection and set difference (A-B) on text files
Finding and deleting duplicate files
Working with file permissions, ownership, and the sticky bit
Making files immutable
Generating blank files in bulk
Finding symbolic links and their targets
Enumerating file type statistics
Using loopback files
Creating ISO files and hybrid ISO
Finding the difference between files, and patching
Using head and tail for printing the last or first 10 lines
Listing only directories - alternative methods
Fast command-line navigation using pushd and popd
Counting the number of lines, words, and characters in a file
Printing the directory tree
Manipulating video and image files
Chapter 4: Texting and Driving
Introduction
Using regular expressions
Searching and mining text inside a file with grep
Cutting a file column-wise with cut
Using sed to perform text replacement
Using awk for advanced text processing
Finding the frequency of words used in a given file
Compressing or decompressing JavaScript
Merging multiple files as columns
Printing the nth word or column in a file or line
Printing text between line numbers or patterns
Printing lines in the reverse order
Parsing e-mail address and URLs from text
Removing a sentence in a file containing a word
Replacing a pattern with text in all the files in a directory
Text slicing and parameter operations
Chapter 5: Tangled Web? Not At All!
Introduction
Downloading from a web page
Downloading a web page as plain text
A primer on cURL
Accessing unread Gmail e-mails from the command line
Parsing data from a website
Image crawler and downloader
Web photo album generator
Twitter command-line client
Accessing word definitions via a web server
Finding broken links in a website
Tracking changes to a website
Posting to a web page and reading the response
Downloading a video from the Internet
Summarizing text with OTS
Translating text from the command line
Chapter 6: Repository Management
Introduction
Creating a new git repository
Cloning a remote git repository
Adding and committing changes with git
Creating and merging branches with git
Sharing your work
Pushing a branch to a server
Checking the status of a git repository
Viewing git history
Finding bugs
Tagging snapshots
Committing message ethics
Using fossil
Creating a new fossil repository
Cloning a remote fossil repository
Opening a fossil project
Adding and committing changes with fossil
Using branches and forks with fossil
Sharing your work with fossil
Updating your local fossil repository
Checking the status of a fossil repository
Viewing fossil history
Chapter 7: The Backup Plan
Introduction
Archiving with tar
Archiving with cpio
Compressing data with gzip
Archiving and compressing with zip
Faster archiving with pbzip2
Creating filesystems with compression
Backing up snapshots with rsync
Differential archives
Creating entire disk images using fsarchiver
Chapter 8: The Old-Boy Network
Introduction
Setting up the network
Let us ping!
Tracing IP routes
Listing all available machines on a network
Running commands on a remote host with SSH
Running graphical commands on a remote machine
Transferring files through the network
Connecting to a wireless network
Password-less auto-login with SSH
Port forwarding using SSH
Mounting a remote drive at a local mount point
Network traffic and port analysis
Measuring network bandwidth
Creating arbitrary sockets
Building a bridge
Sharing an Internet connection
Basic firewall using iptables
Creating a Virtual Private Network
Chapter 9: Put On the Monitors Cap
Introduction
Monitoring disk usage
Calculating the execution time for a command
Collecting information about logged in users, boot logs, and boot failures
Listing the top ten CPU– consuming processes in an hour
Monitoring command outputs with watch
Logging access to files and directories
Logging with syslog
Managing log files with logrotate
Monitoring user logins to find intruders
Monitoring remote disk usage health
Determining active user hours on a system
Measuring and optimizing power usage
Monitoring disk activity
Checking disks and filesystems for errors
Examining disk health
Getting disk statistics
Chapter 10: Administration Calls
Introduction
Gathering information about processes
What's what – which, whereis, whatis, and file  
Killing processes, and sending and responding to signals
Sending messages to user terminals
The /proc filesystem
Gathering system information
Scheduling with a cron
Database styles and uses
Writing and reading SQLite databases
Writing and reading a MySQL database from Bash
User administration scripts
Bulk image resizing and format conversion
Taking screenshots from the terminal
Managing multiple terminals from one
Chapter 11: Tracing the Clues
Introduction
Tracing packets with tcpdump
Finding packets with ngrep
Tracing network routes with ip
Tracing system calls with strace
Tracing dynamic library functions with ltrace
Chapter 12: Tuning a Linux System
Introduction
Identifying services
Gathering socket data with ss
Gathering system I/O usage with dstat
Identifying a resource hog with pidstat
Tuning the Linux kernel with sysctl
Tuning a Linux system with config files
Changing scheduler priority using the nice command
Chapter 13: Containers, Virtual Machines, and the Cloud
Introduction
Using Linux containers
Using Docker
Using Virtual Machines in Linux
Linux in the cloud

What You Will Learn

  • Interact with websites via scripts
  • Write shell scripts to mine and process data from the Web
  • Automate system backups and other repetitive tasks with crontab
  • Create, compress, and encrypt archives of your critical data.
  • Configure and monitor Ethernet and wireless networks
  • Monitor and log network and system activity
  • Tune your system for optimal performance
  • Improve your system's security
  • Identify resource hogs and network bottlenecks
  • Extract audio from video files
  • Create web photo albums
  • Use git or fossil to manage revision control and interact with FOSS projects
  • Create and maintain Linux containers and Virtual Machines
  • Run a private Cloud server

Authors

Table of Contents

Chapter 1: Shell Something Out
Introduction
Displaying output in a terminal
Using variables and environment variables
Function to prepend to environment variables
Math with the shell
Playing with file descriptors and redirection
Arrays and associative arrays
Visiting aliases
Grabbing information about the terminal
Getting and setting dates and delays
Debugging the script
Functions and arguments
Sending output from one command to another
Reading n characters without pressing the return key
Running a command until it succeeds
Field separators and iterators
Comparisons and tests
Customizing bash with configuration files
Chapter 2: Have a Good Command
Introduction
Concatenating with cat
Recording and playing back terminal sessions
Finding files and file listing
Playing with xargs
Translating with tr
Checksum and verification
Cryptographic tools and hashes
Sorting unique and duplicate lines
Temporary file naming and random numbers
Splitting files and data
Slicing filenames based on extensions
Renaming and moving files in bulk
Spell–checking and dictionary manipulation
Automating interactive input
Making commands quicker by running parallel processes
Examining a directory, files and subdirectories in it
Chapter 3: File In, File Out
Introduction
Generating files of any size
The intersection and set difference (A-B) on text files
Finding and deleting duplicate files
Working with file permissions, ownership, and the sticky bit
Making files immutable
Generating blank files in bulk
Finding symbolic links and their targets
Enumerating file type statistics
Using loopback files
Creating ISO files and hybrid ISO
Finding the difference between files, and patching
Using head and tail for printing the last or first 10 lines
Listing only directories - alternative methods
Fast command-line navigation using pushd and popd
Counting the number of lines, words, and characters in a file
Printing the directory tree
Manipulating video and image files
Chapter 4: Texting and Driving
Introduction
Using regular expressions
Searching and mining text inside a file with grep
Cutting a file column-wise with cut
Using sed to perform text replacement
Using awk for advanced text processing
Finding the frequency of words used in a given file
Compressing or decompressing JavaScript
Merging multiple files as columns
Printing the nth word or column in a file or line
Printing text between line numbers or patterns
Printing lines in the reverse order
Parsing e-mail address and URLs from text
Removing a sentence in a file containing a word
Replacing a pattern with text in all the files in a directory
Text slicing and parameter operations
Chapter 5: Tangled Web? Not At All!
Introduction
Downloading from a web page
Downloading a web page as plain text
A primer on cURL
Accessing unread Gmail e-mails from the command line
Parsing data from a website
Image crawler and downloader
Web photo album generator
Twitter command-line client
Accessing word definitions via a web server
Finding broken links in a website
Tracking changes to a website
Posting to a web page and reading the response
Downloading a video from the Internet
Summarizing text with OTS
Translating text from the command line
Chapter 6: Repository Management
Introduction
Creating a new git repository
Cloning a remote git repository
Adding and committing changes with git
Creating and merging branches with git
Sharing your work
Pushing a branch to a server
Checking the status of a git repository
Viewing git history
Finding bugs
Tagging snapshots
Committing message ethics
Using fossil
Creating a new fossil repository
Cloning a remote fossil repository
Opening a fossil project
Adding and committing changes with fossil
Using branches and forks with fossil
Sharing your work with fossil
Updating your local fossil repository
Checking the status of a fossil repository
Viewing fossil history
Chapter 7: The Backup Plan
Introduction
Archiving with tar
Archiving with cpio
Compressing data with gzip
Archiving and compressing with zip
Faster archiving with pbzip2
Creating filesystems with compression
Backing up snapshots with rsync
Differential archives
Creating entire disk images using fsarchiver
Chapter 8: The Old-Boy Network
Introduction
Setting up the network
Let us ping!
Tracing IP routes
Listing all available machines on a network
Running commands on a remote host with SSH
Running graphical commands on a remote machine
Transferring files through the network
Connecting to a wireless network
Password-less auto-login with SSH
Port forwarding using SSH
Mounting a remote drive at a local mount point
Network traffic and port analysis
Measuring network bandwidth
Creating arbitrary sockets
Building a bridge
Sharing an Internet connection
Basic firewall using iptables
Creating a Virtual Private Network
Chapter 9: Put On the Monitors Cap
Introduction
Monitoring disk usage
Calculating the execution time for a command
Collecting information about logged in users, boot logs, and boot failures
Listing the top ten CPU– consuming processes in an hour
Monitoring command outputs with watch
Logging access to files and directories
Logging with syslog
Managing log files with logrotate
Monitoring user logins to find intruders
Monitoring remote disk usage health
Determining active user hours on a system
Measuring and optimizing power usage
Monitoring disk activity
Checking disks and filesystems for errors
Examining disk health
Getting disk statistics
Chapter 10: Administration Calls
Introduction
Gathering information about processes
What's what – which, whereis, whatis, and file  
Killing processes, and sending and responding to signals
Sending messages to user terminals
The /proc filesystem
Gathering system information
Scheduling with a cron
Database styles and uses
Writing and reading SQLite databases
Writing and reading a MySQL database from Bash
User administration scripts
Bulk image resizing and format conversion
Taking screenshots from the terminal
Managing multiple terminals from one
Chapter 11: Tracing the Clues
Introduction
Tracing packets with tcpdump
Finding packets with ngrep
Tracing network routes with ip
Tracing system calls with strace
Tracing dynamic library functions with ltrace
Chapter 12: Tuning a Linux System
Introduction
Identifying services
Gathering socket data with ss
Gathering system I/O usage with dstat
Identifying a resource hog with pidstat
Tuning the Linux kernel with sysctl
Tuning a Linux system with config files
Changing scheduler priority using the nice command
Chapter 13: Containers, Virtual Machines, and the Cloud
Introduction
Using Linux containers
Using Docker
Using Virtual Machines in Linux
Linux in the cloud

Book Details

ISBN 139781785881985
Paperback552 pages
Read More

Read More Reviews

Recommended for You

Swift 3 Functional Programming Book Cover
Swift 3 Functional Programming
$ 35.99
$ 25.20
iOS 10 Programming for Beginners Book Cover
iOS 10 Programming for Beginners
$ 39.99
$ 28.00
Swift 3 Protocol-Oriented Programming - Second Edition Book Cover
Swift 3 Protocol-Oriented Programming - Second Edition
$ 31.99
$ 22.40
Microsoft Exchange Server PowerShell Cookbook - Third Edition Book Cover
Microsoft Exchange Server PowerShell Cookbook - Third Edition
$ 47.99
$ 33.60
Linux Shell Scripting Cookbook, Second Edition Book Cover
Linux Shell Scripting Cookbook, Second Edition
$ 26.99
$ 5.40
Kali Linux 2 - Assuring Security by Penetration Testing - Third Edition Book Cover
Kali Linux 2 - Assuring Security by Penetration Testing - Third Edition
$ 35.99
$ 25.20