Linux Shell Scripting Cookbook, Second Edition


Linux Shell Scripting Cookbook, Second Edition
eBook: $26.99
Formats: PDF, PacktLib, ePub and Mobi formats
$22.94
save 15%!
Print + free eBook + free PacktLib access to the book: $71.98    Print cover: $44.99
$44.99
save 37%!
Free Shipping!
UK, US, Europe and selected countries in Asia.
Also available on:
Overview
Table of Contents
Author
Reviews
Support
Sample Chapters
  • Master the art of crafting one-liner command sequence to perform text processing, digging data from files, backups to sysadmin tools, and a lot more
  • And if powerful text processing isn't enough, see how to make your scripts interact with the web-services like Twitter, Gmail
  • Explores the possibilities with the shell in a simple and elegant way - you will see how to effectively solve problems in your day to day life

Book Details

Language : English
Paperback : 384 pages [ 235mm x 191mm ]
Release Date : May 2013
ISBN : 1782162747
ISBN 13 : 9781782162742
Author(s) : Shantanu Tushar, Sarath Lakshman
Topics and Technologies : All Books, Other, Cookbooks, Linux Servers, Open Source

Table of Contents

Preface
Chapter 1: Shell Something Out
Chapter 2: Have a Good Command
Chapter 3: File In, File Out
Chapter 4: Texting and Driving
Chapter 5: Tangled Web? Not At All!
Chapter 6: The Backup Plan
Chapter 7: The Old-boy Network
Chapter 8: Put on the Monitor's Cap
Chapter 9: Administration Calls
Index
  • Chapter 1: Shell Something Out
    • Introduction
    • Printing in the terminal
    • Playing with variables and environment variables
    • Function to prepend to environment variables
    • Math with the shell
    • Playing with file descriptors and redirection
    • Arrays and associative arrays
    • Visiting aliases
    • Grabbing information about the terminal
    • Getting and setting dates and delays
    • Debugging the script
    • Functions and arguments
    • Reading the output of a sequence of commands
    • Reading n characters without pressing the return key
    • Running a command until it succeeds
    • Field separators and iterators
    • Comparisons and tests
    • Chapter 2: Have a Good Command
      • Introduction
      • Concatenating with cat
      • Recording and playing back of terminal sessions
      • Finding files and file listing
      • Playing with xargs
      • Translating with tr
      • Checksum and verification
      • Cryptographic tools and hashes
      • Sorting unique and duplicates
      • Temporary file naming and random numbers
      • Splitting files and data
      • Slicing filenames based on extension
      • Renaming and moving files in bulk
      • Spell checking and dictionary manipulation
      • Automating interactive input
      • Making commands quicker by running parallel processes
      • Chapter 3: File In, File Out
        • Introduction
        • Generating files of any size
        • The intersection and set difference (A-B) on text files
        • Finding and deleting duplicate files
        • Working with file permissions, ownership, and the sticky bit
        • Making files immutable
        • Generating blank files in bulk
        • Finding symbolic links and their targets
        • Enumerating file type statistics
        • Using loopback files
        • Creating ISO files and hybrid ISO
        • Finding the difference between files, patching
        • Using head and tail for printing the last or first 10 lines
        • Listing only directories – alternative methods
        • Fast command-line navigation using pushd and popd
        • Counting the number of lines, words, and characters in a file
        • Printing the directory tree
        • Chapter 4: Texting and Driving
          • Introduction
          • Using regular expressions
          • Searching and mining a text inside a file with grep
          • Cutting a file column-wise with cut
          • Using sed to perform text replacement
          • Using awk for advanced text processing
          • Finding the frequency of words used in a given file
          • Compressing or decompressing JavaScript
          • Merging multiple files as columns
          • Printing the nth word or column in a file or line
          • Printing text between line numbers or patterns
          • Printing lines in the reverse order
          • Parsing e-mail addresses and URLs from a text
          • Removing a sentence in a file containing a word
          • Replacing a pattern with text in all the files in a directory
          • Text slicing and parameter operations
          • Chapter 5: Tangled Web? Not At All!
            • Introduction
            • Downloading from a web page
            • Downloading a web page as plain text
            • A primer on cURL
            • Accessing Gmail e-mails from the command line
            • Parsing data from a website
            • Image crawler and downloader
            • Web photo album generator
            • Twitter command-line client
            • Creating a "define" utility by using the Web backend
            • Finding broken links in a website
            • Tracking changes to a website
            • Posting to a web page and reading the response
            • Chapter 6: The Backup Plan
              • Introduction
              • Archiving with tar
              • Archiving with cpio
              • Compressing data with gzip
              • Archiving and compressing with zip
              • Faster archiving with pbzip2
              • Creating filesystems with compression
              • Backup snapshots with rsync
              • Version control based backup with Git
              • Creating entire disk images with fsarchiver
              • Chapter 7: The Old-boy Network
                • Introduction
                • Setting up the network
                • Let us ping!
                • Listing all the machines alive on a network
                • Running commands on a remote host with SSH
                • Transferring files through the network
                • Connecting to a wireless network
                • Password-less auto-login with SSH
                • Port forwarding using SSH
                • Mounting a remote drive at a local mount point
                • Network traffic and port analysis
                • Creating arbitrary sockets
                • Sharing an Internet connection
                • Basic firewall using iptables
                • Chapter 8: Put on the Monitor's Cap
                  • Introduction
                  • Monitoring disk usage
                  • Calculating the execution time for a command
                  • Collecting information about logged in users, boot logs, and boot failures
                  • Listing the top 10 CPU consuming processes in an hour
                  • Monitoring command outputs with watch
                  • Logging access to files and directories
                  • Logfile management with logrotate
                  • Logging with syslog
                  • Monitoring user logins to find intruders
                  • Remote disk usage health monitor
                  • Finding out active user hours on a system
                  • Measuring and optimizing power usage
                  • Monitoring disk activity
                  • Checking disks and filesystems for errors
                  • Chapter 9: Administration Calls
                    • Introduction
                    • Gathering information about processes
                    • Killing processes and send or respond to signals
                    • Sending messages to user terminals
                    • Gathering system information
                    • Using /proc for gathering information
                    • Scheduling with cron
                    • Writing and reading the MySQL database from Bash
                    • User administration script
                    • Bulk image resizing and format conversion
                    • Taking screenshots from the terminal
                    • Managing multiple terminals from one

                    Shantanu Tushar

                    Shantanu Tushar is an advanced GNU/Linux user since his college days. He works as an application developer and contributes to the software in the KDE projects. Shantanu has been fascinated by computers since he was a child, and spent most of his high school time writing C code to perform daily activities. Since he started using GNU/Linux, he has been using shell scripts to make the computer do all the hard work for him. He also takes time to visit students at various colleges to introduce them to the power of Free Software, including its various tools. Shantanu is a well-known contributor in the KDE community and works on Calligra, Gluon and the Plasma subprojects. He looks after maintaining Calligra Active – KDE's office document viewer for tablets, Plasma Media Center, and the Gluon Player. One day, he believes, programming will be so easy that everybody will love to write programs for their computers. Shantanu can be reached by e-mail on shantanu@kde.org, shantanutushar on identi.ca/twitter, or his website http://www.shantanutushar.com.

                    Sarath Lakshman

                    Sarath Lakshman is a 23 year old who was bitten by the Linux bug during his teenage years. He is a software engineer working in ZCloud engineering group at Zynga, India. He is a life hacker who loves to explore innovations. He is a GNU/Linux enthusiast and hactivist of free and open source software. He spends most of his time hacking with computers and having fun with his great friends. Sarath is well known as the developer of SLYNUX (2005)—a user friendly GNU/Linux distribution for Linux newbies. The free and open source software projects he has contributed to are PiTiVi Video editor, SLYNUX GNU/Linux distro, Swathantra Malayalam Computing, School-Admin, Istanbul, and the Pardus Project. He has authored many articles for the Linux For You magazine on various domains of FOSS technologies. He had made a contribution to several different open source projects during his multiple Google Summer of Code projects. Currently, he is exploring his passion about scalable distributed systems in his spare time. Sarath can be reached via his website http://www.sarathlakshman.com.

                    Code Downloads

                    Download the code and support files for this book.


                    Submit Errata

                    Please let us know if you have found any errors not listed on this list by completing our errata submission form. Our editors will check them and add them to this list. Thank you.


                    Errata

                    - 8 submitted: last submission 15 May 2014

                    Chapter 1. Shell Something  
                    Out > Arrays and associative arrays
                    
                     The errata.: $ ass_array'index2]=val2
                    typo
                    
                    
                    Solution: $ ass_array[index2]=val2

                    Errata Type: Typo Errata Page: 29

                    In chapter 'Shell something out' > Arrays and associative arrays:
                    $ echo ${!array_var[@]
                    should be
                    $ echo ${!array_var[@]}

                    Errata Type: Typo | Errata page: 21

                    echo "10^10" | bc #Square

                    Should be

                    echo "10^2" | bc #Square

                    Errata Type: Code | Errata page: 86

                    $ sort -nk 2,3 data.txt

                    the right format is

                    $ sort -nk 1.2,1.3 data.txt

                    "The format is F[.C] where F is field number and C is the (optional) character position"

                    Another command which needs to be fixed is on the same page 86, instead of-

                    $ sort -nk 1,1 data.txt

                    the right format will be

                    $ sort -nk 1.1,1.1 data.txt

                    Errata Type: Code | Errata page: 11

                    The code is:

                    $ echo "Hello world \!" #Escape character \ prefixed.

                    It should be:

                    echo Hello world \! #Escape character \ prefixed.

                    Errata Type: Code | Errata page: 23

                    The code is:

                    $ cmd 2>&1 output.txt

                    It should be:

                    cmd > output.txt 2>&1

                    Errata Type: Technical| Errata page: 21

                    The second line on this page is:

                    Let us convert from decimal to binary, and binary to octal:

                    It should be:

                    Let us convert from decimal to binary, and binary to decimal:

                    Errata Type: Typo| Errata page: 17

                    The last line on this page is:

                    LD_LIBRARY_PATH=/usr/lib;/lib

                    It should be:

                    LD_LIBRARY_PATH=/usr/lib:/lib



                     

                    Errata page: 18

                    The line in the "How to do it..." section is:

                    LD_LIBRARY_PATH=/opt/myapp/lib:/usr/lib;/lib


                    It should be:

                    LD_LIBRARY_PATH=/opt/myapp/lib:/usr/lib:/lib


                     

                    Note: Only colon : is supposed to use as a delimiter for both PATH and LD_LIBRARY_PATH.

                    Sample chapters

                    You can view our sample chapters and prefaces of this title on PacktLib or download sample chapters in PDF format.

                    Frequently bought together

                    Linux Shell Scripting Cookbook, Second Edition +    Java EE 7 Developer Handbook =
                    50% Off
                    the second eBook
                    Price for both: $44.10

                    Buy both these recommended eBooks together and get 50% off the cheapest eBook.

                    What you will learn from this book

                    • Explore a variety of regular usage tasks and how it can be made faster using shell command
                    • Write shell scripts that can dig data from web and process it with few lines of code
                    • Use different kinds of tools together to create solutions
                    • Interact with simple web API from scripts
                    • Perform and automate tasks such as automating backups and restore with archiving tools
                    • Create and maintain file/folder archives, compression formats and encrypting techniques with shell
                    • Set up Ethernet and Wireless LAN with the shell script
                    • Monitor different activities on the network using logging techniques

                    In Detail

                    The shell remains one of the most powerful tools on a computer system — yet a large number of users are unaware of how much one can accomplish with it. Using a combination of simple commands, we will see how to solve complex problems in day to day computer usage.

                    Linux Shell Scripting Cookbook, Second Edition will take you through useful real-world recipes designed to make your daily life easy when working with the shell. The book shows the reader how to effectively use the shell to accomplish complex tasks with ease.

                    The book discusses basics of using the shell, general commands and proceeds to show the reader how to use them to perform complex tasks with ease.

                    Starting with the basics of the shell, we will learn simple commands with their usages allowing us to perform operations on files of different kind. The book then proceeds to explain text processing, web interaction and concludes with backups, monitoring and other sysadmin tasks.

                    Linux Shell Scripting Cookbook, Second Edition serves as an excellent guide to solving day to day problems using the shell and few powerful commands together to create solutions.

                    Approach

                    This book is written in cookbook style and it offers learning through recipes with examples and illustrations. Each recipe contains step-by-step instructions about everything necessary to execute a particular task. The book is designed so that you can read it from start to end for beginner's or just open up any chapter and start following the recipes as a reference for advanced users.

                    Who this book is for

                    This book is both for the casual GNU/Linux users who want to do amazing things with the shell, and for advanced users looking for ways to make their lives with the shell more productive.

                    You can start writing scripts and one-liners by simply looking at the similar recipe and its descriptions without any working knowledge of shell scripting or Linux. Intermediate/advanced users as well as system administrators/ developers and programmers can use this book as a reference when they face problems while coding.

                    Code Download and Errata
                    Packt Anytime, Anywhere
                    Register Books
                    Print Upgrades
                    eBook Downloads
                    Video Support
                    Contact Us
                    Awards Voting Nominations Previous Winners
                    Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
                    Resources
                    Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software