Pattern Matching and Regular Expressions with sed and awk

In the previous chapter, you learned about a Linux system's startup process, from power-on to user login, and how to customize a Linux system environment.

In this chapter, we will cover the following topics:

• Understanding regular expressions
• Stream editor (sed) for text processing
• Using awk for text processing

The basics of regular expressions

A sequence of characters that have certain patterns of text (with meta-characters) that are searched for in a larger text file are called regular expressions:

    $ ll /proc | grep cpuinfo
  

In the preceding command, the grep utility will search for the cpuinfo text in all lines of input text and will print lines that have the cpuinfo text.

Utilities such as grep, sed, and awk use regular expressions for filtering text and then apply various processing commands as required by the user. The lines that do not match the pattern will be rejected. The following diagram explains the same concept:

In Chapter 3, Using Test Processing and Filters in Your Scripts, you learned about
the basics of regular expressions and pattern matching using the vi editor and
the grep utility.

sed – non-interactive stream editor

The stream editor (sed) is a very popular non-interactive stream editor. Normally, whenever we edit files using the vi editor, we need to open the file using the vi command, then we interact with the file to see the content of the file on screen, then edit it, and then save the file. Using sed, we can type commands on the command line and sed will make the changes to the text file. sed is a non-destructive editor. sed makes the changes to the file and displays the content on screen. If we want to save the changed file, then we need to redirect the output of sed to the file.

The procedure to install sed is shown here.

For Ubuntu or any Debian-based distributions, enter the following command:

    $ apt-get install sed
  

For Red Hat or any rpm-based distribution enter the following command:

    $ yum install sed
  

To check the version...

Using awk

awk is a program that has its own programming language for performing data-processing and generating reports.

The GNU version of awk is gawk.

awk processes data, which can be received from a standard input, input file, or as the output of any other command or process.

awk processes data similar to sed, line by line. It processes every line for the specified pattern and performs specified actions. If the pattern is specified, then all the lines containing specified patterns will be displayed. If pattern is not specified, then the specified actions will be performed on all the lines.

The meaning of awk

The name of the program awk is made from the initials of the three authors of the language, namely Alfred Aho, Peter...

Summary

In this chapter, you learned about regular expressions and about using sed and awk for text processing. You learned various commands and the usage of options along with a lot of examples for using sed and awk. In this example, the value of x is set in the body of the loop using the auto-increment operator. The body of the loop is executed once and the expression is evaluated.