In this article by Sinny Kumari, author of the book Linux Shell Scripting Essentials, we will cover I/O redirection pipes and filters.
In day-to-day work, we come across different kinds of files such as text files, source code files from different programming languages (for example, file.sh, file.c, and file.cpp), and so on. While working, we more often perform various operations on files or directories such as searching for a given string or pattern, replacing strings, printing few lines of a file, and so on. Performing these operations is not easy if we have to do it manually. Manual searching for a string or pattern in a directory having thousands of files can take months, and has high chances of making errors.
Shell provides many powerful commands to make our work easier, faster, and error-free. Shell commands have the ability to manipulate and filter text from different streams such as standard input, file, and so on. Some of these commands are grep, sed, head, tr, sort, and so on. Shell also comes with a feature of redirecting output from one command to another with the pipe ('|'). Using pipe helps to avoids creation of unnecessary temporary files.
One of the best qualities of these commands is that they come along with the man pages. We can directly go to the man page and see what all features they provide by running the man command. Most of the commands have options such as --help to find the help usage and --version to know the version number of the command.
This article will cover the following topics in detail:
Standard I/O and error streams
Redirecting the standard I/O and error streams
Pipe and pipelines—connecting commands
Regular expressions
Filtering output using grep
(For more resources related to this topic, see here.)
Standard I/O and error streams
In shell programming, there are different ways to provide an input (for example, via a keyboard and terminal) and display an output (for example, terminal and file) and error (for example, terminal), if any, during the execution of a command or program.
The following examples show the input, output, and error while running the commands:
The input from a user by a keyboard and the input obtained by a program via a standard input stream, that is terminal, is taken as follows:
$ read -p "Enter your name:"
Enter your name:Foo
The output printed on the standard output stream, that is terminal, is as follows:
$ echo "Linux Shell Scripting"
Linux Shell Scripting
The error message printed on the standard error stream, that is terminal, is as follows:
$ cat hello.txt
cat: hello.txt: No such file or directory
When a program executes, by default, three files get opened with it which are stdin, stdout, and stderr. The following table provides a short description about them:
File descriptor number
File name
Description
0
stdin
This is standard input being read from the terminal
1
stdout
This is standard output to the terminal
2
stderr
This is standard error to the terminal
File descriptors
File descriptors are integer numbers representing opened files in an operating system. The unique file descriptor numbers are provided to each opened files. File descriptors' numbers go up from 0.
Whenever a new process in Linux is created, then standard input, output, and error files are provided to it along with other needed opened files to process.
To know what all open file descriptors are associated with a process, we will consider the following example:
Run an application and get its process ID first. Consider running bash as an example to get PID of bash:
$ pidof bash
2508 2480 2464 2431 1281
We see that multiple bash processes are running. Take one of the bash PID example, 2508, and run the following command:
$ ls -l /proc/2508/fd
total 0
lrwx------. 1 sinny sinny 64 May 20 00:03 0 -> /dev/pts/5
lrwx------. 1 sinny sinny 64 May 20 00:03 1 -> /dev/pts/5
lrwx------. 1 sinny sinny 64 May 19 23:22 2 -> /dev/pts/5
lrwx------. 1 sinny sinny 64 May 20 00:03 255 -> /dev/pts/5
We see that 0, 1, and 2 opened file descriptors are associated with process bash. Currently, all of them are pointing to /dev/pts/5. pts that is pseudo terminal slave.
So, whatever we will do in this bash, input, output, and error related to this PID, output will be written to the /dev/pts/5 file. However, the pts files are pseudo files and contents are in memory, so you won't see anything when you open the file.
Redirecting the standard I/O and error streams
We have an option to redirect standard input, output, and errors, for example, to a file, another command, intended stream, and so on. Redirection is useful in different ways. For example, I have a bash script whose output and errors are displayed on a standard output—that is, terminal. We can avoid mixing an error and output by redirecting one of them or both to a file. Different operators are used for redirection. The following table shows some of operators used for redirection, along with its description:
Operator
Description
>
This redirects a standard output to a file
>>
This appends a standard output to a file
<
This redirects a standard input from a file
>&
This redirects a standard output and error to a file
>>&
This appends a standard output and error to a file
|
This redirects an output to another command
Redirecting standard output: An output of a program or command can be redirected to a file. Saving an output to a file can be useful when we have to look into the output in the future. A large number of output files for a program that runs with different inputs can be used in studying program output behavior.
For example, showing redirecting echo output to output.txt is as follows:
$ echo "I am redirecting output to a file" > output.txt
$
We can see that no output is displayed on the terminal. This is because output was redirected to output.txt. The operator '>' (greater than) tells the shell to redirect the output to whatever filename mentioned after the operator. In our case, its output.txt:
$ cat output.txt
I am redirecting output to a file
Now, let's add some more output to the output.txt file:
$ echo "I am adding another line to file" > output.txt
$ cat output.txt
I am adding another line to file
We noticed that the previous content of the output.txt file got erased and it only has the latest redirected content. To retain the previous content and append the latest redirected output to a file, use the operator '>>':
$ echo "Adding one more line" >> output.txt
$ cat output.txt
I am adding another line to file
Adding one more line
We can also redirect an output of a program/command to another command in bash using the operator '|' (pipe):
$ ls /usr/lib64/ | grep libc.so
libc.so
libc.so.6
In this example, we gave the output of ls to the grep command using the '|' (pipe) operator, and grep gave the matching search result of the libc.so library:
Redirecting standard input
Instead of getting an input from a standard input to a command, it can be redirected from a file using the < (less than) operator. For example, we want to count the number of words in the output.txt file created from the Redirecting standard output section:
$ cat output.txt
I am adding another line to file
Adding one more line
$ wc -w < output.txt
11
We can sort the content of output.txt:
$ sort < output.txt # Sorting output.txt on stdout
Adding one more line
I am adding another line to file
We can also give a patch file as an input to the patch command in order to apply a patch.diff in a source code. The command patch is used to apply additional changes made in a file. Additional changes are provided as a diff file. A diff file contains the changes between the original and the modified file by running the diff command. For example, I have a patch file to apply on output.txt:
$ cat patch.diff # Content of patch.diff file
2a3
> Testing patch command
$ patch output.txt < patch.diff # Applying patch.diff to output.txt
$ cat output.txt # Checking output.txt content after applying patch
I am adding another line to file
Adding one more line
Testing patch command
Redirecting standard error
There is a possibility of getting an error while executing a command/program in bash because of different reasons such as invalid input, insufficient arguments, file not found, bug in program, and so on:
$ cd /root # Doing cd to root directory from a normal user
bash: cd: /root/: Permission denied
Bash prints the error on a terminal saying, permission denied.
In general, errors are printed on a terminal so that it's easy for us to know the reason of an error. Printing both the errors and output on the terminal can be annoying because we have to manually look into each line and check whether the program encountered any error:
$ cd / ; ls; cat hello.txt; cd /bin/; ls *.{py,sh}
We ran a series of commands in the preceding section. First cd to /, ls content of /, cat file hello.txt, cd to /bin and see files matching *.py and *.sh in /bin/. The output will be as follows:
bin boot dev etc home lib lib64 lost+found media mnt opt proc root
run sbin srv sys tmp usr var
cat: hello.txt: No such file or directory
alsa-info.sh kmail_clamav.sh sb_bnfilter.py sb_mailsort.py setup-nsssysinit.sh amuFormat.sh kmail_fprot.sh sb_bnserver.pysb_mboxtrain.py struct2osd.sh core_server.py kmail_sav.shsb_chkopts.py sb_notesfilter.py
We see that hello.txt doesn't exist in the / directory and because of this there is an error printed on the terminal as well, along with other output. We can redirect the error as follows:
$ (cd / ; ls; cat hello.txt; cd /bin/; ls *.{py,sh}) 2> error.txt
bin boot dev etc home lib lib64 lost+found media mnt opt proc rootrun sbin srv sys tmp usr var
alsa-info.sh kmail_clamav.sh sb_bnfilter.py sb_mailsort.py setup-nsssysinit.sh amuFormat.sh kmail_fprot.sh sb_bnserver.pysb_mboxtrain.py struct2osd.sh core_server.py kmail_sav.shsb_chkopts.py sb_notesfilter.py
We can see that the error has been redirected to the error.txt file. To verify, check the error.txt content:
$ cat error.txt
cat: hello.txt: No such file or directory
Multiple redirection: We can redirect stdin, stdout, and stderr together in a command or script or a combination of some of them.
The following command redirects both stdout and stder:
$ (ls /home/ ;cat hello.txt;) > log.txt 2>&1
Here, stdout is redirected to log.txt and error messages are redirected to log.txt as well. In 2>&1, 2> means redirect an error and &1 means redirect to stdout. In our case, we have already redirected stdout to the log.txt file. So, now both the stdout and stderr outputs will be written into log.txt and nothing will be printed on the terminal. To verify, we will check the content of log.txt:
$ cat log.txt
lost+found
sinny
cat: hello.txt: No such file or directory
The following example shows the stdin, stdout, and stderr redirection:
$ cat < ~/.bashrc > out.txt 2> err.txt
Here, the .bashrc file present in the home directory acts as an input to the cat command and its output is redirected to the out.txt file. Any error encountered in between is redirected to the err.txt file.
The following bash script will explain stdin, stdout, stderr, and their redirection with even more clarity:
#!/bin/bash
# Filename: redirection.sh
# Description: Illustrating standard input, output, error
# and redirecting them
ps -A -o pid -o command > p_snapshot1.txt
echo -n "Running process count at snapshot1: "
wc -l < p_snapshot1.txt
echo -n "Create a new process with pid = "
tail -f /dev/null & echo $! # Creating a new process
echo -n "Running process count at snapshot2: "
ps -A -o pid -o command > p_snapshot2.txt
wc -l < p_snapshot2.txt
echo
echo "Diff bewteen two snapshot:"
diff p_snapshot1.txt p_snapshot2.txt
This script saves two snapshots of running all the currently running processes in the system and generates diff. The output after running the process will look somewhat as follows:
$ sh redirection.sh
Running process count at snapshot1: 246
Create a new process with pid = 23874
Running process count at snapshot2: 247
Diff bewteen two snapshot:
246c246,247
< 23872 ps -A -o pid -o command
---
> 23874 tail -f /dev/null
> 23875 ps -A -o pid -o command
Pipe and pipelines – connecting commands
The outputs of the programs are generally saved in files for further use. Sometimes, temporary files are created in order to use an output of a program as an input to another program. We can avoid creating temporary files and feed the output of a program as an input to another program using bash pipe and pipelines.
Pipe
The pipe denoted by the operator | connects the standard output of a process in the left to the standard input in the right process by inter process communication mechanism. In other words, the | (pipe) connects commands by providing the output of a command as the input to another command.
Consider the following example:
$ cat /proc/cpuinfo | less
Here, the cat command, instead of displaying the content of the /proc/cpuinfo file on stdout, passes its output as an input to the less command. The less command takes the input from cat and displays on the stdout per page.
Another example using pipe is as follows:
$ ps -aux | wc -l # Showing number of currently running processes in system
254
Pipeline
Pipeline is a sequence of programs/commands separated by the operator ' | ' where the output of execution of each command is given as an input to the next command. Each command in a pipeline is executed in a new subshell. The syntax will be as follows:
command1 | command2 | command3 …
Examples showing pipeline are as follows:
$ ls /usr/lib64/*.so | grep libc | wc -l
13
Here, we are first getting a list of files from the /usr/lib64 directory that has the.so extension. The output obtained is passed as an input to the next grep command to look for the libc string. The output is further given to the wc command to count the number of lines.
Regular expression
Regular expression (also known as regex or regexp) provides a way of specifying a pattern to be matched in a given big chunk of text data. It supports a set of characters to specify the pattern. It is widely used for a text search and string manipulation. A lot of shell commands provide an option to specify regex such as grep, sed, find, and so on.
The regular expression concept is also used in other programming languages such as C++, Python, Java, Perl, and so on. Libraries are available in different languages to support regular expression's features.
Regular expression metacharacters
The metacharacters used in regular expressions are explained in the following table:
Metacharacters
Description
* (Asterisk)
This matches zero or more occurrences of the previous character
+ (Plus)
This matches one or more occurrences of the previous character
?
This matches zero or one occurrence of the previous element
. (Dot)
This matches any one character
^
This matches the start of the line
$
This matches the end of line
[... ]
This matches any one character within a square bracket
[^... ]
This matches any one character that is not within a square bracket
| (Bar)
This matches either the left side or the right side element of |
{X}
This matches exactly X occurrences of the previous element
{X,}
This matches X or more occurrences of the previous element
{X,Y}
This matches X to Y occurrences of the previous element
(...)
This groups all the elements
<
This matches the empty string at the beginning of a word
>
This matches the empty string at the end of a word
This disables the special meaning of the next character
Character ranges and classes
When we look into a human readable file or data, its major content contains alphabets (a to z) and numbers (0-9). While writing regex for matching a pattern consisting of alphabets or numbers, we can make use character ranges or classes.
Character ranges
We can use character ranges in a regular expression as well. We can specify a range by a pair of characters separated by a hyphen. Any characters that fall in between that range, inclusive, are matched. Character ranges are enclosed inside square brackets.
The following table shows some of character ranges:
Character range
Description
[a-z]
This matches any single lowercase letter from a to z
[A-Z]
This matches any single uppercase letter from A to Z
[0-9]
This matches any single digit from 0 to 9
[a-zA-Z0-9]
This matches any single alphabetic or numeric characters
[h-k]
This matches any single letter from h to k
[2-46-8j-lB-M]
This matches any single digit from 2 to 4 or 6 to 8 or any letter from j to l or B to M
Character classes: Another way of specifying a range of character matches is by using Character classes. It is specified within the square brackets [:class:]. The possible class value is mentioned in the following table:
Character Class
Description
[:alnum:]
This matches any single alphabetic or numeric character; for example, [a-zA-Z0-9]
[:alpha:]
This matches any single alphabetic character; for example, [a-zA-Z]
[:digit:]
This matches any single digit; for example, [0-9]
[:lower:]
This matches any single lowercase alphabet; for example, [a-z]
[:upper:]
This matches any single uppercase alphabet; for example, [A-Z]
[:blank:]
This matches a space or tab
[:graph:]
This matches a character in the range of ASCII—for example 33-126—excluding a space character
[:print:]
This matches a character in the range of ASCII—for example. 32-126—including a space character
[:punct:]
This matches any punctuation marks such as '?', '!', '.', ',', and so on
[:xdigit:]
This matches any hexadecimal characters; for example, [a-fA-F0-9]
[:cntrl:]
This matches any control characters
Creating your own regex: In the previous sections of regular expression, we discussed about metacharacters, character ranges, character class, and their usage. Using these concepts, we can create powerful regex that can be used to filter out text data as per our need. Now, we will create a few regex using the concepts we have learned.
Matching dates in mm-dd-yyyy format
We will consider our valid date starting from UNIX Epoch—that is, 1st January 1970. In this example, we will consider all the dates between UNIX Epoch and 30th December 2099 as valid dates. An explanation of forming its regex is given in the following subsections:
Matching a valid month
0[1-9] matches 01st to 09th month
1[0-2] matches 10th, 11th, and 12th month
'|' matches either left or right expression
Putting it all together, the regex for matching a valid month of date will be 0[1-9]|1[0-2].
Matching a valid day
0[1-9] matches 01st to 09th day
[12][0-9] matches 10th to 29th day
3[0-1] matches 30th to 31st day
'|' matches either left or right expression
0[1-9]|[12][0-9]|3[0-1] matches all the valid days in a date
Matching the valid year in a date
19[7-9][[0-9] matches years from 1970 to 1999
20[0-9]{2} matches years from 2000 to 2099
'|' matches either left or right expression
19[7-9][0-9]|20[0-9]{2} matches all the valid years between 1970 to 2099
Combining valid months, days, and years regex to form valid dates
Our date will be in mm-dd-yyyy format. By putting together regex formed in the preceding sections for months, days, and years, we will get regex for the valid date:
(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[0-1])-(19[7-9][0-9]|20[0-9]{2})
There is a nice website http://regexr.com/, where you can also validate regular expression. The following screenshot shows the matching of the valid date among the given input:
Regex for a valid shell variable
A valid variable name can contain a character from alphanumeric and underscore, and the first letter of the variable can't be a digit.
Keeping these rules in mind, a valid shell variable regex can be written as follows:
^[_a-zA-Z][_a-zA-Z0-9]*$
Here, ^ (caret) matches the start of a line.
The regex [_a-zA-Z] matches _ or any upper or lower case alphabet [_a-zA-Z0-9]* matches zero or multiple occurrences of _,any digit or upper and lower case alphabet $ (Dollar) matches the end of the line.
In character class format, we can write regex as ^[_[:alpha:]][_[:alnum:]]*$.
The following screenshot shows valid shell variables using regex formed:
Enclose regular expression in single quotes (') to avoid pre-shell expansion.
Use back slash () before a character to escape the special meaning of metacharacters.
Metacharacters such as ?, +, {, |, (, and ) are known to be extended regex. They lose their special meaning when used in basic regex. To avoid this, use them with backslash '?', '+', '{', '|', '(', and ')'.
Filtering an output using grep
One of the powerful and widely used command in shell is grep. It searches in an input file and matches lines in which the given pattern is found. By default, all the matched patterns are printed on stdout that is usually terminal. We can also redirect the matched output to other streams such as file. Instead of giving an input from a file, grep can also take the input from the redirected output of the command executed on the left-hand side of '|'.
Syntax
The syntax of using the grep command is as follows:
grep [OPTIONS] PATTERN [FILE...]
Here, FILE can be multiple files for a search. If no file is given as an input for a search, it will search the standard input.
PATTERN can be any valid regular expression. Put PATTERN within single quotes (') or double quotes (") as per need. For example, use single quotes (') to avoid any bash expansion and double quotes (") for expansion.
A lot of OPTIONS are available in grep. Some of the important and widely used options are discussed in the following table.
Option
Usage
-i
This enforces case insensitive match in both pattern and input file(s)
-v
This displays the non-matching line
-o
This displays only the matched part in the matching line
-f FILE
This obtains a pattern from a file, one per line
-e PATTERN
This specifies multiple search pattern
-E
This considers pattern as an extended regex (egrp)
-r
This reads all the files in a directory recursively, excluding resolving of symbolic links unless explicitly specified as an input file
-R
This reads all the files in a directory recursively and resolving symbolic if any
-a
This processes binary file as a text file
-n
This prefixes each matched line along with a line number
-q
Don't print anything on stdout
-s
Don't print error messages
-c
This prints the count of matching lines of each input file
-A NUM
This prints NUM lines after the actual string match. No effect with the -o option
-B NUM
This prints NUM lines before the actual string match. No effect with the -o option
-C NUM
This prints NUM lines after and before the actual string match. No effect with the -o option
Looking for a pattern in a file: A lot of times we have to search for a given string or a pattern in a file. The grep command provides us the capability to do it in a single line. Let's see the following example:
The input file for our example will be input1.txt:
$ cat input1.txt # Input file for our example
This file is a text file to show demonstration
of grep command. grep is a very important and
powerful command in shell.
This file has been used in chapter 2
We will try to get the following information from the input1.txt file using the grep command:
Number of lines
Line starting with a capital letter
Line ending with a period (.)
Number of sentences
Searching sub-string "sent" Lines that don't have a periodNumber of times the string "file" is used
The following shell script demonstrates how to do the above mentioned tasks:
#!/bin/bash
#Filename: pattern_search.sh
#Description: Searching for a pattern using input1.txt file
echo "Number of lines = `grep -c '.*' input1.txt`"
echo "Line starting with capital letter:"
grep -c ^[A-Z].* input1.txt
echo
echo "Line ending with full stop (.):"
grep '.*.$' input1.txt
echo
echo -n "Number of sentence = "
grep -c '.' input1.txt
echo "Strings matching sub-string sent:"
grep -o "sent" input1.txt
echo
echo "Lines not having full stop are:"
grep -v '.' input1.txt
echo
echo -n "Number of times string file used: = "
grep -o "file" input1.txt | wc -w
The output after running the pattern_search.sh shell script will be as follows:
Number of lines = 4
Line starting with capital letter:
2
Line ending with full stop (.):
powerful command in shell.
Number of sentence = 2
Strings matching sub-string sent:
Lines not having full stop are:
This file is a text file to show demonstration
This file has been used in chapter 2
Number of times string file used: = 3
Looking for a pattern in multiple files
The grep command also allow us to search for a pattern in multiple files as an input. To explain this in detail, we will head directly to the following example.
The input files, in our case, will be input1.txt and input2.txt.
We will reuse the content of the input1.txt file from the previous example.
The content of input2.txt is as follows:
$ cat input2.txt
Another file for demonstrating grep CommaNd usage.
It allows us to do CASE Insensitive string test
as well.
We can also do recursive SEARCH in a directory
using -R and -r Options.
grep allows to give a regular expression to
search for a PATTERN.
Some special characters like . * ( ) { } $ ^ ?
are used to form regexp.
Range of digit can be given to regexp e.g. [3-6],
[7-9], [0-9]
We will try to get the following information from the input1.txt and input2.txt files using the grep command:
Search for the string command
Case-insensitive search of the string command
Print the line number where the string grep matches
Search for punctuation marks
Print one line followed by the matching lines while searching for the string important
The following shell script demonstrates how to follow the preceding steps:
#!/bin/bash
# Filename: multiple_file_search.sh
# Description: Demonstrating search in multiple input files
echo "This program searches in files input1.txt and input2.txt"
echo "Search result for string "command":"
grep "command" input1.txt input2.txt
echo
echo "Case insensitive search of string "command":"
# input{1,2}.txt will be expanded by bash to input1.txt input2.txt
grep -i "command" input{1,2}.txt
echo
echo "Search for string "grep" and print matching line too:"
grep -n "grep" input{1,2}.txt
echo
echo "Punctuation marks in files:"
grep -n [[:punct:]] input{1,2}.txt
echo
echo "Next line content whose previous line has string "important":"
grep -A 1 'important' input1.txt input2.txt
The following screenshot is the output after running the shell script pattern_search.sh. The matched pattern string has been highlighted:
A few more grep usages
The following subsections will cover a few more usages of the grep command.
Searching in a binary file
So far, we have seen all the grep examples running on text files. We can also search for a pattern in binary files using grep. For this, we have to tell the grep command to treat a binary file as a text file too. The option -a or –text tells grep to consider a binary file as a test file.
We know that the grep command itself is a binary file that executes and gives a search result.
One of options in grep is --text. The string --text should be somewhere available in the grep binary file. Let's search for it as follows:
$ grep --text '--text' /usr/bin/grep
-a, --text equivalent to –binary-files=text
We saw that the string --text is found in the search path /usr/bin/grep. The character backslash ('') is used to escape its special meaning.
Now, let's search for the -w string in the wc binary. We know that the wc command has an option -w that counts the number of words in an input text.
$ grep -a '-w' /usr/bin/wc
-w, --words print the word counts
Searching in a directory
We can also tell grep to search into all files/directories in a directory recursively using the option -R. This avoids the hassle of specifying each file as an input text file to grep.
For example, we are interested in knowing at how many places #include <stdio.h> is used in a standard include directory:
$ grep -R '#include <stdio.h>' /usr/include/ | wc -l
77
This means that the #include <stdio.h> string is found at 77 places in the /usr/include directory.
In another example, we want to know how many python files (the extension .py) in /usr/lib64/python2.7/ does "import os". We can check that as follows:
$ grep -R "import os" /usr/lib64/python2.7/*.py | wc -l
93
Excluding files/directories from search
We can also specify the grep command to exclude a particular directory or file from search. This is useful when we don't want grep to look into a file or directory that has some confidential information. This is also useful in the case where we are sure that searching into a certain directory will be of no use. So, excluding them will reduce search time.
Suppose, there is a source code directory called s0, which uses the git version control. Now, we are interested in searching for a text or pattern in source files. In this case, searching in the .git subdirectory will be of no use. We can exclude .git from search as follows:
$ grep -R --exclude-dir=.git "search_string" s0
Here, we are searching for the search_string string in the s0 directory and telling grep to not to search in the .git directory.
Instead of excluding a directory, to exclude a file, use the --exclude-from=FILE option.
Display a filename with a matching pattern
In some use-cases, we don't bother with where the search matched and at how many places the search matched in a file. Instead, we are interested in knowing only the filename where at least one search matched.
For example, I want to save filenames that have a particular search pattern found in a file, or redirect to some other command for further processing. We can achieve this using the -l option:
$ grep -Rl "import os" /usr/lib64/python2.7/*.py > search_result.txt
$ wc -l search_result.txt
79
This example gets name of the file in which import os is written and saves result in file search_result.txt.
Matching an exact word
The exact matching of the word is also possible using word boundary that is b on both the sides of the search pattern.
Here, we will reuse the input1.txt file and its content:
$ grep -i --color "bab" input1.txt
The --color option allows colored printing of the matched search result.
The "bab" option tells grep to only look for the character a that is alone. In search results, it won't match the character a present as a sub-string in a string.
The following screenshot shows the output:
This is an input file.
It conatins special character like ?, ! etc
&^var is an invalid shll variable.
_var1_ is a valid shell variable
To delete characters other than alphanumeric, newline, and white-space, we can run the following command:
tr -cd '[:alnum:] n' < tr2.txt
This is an input file
It conatins special character like etc
var is an invalid shll variable
var1 is a valid shell variable
Summary
After reading this article, you know how to provide an input to commands and print or save its result. You are also familiar with redirecting an output and input from one command to another. Now, you can easily search, replace strings or pattern in a file, and filter out data based on needs.
From this article, we now have a good control on transforming/filtering text data.
Resources for Article:
Further resources on this subject:
Linux Shell Scripting [article]
Embedded Linux and Its Elements [article]
Getting started with Cocos2d-x [article]
Read more
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
Singapore
Canary Islands
Hungary
Ukraine
Luxembourg
Estonia
Lithuania
South Korea
Turkey
Switzerland
Colombia
Taiwan
Chile
Norway
Ecuador
Indonesia
New Zealand
Cyprus
Denmark
Finland
Poland
Malta
Czechia
Austria
Sweden
Italy
Egypt
Belgium
Portugal
Slovenia
Ireland
Romania
Greece
Argentina
Netherlands
Bulgaria
Latvia
South Africa
Malaysia
Japan
Slovakia
Philippines
Mexico
Thailand
