Unix/Linux Shell Scripting : How to search using AWK

AWK

Every unix/linux guy knows awk as a powerful tool, while writing a shell script, so today we will know 

• How AWK can be used to search string from a file and return desired value as output.
• How AWK can be used to search for a specific pattern
• How can we use AWK to print based upon line number. 

We will start with our file, here is our file named demo.txt and it's content

cat demo.txt

FName|LName|Age|City|Country
Jill|Smith|50|NewYork|USA
Emi|Jackson|94|London|Britain
Raj|Dhinga|35|Delhi|India
Yan|Yun|21|Bejing|China
Yusuf|Khan|44|Mulheim|Germany
Will|Smith|40|NewYork|USA
Shail|Raj|21|Delhi|India
Vikram|Rajnand|34|Pune|India
HansRaj|Kedia|18|Mumbai|India
Devraj|Shukla|55|Dhanbad|India

AWK Basic Syntax :

  # awk options 'selection criteria { actions }' inputFile

How to use AWK with delimiter.

So we will start by printing the first field that is Fname and we will know, how we will separate fields when we have common delimiter.

Our file demo.txt has | as delimiter, so use the below one liner to separate fields.

  # awk -F '|' '{print $1}' demo.txt

-F followed by delimiter is used to process our requirement.

output

FName
Jill
Emi
Raj
Yan
Yusuf
Will
Shail
Vikram
HansRaj
Devraj

How to Use AWK to search and match for a string and print the entire field.

In our example file ( demo.txt), we will try to find a string "Raj" that can be part of the entire file, in case it founds the string it will print the entire line that contain the field with value "Raj"

  # awk -F "|" '/Raj/' demo.txt

-F followed by delimiter(|) and search string ( Raj ) placed within inverted comma and two backward slashes is used to process our requirement.

output

Raj|Dhinga|35|Delhi|India
Shail|Raj|21|Delhi|India
Vikram|Rajnand|34|Pune|India
HansRaj|Kedia|18|Mumbai|India

Output Analysis:

Field doesn't matter :

it's matching the word "Raj" irrespective of field,  thus printing the value of the entire line wherever it matches the search criteria.

Sub-Strings are also True matches: It doesn't matters if search string is a part of any sub-string.

Our input file has the below line that proves the statement.

Vikram|Rajnand|34|Pune|India
HansRaj|Kedia|18|Mumbai|India

Strict Matching : It strictly differentiate between upper case and lower case, our demo.txt file contains "raj" also as a part of substring but it ignores it as we have passed "Raj"

Devraj|Shukla|55|Dhanbad|India

How to Use AWK to search and match for a string and print only selected fields as an output.

So from our input file, we will match the same string "Raj" and try to print it's selected fields that is Fname($1), Age($3) and City($4)

  # awk -F "|" '/Raj/ {print $1, $3, $4}' demo.txt

-F followed by search string and  print statement with comma for space to create space between fields.

 

output

Raj 35 Delhi
Shail 21 Delhi
Vikram 34 Pune
HansRaj 18 Mumbai

How to Use AWK to search and match for a string and print only selected fields as an output with Line numbers.

So for all above search string matched lines, along with the output in order to print matched line numbers we can use NR combined with print statement, here is the one liner.

  # awk -F "|" '/Raj/ {print NR, $1, $3, $4}' demo.txt

-F followed by search string and  print statement starting with NR

output

4 Raj 35 Delhi
8 Shail 21 Delhi
9 Vikram 34 Pune
10 HansRaj 18 Mumbai

AWK Search pattern to ignore difference between upper and lower case.

From our input file, we want to add lines that contain "Raj" and "raj" both

  # awk -F "|" '/[Rr]aj/' demo.txt

-F followed by pattern [Rr] to add both the values.

output

Raj|Dhinga|35|Delhi|India
Shail|Raj|21|Delhi|India
Vikram|Rajnand|34|Pune|India
HansRaj|Kedia|18|Mumbai|India
Devraj|Shukla|55|Dhanbad|India

AWK to print by line number only.

From our input file,if we just want to print only 3rd line only, below is the one liner and it's output.

 # awk -F "|" 'NR==3'  demo.txt

-F followed by delimiter and "NR==3" where 3 is the line number,

output

Emi|Jackson|94|London|Britain

AWK to print between two line number.

From our input file,if we just want to print between line number 2 and 5, below is the one liner and it's output.

 # awk -F "|" ' NR==2, NR==5 {print NR,  $1, $3, $4}' demo.txt

-F followed by delimiter and "NR==2" a comma(,) and "NR==5"

output

2 Jill 50 NewYork
3 Emi 94 London
4 Raj 35 Delhi
5 Yan 21 Bejing

AWK to print between two line number and it's output value to a file.

From our input file,if we just want to print between line number 2 and 5, below is the one liner and it's output.

 # awk -F "|" ' NR==2, NR==5 {print NR,  $1, $3, $4}' demo.txt >> /tmp/newfile.txt