|^^|
source: http://www.devopsservice.com/awk-command-examples-in-linux/
awk is a pattern scanning and processing language. An awk script is composed of one or more condition-action pairs. The condition is applied to each line in the file or files passed on the command line or to the standard input if no files are given. When the condition resolves successfully, the corresponding action is performed.
Field processor based on whitespace, by default.
Usage: awk ‘/pattern/ { procedure }’ file
Below is the employees_data.txt file which we will use for examples.
1
2
3
4
5
6
7
8
9
10
1 John Administrator $3000
2 Jonathan Developer $4000
3 William Manager $7000
4 Micheal Support $4000
5 James Sales $4500
6 Noah QA $3000
7 Daniel UI $6500
– Prints every line from the file.
1
2
3
4
5
6
7
8
9
10
11
12
devops@devopsservice:~$ awk '{print;}' employees_data.txt
1 John Administrator $3000
2 Jonathan Developer $4000
3 William Manager $7000
4 Micheal Support $4000
5 James Sales $4500
6 Noah QA $3000
7 Daniel UI $6500
devops@devopsservice:~$
You can use below command also
1
2
3
4
5
6
7
8
9
10
11
12
devops@devopsservice:~$ awk '{ print $0 }' employees_data.txt
1 John Administrator $3000
2 Jonathan Developer $4000
3 William Manager $7000
4 Micheal Support $4000
5 James Sales $4500
6 Noah QA $3000
7 Daniel UI $6500
devops@devopsservice:~$
– prints column 2 from each line
1
2
3
4
5
6
7
8
9
10
11
12
devops@devopsservice:~$ awk '{ print $2 }' employees_data.txt
John
Jonathan
William
Micheal
James
Noah
Daniel
devops@devopsservice:~$
– prints column 2 and 3
1
2
3
4
5
6
7
8
9
devops@devopsservice:~$ awk '{ print $2,$3 }' employees_data.txt
John Administrator
Jonathan Developer
William Manager
Micheal Support
James Sales
Noah QA
– prints ALL columns where line includes ‘Jo’
1
2
3
4
5
6
devops@devopsservice:~$ awk '/Jo/ { print $0}' employees_data.txt
1 John Administrator $3000
2 Jonathan Developer $4000
devops@devopsservice:~$
– Find the employees who has salary greater than $3500
1
2
3
4
5
6
7
8
9
devops@devopsservice:~$ awk '$4 > "$3500"' employees_data.txt
2 Jonathan Developer $4000
3 William Manager $7000
4 Micheal Support $4000
5 James Sales $4500
7 Daniel UI $6500
devops@devopsservice:~$
– prints ALL columns of records containing ‘3000’ in the fourth column
1
2
3
4
5
6
devops@devopsservice:~$ awk '{ if ($4 ~ /3000/) print $0 }' employees_data.txt
1 John Administrator $3000
6 Noah QA $3000
devops@devopsservice:~$
– prints lines ending with 3000
1
2
3
4
5
6
devops@devopsservice:~$ awk '/3000$/ { print $0 }' employees_data.txt
1 John Administrator $3000
6 Noah QA $3000
devops@devopsservice:~$
– Print blank lines
1
2
3
4
5
devops@devopsservice:~$ awk '/^$/ {print }' employees_data.txt
devops@devopsservice:~$
– Default delimiter is whitespace. You can use -F option to use different delimiter.
Print column 1 and 6 from /etc/passwd file
1
2
3
4
5
6
7
8
9
10
11
12
devops@devopsservice:~$ awk -F: '{print $1, $6}' /etc/passwd
root /root
daemon /usr/sbin
bin /bin
sys /dev
sync /bin
games /usr/games
man /var/cache/man
lp /var/spool/lpd
– Remove duplicate entries in a file
Below is the content of test.txt file
1
2
3
4
5
6
7
John
William
John
Jonathan
John
Use below command to remove duplicate entry
1
2
3
4
5
6
7
devops@devopsservice:~$ awk '!x[$0]++' test.txt
John
William
Jonathan
devops@devopsservice:~$
– Identify line having particular size in file
1
2
3
4
5
6
devops@devopsservice:~$ awk 'length>80' /etc/passwd
speech-dispatcher:x:112:29:Speech Dispatcher,,,:/var/run/speech-dispatcher:/bin/sh
logstash-forwarder:x:997:997:logstash-forwarder Service User:/var/lib/logstash-forwarder:/usr/sbin/nologin
devops@devopsservice:~$
Above command displays a list of lines that are longer than 80 characters.
– Initialization and Final Action on Script
Syntax:
BEGIN { Actions}
{ACTION} # on File
END { Actions }
Awk script consists of 3 parts.
– Before (denoted using: BEGIN) – Execute prior to first line of input being read.
– During (Main Awk loop) – Focuses on looping through lines of input.
– After (denoted using: END) – Executed after the last line of input has been processed.
Both BEGIN and END components of Awk scripts are optional.
Example:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
devops@devopsservice:~$ awk 'BEGIN {print "Emp_ID\tName\tDesignation\tSalary";}
> {print $1,"\t",$2,"\t",$3,"\t",$4;}
> END{print ".................\n";
> }' employees_data.txt
Emp_ID Name Designation Salary
1 John Administrator $3000
2 Jonathan Developer $4000
3 William Manager $7000
4 Micheal Support $4000
5 James Sales $4500
6 Noah QA $3000
7 Daniel UI $6500
.................
devops@devopsservice:~$
EOF