Awkward Ranger

What is Awk?

AWK is a pattern–action text processing language great for logs, CSVs, reports, and automation.

Print

1awk 'BEGIN { print "Hello, AWK!" }'
2
3Hello, AWK!

Lines

1awk '{ print }' file.txt
2
3Hello World!
4hello world!
5<pusa>
61 2 3 takbo...
7pelepens; flood control;
8Dunk it Jonathan!

Match and print

1awk '/Dunk/ { print }' file.txt
2
3Dunk it Jonathan!

Line numbers

NR - current line numbers
$0 - whole line

1awk '{ print NR, $0 }' file.txt
2
31 Hello World!
42 hello world!
53 <pusa>
64 1 2 3 takbo...
75 pelepens; flood control;
86 Dunk it Jonathan!

Fields and Columns

file.txt

1Jonathan 28 Manila
2Berto 30 Luna
3Betsy 23 Rizal

Print column

1awk '{ print $1 }' file.txt
2
3Jonathan
4Berto
5Betsy

1awk '{ print $1, $3 }' file.txt
2
3Jonathan Manila
4Berto Luna
5Betsy Rizal

Field separator

1echo "apple,banana,orange" | awk 'BEGIN {FS=","} {print $2}'
2
3banana

Or using -F followed by the separator, in this example ,.

1echo "apple,banana,orange" | awk -F, '{print $1}'
2
3apple

Changing the output separator

1echo "John Doe 101 Sales" | awk -v OFS=":" '{print $1, $2, $3, $4}'
2
3John:Doe:101:Sales

Conditions - patterns

Print lines age > 25.

1awk '$2 > 25 { print $1, $2 }' file.txt
2
3Jonathan 28
4Berto 30

String match.

1awk '$1 == "Betsy" { print }' file.txt
2
3Betsy 23 Rizal

Regex match.

1awk '/Manila/' file.txt
2
3Jonathan 28 Manila

Calculations and Fields

Basic Calculation

1awk '{ sum += $2 } END { print sum }' file.txt
2
381

1awk '{ sum += $2; count++ } END { print sum/count }' file.txt
2
327

Built-n Variables

Variable	Meaning
`NR`	Record number
`NF`	Number of fields
`$NF`	Last field
`FS`	Input separator
`OFS`	Output separator

NR - Record Number

Count the current line being processed, $0 is the whole line

1awk '{ print NR, $0 }' file.txt
2
31 Jonathan 28 Manila
42 Berto 30 Luna
53 Betsy 23 Rizal

1awk '{ print NR, $3 }' file.txt
2
31 Manila
42 Luna
53 Rizal

NF – Number of Fields

Print number of columns in each row.

1awk '{ print $1, "has", NF, "fields" }' file.txt
2
3Jonathan has 3 fields
4Berto has 3 fields
5Betsy has 3 fields

$NF – Last Field

Print the last column.

1awk '{ print $1, "lives in", $NF }' file.txt
2
3Jonathan lives in Manila
4Berto lives in Luna
5Betsy lives in Rizal

FS – Input Field Separator

1awk 'BEGIN { FS = " " } { print $1, "is", $2, "years old" }' file.txt
2
3Jonathan is 28 years old
4Berto is 30 years old
5Betsy is 23 years old

To have better demonstration, lets change the data. file.txt

1Jonathan,28,Manila
2Berto,30,Luna
3Betsy,23,Rizal

1awk 'BEGIN { FS = "," } { print $1, "is", $2, "years old" }' file.txt
2
3Jonathan is 28 years old
4172 Berto is 30 years old
5173 Betsy is 23 years old

OFS – Output Field Separator

Use to change the output separator.

1Jonathan-28-Manila
2Berto-30-Luna
3Betsy-23-Rizal

Changing Fields

1awk '{ $2 = $2 + 5; print }' file.txt
2
3Jonathan 33 Manila
4Berto 35 Luna
5Betsy 28 Rizal

Filtering Rows

Print lines with more that 2 fields.

1awk 'NF > 2' file.txt
2
3Jonathan 28 Manila
4Berto 30 Luna
5Betsy 23 Rizal

Multiple Conditions

1awk '$2 > 20 && $3 == "Manila"' file.txt
2
3Jonathan 28 Manila

Strings and Texts

String Functions

Function	Description
`length($0)`	Returns the length of a string (number of characters). `$0` refers to the entire line.
`tolower(string)`	Converts all characters in a string to lowercase.
`toupper(string)`	Converts all characters in a string to uppercase.
`substr(string, start, length)`	Extracts a substring from `string` starting at `start` for `length` characters.
`index(string, search)`	Returns the position of `search` within `string` (1-based). Returns 0 if not found.
`split(string, array, sep)`	Splits `string` into an array using the separator `sep`. Returns the number of elements.

length($0)

1awk '{ print $1, length($0) }' file.txt
2
3Jonathan 18
4Berto 13
5Betsy 14

tolower(string)

1awk '{ print tolower($1) }' file.txt
2
3jonathan
4berto
5betsy

toupper(string)

1awk '{ print toupper($1) }' file.txt
2
3JONATHAN
4BERTO
5BETSY

substr(string, start, length)

1awk '{ print substr($1,1,4) }' file.txt
2
3Jona
4Bert
5Bets

index(string, search)

1awk '{ print index($3,"Man") }' file.txt
2
31
40
50

split(string, array, sep)

1awk '{ n = split($0, a, " "); print a[1], a[3] }' file.txt
2
3Jonathan Manila
4Berto Luna
5Betsy Rizal

Find and Replace

1awk '{ gsub("Manila", "MNL"); print }' file.txt
2
3Jonathan 28 MNL
4Berto 30 Luna
5Betsy 23 Rizal

Replace only first match:

1awk '{ sub("Manila", "MNL"); print }' file.txt
2
3Jonathan 28 MNL
4Berto 30 Luna
5Betsy 23 Rizal

Control Flow

if/else

 1awk '{
 2  if ($2 >= 30)
 3    print $1, "Senior"
 4  else
 5    print $1, "Junior"
 6}' file.txt
 7
 8Jonathan Junior
 9Berto Senior
10Betsy Junior

loops

 1awk '{
 2  for (i=1; i<=NF; i++)
 3    print $i
 4}' file.txt
 5
 6Jonathan
 728
 8Manila
 9Berto
1030
11Luna
12Betsy
1323
14Rizal

Arrays

Counting Occurrences

1awk '{ count[$1]++ } END {
2  for (name in count)
3    print name, count[name]
4}' file.txt
5
6Jonathan 1
7Berto 1
8Betsy 1

Summing by Category