====== awk ======

A tiny language for column/record processing. Pattern: ''pattern { action }''
per line. Ubuntu ships **mawk** (fast, smaller feature set) : 
''sudo apt install gawk'' gets you the full GNU dialect (''gensub'',
''strftime'', ''mktime'', ''asort'', ''BEGINFILE'').

===== Model =====

<code bash>
awk '{print $1}'              # $0=whole line, $1..$NF=fields, NF=count
awk 'NR==5'                   # NR=line number (bare pattern = print)
awk -F: '{print $1}' /etc/passwd   # field separator
awk -F: -v OFS='|' '{print $1,$3}' # output separator
</code>

Built-ins: ''NR'' record#, ''NF'' field count, ''FS''/''OFS''/''RS''/''ORS'',
''FILENAME'', ''$0'', ''$1..$NF'', ''$(NF-1)'' (second-to-last).

===== Patterns =====

<code bash>
awk '$3 > 100'                       # numeric compare
awk '$1 ~ /foo/'                     # field matches regex (== !~ for not)
awk '/start/,/end/'                  # range pattern
awk 'NR>=10 && NR<=20'
awk 'length($0) > 80'                # long lines
awk 'END{print}'                     # last line
</code>

===== BEGIN / END: headers, totals =====

<code bash>
awk 'BEGIN{FS=":"} {s+=$3} END{print "total:",s}' /etc/passwd
ps aux | awk '{m+=$6} END{printf "%.1f MB\n", m/1024}'   # total RSS
</code>

===== Associative arrays: the main feature =====

<code bash>
awk '{count[$1]++} END{for(k in count) print k, count[k]}'   # histogram
awk '{seen[$0]++} !seen[$0]-1' ...                            # just: awk '!seen[$0]++' dedupes
awk '{sum[$1]+=$2} END{for(k in sum) print k":"sum[k]}'       # group+sum
</code>

===== String functions =====

<code bash>
length(x) substr(s,i,n) index(s,t) tolower(s) toupper(s)
split(s,a,sep)         # array from string
gsub(re,rep,str)       # global replace IN the string (returns count)
sub(re,rep,str)        # first only
sprintf(fmt,...)       # returns formatted (printf prints)
match(s,re)            # sets RSTART/RLENGTH
</code>

<code bash>
awk '{gsub(/"/,""); print}' csvish.txt
awk '{split($1,a,"."); print a[1]}'       # first dotted part
awk '{printf "%-20s %6.2f\n", $1, $2}'    # aligned columns
</code>

===== Multiline / CSV-ish handling =====

<code bash>
awk -v RS='' '{print NR": "$0}' f    # paragraph mode (blank-line records)
awk -F'";"' '{print $2}' file        # crude quoted-CSV: real CSV → miller
</code>

===== Recipes =====

<code bash>
ps aux | awk '$3>5 {print $2, $3, $11}'        # processes >5% CPU
df -h | awk 'NR>1 && $5+0>80 {print $6,$5}'    # mounts over 80%
awk '{s[$1]++} END{for(x in s) print s[x],x}' access.log | sort -rn | head
awk 'NR==FNR{a[$1]=1;next} !($1 in a)' a.txt b.txt   # lines in b not in a
awk '{print $NF}'                               # last field
seq 100 | awk '{sum+=$1} END{print sum/NR}'     # mean
</code>

===== Gotchas =====

  * Fields are **1-indexed**, ''$0'' is the whole line.
  * ''-F'' takes a regex: ''-F'[,:]''' splits on either.
  * String vs number: ''$1 == "5"'' string compare, ''$1 == 5'' numeric.
  * ''print'' adds ORS (newline); ''printf'' doesn't.
  * mawk lacks ''gensub''/''strftime'': install ''gawk'' if a script needs them.


===== See Also =====

  * [[terminal:scripting:start|Scripting Index]]
  * [[terminal:tools:start|CLI Tools]]
  * [[terminal:start|Return to terminal]]
  * [[start|Return to wiki home]]