folk.zone Wiki

a public knowledge base

Site Tools


terminal:scripting:jq

jq

JSON processor: jq '<filter>'. Reads stdin or files. Exit 0 + output null still means “parsed fine, empty result”.

Reading data

jq . data.json                       # pretty-print (validates too)
jq '.key'                            # field access
jq '.a.b.c' '.a["b-c"]'              # nested / weird keys
jq '.items[0]' '.items[-1]'          # array index / last
jq '.items[1:3]'                     # slice
jq '.[]'                             # every element / value
jq '.[] | .name'                     # field of each element
jq -r '.name'                        # raw output: no quotes
jq '..|objects|.id? // empty'        # all .id keys at any depth

Keys, length, structure

jq 'keys'                            # object keys (sorted)
jq 'keys_unsorted'
jq 'length'                          # array len / string len / object key count
jq 'type'                            # object|array|string|number|boolean|null
jq 'has("key")' 'paths' 'leaf_paths'

Filtering

jq '.[] | select(.age > 30)'
jq '.[] | select(.status == "active") | .id'
jq '.[] | select(.name | test("foo"; "i"))'      # regex match, i-insensitive
jq 'map(select(.enabled))'                        # filter array → array
jq '.users | map(.name)'                          # transform each
jq '[.[] | .name]'                                # collect into new array
jq 'first(.[] | select(.ok))'                     # stop at first match

Building output

jq '{user: .name, total: (.items | length)}'      # reshape object
jq '.[] | "\(.name) is \(.age)"'                  # string interpolation
jq -r '.[] | [.name, .age] | @tsv'                # TSV: paste to awk
jq -r '.[] | @csv'                                # CSV
jq '.[] | @base64'                                # encode
jq -n '{a: 1}'                                    # construct JSON from nothing

Aggregating

jq 'add'                          # sum array of numbers/strings
jq '[.[] | .price] | add'         # sum a field
jq 'max_by(.age) | .name'         # oldest person (also min_by, sort_by)
jq 'group_by(.city)'              # group into arrays
jq 'unique' 'unique_by(.key)' 'flatten' 'reverse'
jq 'reduce .[] as $x (0; . + $x)' # fold: like sum but manual

Combining inputs

jq -s 'add' *.json                # -s: slurp inputs into one array
jq -n '{a: input, b: input}' a.json b.json   # named inputs
jq --slurpfile cfg config.json '. + $cfg[0]' # load as variable
jq --arg v "$val" '.version = $v'            # inject shell var (safe)
jq --argjson n 42 '.count = $n'              # inject JSON (numbers/bools)

Always inject shell values with –arg/–argjson: never interpolate “…$val” into the program or the output (breaks on quotes).

In-place-ish edits & formats

jq '.version = "2.0"' f.json > new.json     # update field (write new file)
jq 'del(.secret)' f.json                    # remove a key
jq -c . f.json                              # compact one-line
jq . <(curl -s api) file2.json              # multiple inputs fine
yq                                          # same idea for YAML (apt: yq)

Recipes

# all running docker container names
docker inspect $(docker ps -q) | jq -r '.[].Name[1:]'
 
# newest file in a dir listing API
jq -r 'sort_by(.mtime) | last | .name'
 
# pretty-print a line-delimited JSON log, filter to errors
jq -c 'select(.level=="error")' app.log
 
# turn a JSON array into a bash-readable list
jq -r '.[]' list.json | while IFS= read -r x; do ...; done

Gotchas

  • .foo on null returns null (not an error): check type when results

look empty. .foo? suppresses errors on non-objects too.

  • Numbers print unquoted; use -r for strings.
  • map(f) ≡ [.[] | f]: same thing, pick the readable one.
  • Big file + jq -s loads it all into RAM: fine for MBs, think twice for GBs.

See Also

terminal/scripting/jq.txt · Last modified: by 127.0.0.1