JQ Usage Examples¶

This article describes some examples of using jq to manipulate standard Digital Rebar JSON output objects.

Note

Starting with DRP v4.2.0 and newer, the drpcli tool has full jq capabilities built in to the CLI. To use this, create a symbolic link for a filename ending wtih jq to the drpcli binary (eg ln -s drpcli drpjq). This allows for use of jq capabilities without requiring installation of a separate tool/binary.

Here are some various examples of how to use jq to manipulate standard Digital Rebar JSON output.

JQ Raw Mode¶

Raw JSON output is usefull when passing the results of one jq command in to another for scripted interaction. Be sure to specify "Raw" mode in this case - to prevent colorization and extraneous quotes being wrapped around Key/Value data output.

<some command> | jq -r ...

Filter Out gohai-inventory¶

The gohai-inventory module is extremely useful for providing Machine classification information for use by other stages or tasks. However, it is very long and causes a lot of content to be output to the console when listing Machine information. Using a simple jq filter, you can delete the gohai-inventory content from the output display.

Note that since the Param name is gohai-inventory, we have to provide some quoting of the Param name, since the dash (-) has special meaning in JSON parsing.

drpcli machines list | jq 'del(.[].Params."gohai-inventory")'

Subsequently, if you are listing an individual Machine, then you can also filter it's gohai-inventory output as well, with:

drpcli machines show <UUID> | jq 'del(.Params."gohai-inventory")'

List BootEnv Names¶

Get list of bootenvs available in the installed content, by name:

drpcli bootenvs list | jq '.[].Name'

Reformat Output With Specific Keys¶

Get list of machines, output "Name:Uuid" pairs from the the JSON output:

drpcli machines list | jq -r '.[] | "\(.Name):\(.Uuid)"'

Output is printed as follows:

machine1:05abe5dc-637a-4952-a1be-5ec85ba00686
machine2:0d8b7684-9d0e-4c3e-9f89-eded02357521

You can modify the output separator (colon in this example) to suit your needs.

Extract Specific Key From Output¶

jq can also pull out only specific Keys from the JSON input. Here is an example to get ISO File name for a bootenv:

drpcli contents show os-discovery | jq '.sections.bootenvs.discovery.OS.IsoFile'

Display Job Logs for Specific Machine¶

The Job Logs provide a lot of information about the provisioning process of your DRP Endpoint. However, you often only want to see Job Logs for a specific Machine to evaluate provisioning status. To get specific Jobs from the job list - based on Machine UUID, do:

export UUID=`abcd-efgh-ijkl-mnop-qrps"
drpcli jobs list | jq ".[] | select(.Machine==\"$UUID\")"

Using --arg deal with pipes and quotes (DRPCLI v4.6+)¶

In scripts, it can become very difficult to correctly pass variables inside of pipes. For this reason, operators may want to use the jq --arg instead of attempting to inject values into jq queries.

In the example below, we needed to match a value inside an array of JSON objects. The command is passing the machine $uuid into the JQ string is difficult because of the required single ticks. Using --arg varname inputvalue makes it simpler to build and manage inputs for jq pipes.

mc=$(jq --arg uuid "$uuid" -r '.[] | select(.Uuid = "$uuid")' <<< "$CLUSTER_MACHINES")

This example looks for a value in an array.

licensed=$(jq --arg m "$mc" -r 'contains(["$m"])' <<< $endpoints)

This is only supported by the DRPCLI v4.6+

Read file data directly into JQ using --rawfile¶

To include contents of a file (including multi-line files) into a JQ operation, use the --rawfile command line flag. This works like the --args flag except the value defined is read from the specified file.

In the example below, we are reading certs generated by Kubeadmin into the kube-lib/certs JSON structure. The process uses the {{$t}} as the JQ variable name.

KEYSET='{{ .ParamAsJSON "kube-lib/certs" }}'
{{ range $key, $value := .Param "kube-lib/certs" }}
  {{ $path := get $value "path" }}
  {{ $types := get $value "types" }}
  echo "  adding {{$key}} to kube-lib/certs from {{ $path }}"
  K='{"{{$key}}":
    {"keys":[{
      {{ range $t := $types }}
      "{{$t}}": ${{$t}},
      {{- end }}
      "description":"{{$types}} generated by kubeadm"
    }]}
  }'
  KEYSET="$(jq {{range $t := $types}}--rawfile {{$t}} {{ $path }}.{{$t}} {{end}}-rc ". * $K" <<< "$KEYSET")"
{{ end }}

Manipulating Arrays¶

Adding to an array¶

To manipulate an array for storage in a bash environment, you can do this:

    # One item
    profiles="$(jq ". += [ \"value\" ]" <<< "$profiles")"
    # Multiple items
    profiles="$(jq ". += [ \"value", \"value2\" ]" <<< "$profiles")"

Removing from an array¶

To manipulate an array for storage in a bash environment, you can do this:

    # Remove a specific item
    profiles="$(jq "[.[] | select(. != \"hw-$mfgr-base\")]" <<< "$profiles")"
    # multiple conditional removal
    profiles="$(jq "[.[] | select(startswith(\"hw-\") != true or endswith(\"-base\") != true)]" <<< "$profiles")"