Two Neat Features of JQ

I did not realise until very recently that jq, the command line utility for working with JSON files, is a fully fledged programming language. Like AWK, script files can be passed in with the -f flag in addition to the one-lines that are passed in as string args.

Test scripts

The first of the two neat features I really like in jq is what happens if instead of passing the -f flag with a filename, you instead pass the --run-tests with a filename. This runs jq in a special mode where it will use the contents of the provided file as a set of tests to be run. Each test comprises a set of lines separated by an empty line. Each test contains a program line, an input lines, and any output lines to be asserted. So if the test file contained this:

.foo
{ "foo": 4 }
4    

Then jq would check that 4 is indeed the value of the object with key foo. I have a test script that summarises all of the core features of jq as a reference guide, plus its just generally quite handy to help build up scripts. Other than that, there's really not much else to it. All modern development tooling should come with testing support built in, and this is great example of how best to provide such key functionality with these kinds of CLI DSLs.

Powerful conditionals

The second feature of jq that I find really quite cool is the set of powerful assignment operators. Jq obviously has the standard assignment (= operator), and it has the other ones that are common to the c family of languages (*=, +=, -=, /=, %=, and //=), but the most powerful of them all is the |= operator that chains together filers that perform assignment. And then, if that wasn't enough, any expression can be placed on the left side of any assignment statement, including calls to select to only apply the assignment to some objects. A concrete example will be more enlightening: here's my implementation of FizzBuzz:

range(1;.+1)
  | { "number": ., "value": "" }
  | select(.number % 3 == 0) |= (.value += "fizz")
  | select(.number % 5 == 0) |= (.value += "buzz")
  | if .value == "" then .number else .value end    

Simply pass a number in on standard-in and it'll perform FizzBuzz up to that number. Here's how it works

(1) We generate the range from 1 to the given number (+1 because range is exclusive).

(2) For each number we produce an object that crucially gives us a string, called value, to append our strings to as we go

(3) Here is where the magic is. We have an invocation of the update operator -- the proper name of the |= symbol -- with a selection of just the numbers that are divisible by three on the left and on the right we have another assignment. This assignment says find the .value string and append "fizz".

(4) We then do the same thing for Buzz and divisible by 5

(5) Finally we output the string if there is anything to show otherwise we output the number

What's really nice about this is how compact the notation is. It reminds me of how lens and optics work in pure functional programming languages like Haskell and PureScript; where functions that abstract over the process of getting a smaller piece of data out of a larger data structure allow for terse descriptions of how the outer structure should be manipulated without a tonne of boilerplate. Just really quite nice bit of syntax.

~~~

Last Updated: 2023-04-01