Branching Compositional Data Transformations in jq, Visually

Abstract

jq is a widely-used command-line tool for filtering and transforming JSON data, in the vein of sed and awk, including a bespoke programming language for writing the filters. The paradigm of that language is unusual: while its appearance is somewhere between a shell pipeline and JavaScript, the language is pure-functional and essentially concatenative, the pipelines branch and interleave data invisibly, implicit output flattening obscures all these effects, and most users are unaware of any of its semantics, leading to confusion when encountering any of these latent elements and difficulty in constructing appropriate non-trivial filters, all the while common debugging techniques are also obstructed by the paradigm. These confusions can be eliminated by visually demonstrating the recursively forking nature of evaluation on actual data, and allowing manipulations of the program or data to be reflected live, but no such tool exists for jq or any similar approaches. We present a visualisation of jq’s execution model that makes the branching nature manifest, illustrating its effects on concrete values provided by the user, and editing affordances that allow manipulating the program with reference to real data.

Authors

Michael Homer

Published in

ACM SIGPLAN International Workshop on Programming Abstractions and Interactive Notations, Tools, and Environments (PAINT), 2023

The final copy of this publication is available from the publisher.

Resources

PDF: mwh.nz/pdf/paint2023
this page: mwh.nz/pubs/paint2023