Apologies for the late post but it has been a busy day. As of today you can buy the early edition of Juggling JSON with jq on Gumroad. I am very excited since this is my first attempt at self publishing a book. Naturally the journey is still continuing as I work towards the final release of the book. All buyers of the book will get updates including the final version when it comes out.
I plan sending out an update on the book and the companion site/API that I am working on.
jq is an amazing tool for querying and manipulating JSON in command-line, that I learned about from one of my good colleagues, Eric Olsen. And I feel that jq deserves a good book describing how to use this tool. Hence I am writing a book called Unjumbling JSON with jq on the topic.
As mentioned in a previous post, I originally planned on writing a single book on both jq and httpie. I divided the original book in two, because there is only a small overlap between the two. I wanted to show examples of grabbing a REST API response via httpie, and parsing the JSON output with jq. However basic querying a REST API is something that could be covered in a short section. By writing the books separately, I will be able to release them faster, and the books will be much more focused.
I plan on selling early drafts of the ebook on August 10th. Buyers of the ebook will get regular versions of the evolving drafts of the ebook, and a free upgrade to the final version of the book. I want to release the early drafts to get early reader feedback. In addition readers of the book will have access to the REST API that accompanies the book.
You can order the early version of Unjumbling JSON with jq from here].
About 2 or 3 months ago, when testing a deployment of a microservice at work
with Eric, our head Ops admin, we were looking at the JSON output of one of
the REST endpoints. Rather than looking at the raw output from curl, I
piped the output through JSON tool in the Python standard library:
$ curl -X GET -s http://rookeries.org/status | python -m json.tool
{
"app": "rookeries",
"version": "0.4.9"
}
(I will give you a couple of examples based on calls to Rookeries, rather than
the actual service call, since that API isn’t available publicly yet.)
Eric seeing that suggested I try out a utility that he uses: jq which is
great for formatting and querying JSON.
jq is a utility in C, and fortunately there are binary packages available for
it in the latest Ubuntu LTS (14.10). Installing it via aptitude (or apt-get):
$ sudo aptitude install jq
The simplest use case for jq is to simply pretty print JSON output. This is
done by piping the result of a CURL command to jq with the parameter ‘.’:
$ curl -X GET -s http://status.bitbucket.org/api/v2/status.json | jq .
{
"status": {
"description": "All Systems Operational",
"indicator": "none"
},
"page": {
"updated_at": "2015-09-03T08:03:55.275Z",
"url": "http://status.bitbucket.org",
"name": "Atlassian Bitbucket",
"id": "bqlf8qjztdtr"
}
}
It is also possible to format the output of the JSON to display only relevant
information. For instance if I want to find out the status of the components
that make up Bitbucket I can do the following:
$ curl -X GET -s http://status.bitbucket.org/api/v2/components.json | jq .
That however will give me a whole lot of extra data that I might not want. So
instead I might to narrow down and re-format the JSON data to something more
manageable with some jq magic:
$ curl -X GET -s http://status.bitbucket.org/api/v2/components.json | \
jq '{src: .page, components: [.components[] | \
{id: .id, name: .name, status: .status}]}'
{
"components": [
{
"status": "operational",
"name": "Website",
"id": "g0lfj4sv2fhf"
},
{
"status": "operational",
"name": "API",
"id": "k0x2yw1435v7"
},
{
"status": "operational",
"name": "SSH",
"id": "qmh4tj8h5kbn"
},
{
"status": "operational",
"name": "Git via HTTPS",
"id": "c1qmcrcbc5zy"
},
{
"status": "operational",
"name": "Mercurial via HTTPS",
"id": "vmbzxbbjz05j"
},
{
"status": "operational",
"name": "Webhooks",
"id": "rfzky0v13fbp"
},
{
"status": "operational",
"name": "Source downloads",
"id": "28h8dvv2qfzw"
}
],
"src": {
"updated_at": "2015-09-03T08:03:55.275Z",
"url": "http://status.bitbucket.org",
"name": "Atlassian Bitbucket",
"id": "bqlf8qjztdtr"
}
}
I won’t explain that particular string in detail. But it will basically
craft a new JSON object, and generate new filtered objects when iterating
over the old array. Overall jq is pretty neat and is extremely fast to work
with.
There is also a Python bindings library for jq. I am considering using it to
help with mapping JSON into Python objects. However I have not played around
with it long enough to know if the extra dependencies are worthwhile or
whether or not it will bring a lot of benefits to Rookeries.