Public | Automated Build

Last pushed: 5 months ago
Short Description
Image wrapping jq to transparently parse and modify both json and yaml files from the same interface
Full Description

JSON/YAML Parser

Small script that allows to read/set a value for a specific path in a JSON or YAML
file.

The main motivation behind this is to quickly be able to read a specific
property from a JSON/YAML file, do something with that value and then change the
original value with a new one in the document. And being able to do that
transparently regardless of the source format.

To accomplish this, YAML input is converted behind the scenes to JSON using
Python, a jq filter is applied on the JSON and the result is converted back
into YAML.

Like I said before, the goal is to get/set a specific value associated with a
key in the document.
If you are looking to run arbitrary jq-like filters on YAML I'd recommend you to
take a look at y2j.

Installation

If you want to use jyparser outside of Docker, ensure you've got all required tools :

./start req

Usage

The script's entry point accepts 2 operations: get and set. The script can
take its inputs from stdin or read from a file if this is passed as the first
parameter. We'll see some examples below.

Get a value

The get command takes an arbitrary jq filter. If the result is a number,
string or boolean then that value is returned. Otherwise, the resulting JSON or
YAML is returned (depending on what the input was).

Given the following JSON file:

$ cat test.json

{"menu": {
  "id": "file",
  "value": "File",
  "popup": {
    "menuitem": [
      {"value": "New", "onclick": "CreateNewDoc()"},
      {"value": "Open", "onclick": "OpenDoc()"},
      {"value": "Close", "onclick": "CloseDoc()"}
    ]
  }
}}

If you wanted to get the value of the id property you could use:

$ cat test.json | docker run -i --rm jlordiales/jyparser get ".menu.id"

"file"

The json is passed via stdin, which is useful if you get that from something like
curl. If you have an actual file that you want to use as input then you can
pass it directly as the first parameter to the script:

$ docker run -i --rm -v `pwd`:/jyparser:ro jlordiales/jyparser test.json get ".menu.id"

"file"

The example above mounts the current dir with the file into /jyparser (which
is the default WORKDIR for the docker image) and then uses that file as input.

Exactly the same command works for YAML as well. Given the YAML file:

$ cat test.yml
menu:
  id: file
  value: File
  popup:
    menuitem:
    - onclick: CreateNewDoc()
      value: New
    - onclick: OpenDoc()
      value: Open
    - onclick: CloseDoc()
      value: Close

We can get the id property with:

$ cat test.yml | docker run -i --rm jlordiales/jyparser get ".menu.id"

"file"

If the result from running the jq filter is not a simple value, then the
corresponding JSON or YAML is returned:

$ cat test.json | docker run -i --rm jlordiales/jyparser get ".menu.popup.menuitem[1]"

{
  "value": "Open",
  "onclick": "OpenDoc()"
}

$ cat test.yml | docker run -i --rm jlordiales/jyparser get ".menu.popup.menuitem[1]"

onclick: OpenDoc()
value: Open

Set a value

Similarly to the get operation, there's a set one. This operation takes 2
parameters: a jq filter to select a specific element of the input and a new
value to update that element to. The result is the original input with the value
updated.


$ cat test.json | docker run -i --rm jlordiales/jyparser set ".menu.id" \"new_id\"
{
  "menu": {
    "id": "new_id",
    "value": "File",
    "popup": {
      "menuitem": [
        {
          "value": "New",
          "onclick": "CreateNewDoc()"
        },
        {
          "value": "Open",
          "onclick": "OpenDoc()"
        },
        {
          "value": "Close",
          "onclick": "CloseDoc()"
        }
      ]
    }
  }
}

Important: given the way bash scripts handle quotes on parameters passed to
them, if the new value you want to set for the property is a string you need to
explicitly escape the quotes as in the example. Otherwise, jq will complain
that the value is not valid (rightfully so). This is not needed for numbers or
booleans.
So the following works as expected:

$ cat test.json | docker run -i --rm jlordiales/jyparser set ".menu.id" 15
{
  "menu": {
    "id": 15,
    "value": "File",
    "popup": {
      "menuitem": [
        {
          "value": "New",
          "onclick": "CreateNewDoc()"
        },
        {
          "value": "Open",
          "onclick": "OpenDoc()"
        },
        {
          "value": "Close",
          "onclick": "CloseDoc()"
        }
      ]
    }
  }
}

The same works for YAML:

$ cat test.yml | docker run -i --rm jlordiales/jyparser set ".menu.id" \"new_id\"

menu:
  id: new_id
  popup:
    menuitem:
    - onclick: CreateNewDoc()
      value: New
    - onclick: OpenDoc()
      value: Open
    - onclick: CloseDoc()
      value: Close
  value: File

Limitations

Since YAML is actually a superset of JSON
this will only work for those YAML files that can be correctly converted to JSON.

Acknowledgments

The JSON <=> YAML conversion to be able to apply jq filters seamlessly is
heavily inspired by y2j.

Links

You can find more info on the use case and motivation for this project on this
blog post.

The automated build for the docker image can be found
here.

License

MIT.
Full license text is available in LICENSE.

Docker Pull Command
Owner
jlordiales
Source Repository