Jsonnet nested template data

[mckunda]

Overview

This PR provides ability to put nested YAML arrays and mappings as serialized JSON string values into jsonnet's vm.ExtVar function and use it later in the jsonnet pipeline template with std.parseJson.

Example pipeline:

kind: template
load: plugin.jsonnet
data:
  stepName: my_step
  image: my_image
  commands: my_command
  additionalSteps:
    - name: my_second_step
      image: my_second_image
      commands:
        - my_second_command1
        - my_second_command2
    - name: my_third_step
      image: my_third_image
      commands:
        - my_third_command

Example jsonnet template:

local stepName = std.extVar("input.stepName");
local image = std.extVar("input.image");
local commands = std.extVar("input.commands");
local additionalSteps = std.parseJson(std.extVar("input.additionalSteps"));

{
  "kind": "pipeline",
  "type": "docker",
  "name": "default",
  "steps": [
    {
      "name": stepName,
      "image": image,
      "commands": [
        commands
      ]
    }
  ] + [step for step in additionalSteps]
}

Processed output:

---
{
   "kind": "pipeline",
   "name": "default",
   "steps": [
      {
         "commands": [
            "my_command"
         ],
         "image": "my_image",
         "name": "my_step"
      },
      {
         "commands": [
            "my_second_command1",
            "my_second_command2"
         ],
         "image": "my_second_image",
         "name": "my_second_step"
      },
      {
         "commands": [
            "my_third_command"
         ],
         "image": "my_third_image",
         "name": "my_third_step"
      }
   ],
   "type": "docker"
}

Of course, the above examples seem a bit of a stretch and make the new functionalty look redundant, but those are for testing purposes only. With the power of jsonnet we can implement much more complex templating solutions for a broader set of use cases. I can try and work on the real-world example if you want.

[CLAassistant]

All committers have signed the CLA.

[bradrydzewski]

If we had to choose between json and yaml, I would probably serialize to json for consistency reasons, since this is how we serialize plugin parameters that are complex slice or map values. But otherwise looks good to me.

[bradrydzewski]

do we need to marshal to yaml and then json? or can we just marshal directly to json, and skip marshaling to yaml?

data, _ := json.Marshal(v)
val = string(data)

[mckunda]

Marshalling directly to json gives the json: unsupported type: map[interface {}]interface {} error, which I don't quite know how to work around. I've looked up the code for Docker runner and found that this exact approach was used for plugin parameter serialization, but I didn't bother to copy over the whole encoder package for my case. Please let me know if you have some better approach in mind

[mckunda]

Also, checking the sources of jsonnet-go lib reveals that std.ParseYaml too just converts the incoming yaml to json, then the behavior is the same as in std.parseJson. So, if you trust jsonnet's yaml conversion enough, we could just stick with yaml and let jsonnet do the yaml-to-json stuff for us

[bot2-harness]

This PR has been automatically closed due to inactivity.