Feature request: read from stdin as a resource

[torsten-liermann]

Would it be possible to have kustomize build read the resource from stdin, just like Unix pipes work?

That would simplify use cases, which at the moment include kustomize unnecessarily via wrapper scripts.

For example: helm template --post-renderer ./kustomize-wrapper.sh

kustomize-wrapper.sh:

cat> input-for-kustomize.yaml
Customize build.

It would be conceivable to have kustomize read from stdin if no resource is specified, or "-" (minus) or /dev/fd/0 for a resource path

Thanks!

[Shell32-Natsu]

@monopole WDYT?

[monopole]

For the particular case of reading from a helm chart, we have this example. It could use some improvement - see #3128

A previously proposed interpretation of a missing resources: field was to treat it as a wildcard, e.g. a missing resources: field would be equivalent to

resources:
- *
- */*
- */*/*   (etc.)

If this implicit inclusion method were supported (i.e. if no resource manifest is specified, then read all the files in the directory tree), then kustomize would have to provide an exclusion manifest to specify the files that should not be included (like a .gitignore file). A program like kustomize must specify either inclusions or exclusions; we opted to specify inclusions (like a Makefile). Also, globbing is not supported.

The goal of these choices is to err on the side of literal declaration of the cluster's configuration. One should only have to consult a kustomization file (and the files it points to) to understand the config, and not also have to understand what was on disk at the time someone ran kustomize build.

Reading from stdin is technically possible of course. We could allow {stdin} (or something YAML friendly) in the list of resources: and use that to trigger a read from stdin.

But it breaks the notion of declaring what you use even more than globbing does. With globbing, you can maybe assume that if the kustomization file and its surroundings were maintained in source control that you have a clear declaration of configuration.

Leaving this issue open for more comment.

Would like to see #3128 done before thinking about allowing stdin, to see if that's good enough.

Meanwhile suggest try following the chartinflator example (see the unit test).

[aodinokov]

kustomize read from stdin if no resource is specified, or "-" (minus) or /dev/fd/0 for a resource path

Not sure it's the best option, because kustomize may call other kustomization in resources and it also may have that "-".

One more idea - to add a flag --as-transformer to kustomize build command that would read stdin and add all read yamls as like they were mentioned in kustomization.yaml resorces section located in DIR passed to kustomize build.

The underlaying kustomizations in that case won't get that resources - only the top one.
Also I think it correlates with the general idea that everything is transformer in a way that right now kustomize is rather a generator, because it can't accept and transform stdin.
But if we make it - it will be possible to put only a chain of transformers in kustomization.yaml and kustomize will apply all of them to stdin and put everything in stdout like a normal transformer... and this is great, because 'everything is transformer' even kustomize itself :) ... btw it will work more like kpt fn run

also it may allow to remove some of that separate commands in kustomize cfg, e.g.

  annotate      [Alpha] Set an annotation on Resources.
  grep          [Alpha] Search for matching Resources in a directory or from stdin

because it's possible to do them with kustomization... to be specific annotate works oob and grep may be done with patch (with target and $patch:delete).

cc: @monopole

[KnVerey]

@monopole and I discussed this at length, and in the end we agreed that this feature should not be accepted. Although the trigger isn't CLI args or env vars in this case, it is definitely still a "build-time side effect". While it may be convenient for certain workflows built around it, facilitating dependence on build-time side-effects violates an important principle of Kustomize's approach. As @monopole commented earlier, "one should only have to consult a kustomization file (and the files it points to) to understand the config, and not also have to understand what was on disk at the time someone ran kustomize build." We will update the Eschewed Features to clarify this.

As for the Helm use case in particular, the example mentioned earlier has been replaced with / promoted to a built-in transformer, and contributions to make it better for real-world Helm use cases are appreciated.

/close

[k8s-ci-robot]

@KnVerey: Closing this issue.

In response to this:

@monopole and I discussed this at length, and in the end we agreed that this feature should not be accepted. Although the trigger isn't CLI args or env vars in this case, it is definitely still a "build-time side effect". While it may be convenient for certain workflows built around it, facilitating dependence on build-time side-effects violates an important principle of Kustomize's approach. As @monopole commented earlier, "one should only have to consult a kustomization file (and the files it points to) to understand the config, and not also have to understand what was on disk at the time someone ran kustomize build." We will update the Eschewed Features to clarify this.

As for the Helm use case in particular, the example mentioned earlier has been replaced with / promoted to a built-in transformer, and contributions to make it better for real-world Helm use cases are appreciated.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[aodinokov]

This is a great explanation...

"one should only have to consult a kustomization file (and the files it points to) to understand the config, and not also have to understand what was on disk at the time someone ran kustomize build."

When I PoC'ed that feature with additional flag --as-transformer I had a gut feeling that it should be a different command. something like kustomize transform rather than kustomize build --as-transformer :) But this implies lots of additional use-cases. This is potentially a completely different tool that has a lot in overlap with kustomize fn run/kpt fn run :)

Thank you @KnVerey :)

[tcurdt]

Disappointing. This would be great:

cat kustomization.yaml | kustomize build - | kubectl apply -f -