Proposal: Provide a secure storage for policy headers

[abuzhynsky]

Currently pomerium-operator doesn't provide a secure way to store policy set_request_headers key.

Maybe it makes sense to add a new annotation like:

ingress.pomerium.io/policy_from_secret: "secretName"

which will point at the existing secret in the same namespace as the ingress/service. The secret will contain the sensitive bits of a policy like headers or a client certificate.

Example secret:

apiVersion: v1
kind: Secret
metadata:
  name: secretName
type: Opaque
stringData:
  set_request_headers:
    Authorization: Basic YWRtaW46YWRtaW4K
    X-Example: example

I can help with the implementation if it sounds like a good idea.

[travisgroth]

I get the use case here. The key name for the annotation isn't sitting well with me, though. Currently we take whatever is after / and treat it like a key in policy. Truthfully, ingress.pomerium.io might have been a bad choice for a name.

1. What are your thoughts on:

policy.pomerium.io/*: inline policy statements
policy-secret.pomerium.io/*: policy statements sourced from a secret

2. The namespace of the secret is tricky
   a. Putting it into the ingress's namespace (where your application is presumably not) is going to create an operational challenge in some environments.
   b. Allowing the operator to view secrets (in your namespace) can be done with a ClusterRole which you grant access to your namespace, I believe the granularity is either "all secrets" or "exactly this set of secrets", which isn't ideal if there's a global/default role that an ingress user may not have control over. I really wish you could use label matching for roles.
   c. We could provide a template Role and document the service account the operator runs as, then leave it to the user to provide a role and binding in their namespace that grants the service account appropriate permissions. This leaves the decision of whitelist vs wildcard permissions tradeoffs to the user.

2c seems at least workable to me. Are there other models I'm not thinking of?

[travisgroth]

Moving this issue to the Pomerium Ingress Controller (docs). See deprecation notice for more info.