Admission controller port missing from cluster agent deployment

[jalaziz]

Output of the info page (if this is a bug)

Describe what happened:

I recently noticed that none of my service pods were starting after installing the Datadog Operator. After some investigation, I realized there was an issue with the admission controller. In particular, I found the following in the k8s apiserver logs:

E1026 06:40:38.020600      11 dispatcher.go:206] failed calling webhook "datadog.webhook.config": failed to call webhook: Post "https://datadog-admission-controller.observability.svc:443/injectconfig?timeout=4s": context deadline exceeded

Looking into the configuration, I noticed the the admission service was targeting port 8000 on the cluster agents, but port 8000 was not exposed on the cluster agent deployment.

Looking through the codebase, it doesn't appear that the operator correctly exposes the target port on any pods.

Describe what you expected:

I expected the admission webhook to work.

The helm chart seems to correctly expose the port: https://github.com/DataDog/helm-charts/blob/cdf0acf8aa46c1884ef7978bfad17c49192caf5c/charts/datadog/templates/cluster-agent-deployment.yaml#L144-L148

Steps to reproduce the issue:

Simply run the operator with the cluster agent and admission webhooks enabled (which I believe is the default configuration)

Additional environment details (Operating System, Cloud provider, etc):

[jalaziz]

Turns out I wasn't allowing port 8000 traffic from the control plane as mentioned here.

I'm still surprised it works without exposing the port on the cluster agent though, given that it doesn't seem to be using host networking.

[celenechang]

Hi @jalaziz , thanks for your question. The containerPort specification is informational and omitting it does not prevent ports from being exposed (ref: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#ports ). However it is considered a "best practice" to include it, so we will plan to add this in the Operator.