Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kube logstream should use it's own auth method, and write events to the build log #13554

Open
deansheather opened this issue Jun 12, 2024 · 1 comment
Labels
feature Something we don't have yet

Comments

@deansheather
Copy link
Member

deansheather commented Jun 12, 2024

Current kube logstream reads agent tokens from the K8s pod spec and then writes to the agent log endpoint. This has a few flaws:

  1. The agent token needs to be readable by kube logstream, which means the CODER_AGENT_TOKEN needs to be plaintext on the pod spec and not referenced via a secret (which is a security issue)
  2. The agent token isn't actually usable until after the job is marked completed, as we only insert the resources/agents into the database on job completion
  3. These logs show as "agent startup script" logs, which doesn't make much sense as they occur before the agent starts

We should refactor kube logstream to use it's own auth method (similarly to provisionerd, proxies, etc.) and custom API endpoints for adding entries to the build log. These logs should appear just like provisioner logs, most likely in their own section.

Pods will need to be linked back to the build via some other way than the token to overcome problem 2. One method would be to require use of a coder.com/workspace-id annotation on the pod. Another method would be to use a JWT for agent tokens that contains the workspace ID, but this isn't future proof if cloud K8s supports instance-identity auth (this also means problem 1 is still an issue).

Ideally whatever method we use can also be used for customers to add to the build log so they don't encounter problem 3.

@coder-labeler coder-labeler bot added the feature Something we don't have yet label Jun 12, 2024
@spikecurtis
Copy link
Contributor

We could probably enhance the kube logstream to pull tokens from secrets, assuming it's given the correct permission.

Minting JWTs for agent tokens is attractive in that we wouldn't have to store anything but the public key in the database, but it introduces an operational burden around the private signing keys. Right now provisionerd mints the agent token, and if we kept that model for the JWT, then external provisioner daemons would need signing keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Something we don't have yet
Projects
None yet
Development

No branches or pull requests

2 participants