Native support for Docker operations

[srivathsanvc]

Summary

Argo workflows to support docker operations natively, without requiring additional cluster privileges.

Motivation

Argo workflows is used in build and deployment situations situations to do docker operations, such as:

docker pull
docker push
docker login
docker build

At Intuit, we use Argo workflows to interact with AWS SageMaker. One of the steps in interacting with SageMaker is to push containers into ECR (via docker push).

The best way to achieve these operations is to have a docker sidecar and a privileged connection to the docker socket. This requires elevated privileges for the workflow on the cluster.

However, many organizations have managed K8s clusters, where privileged access to the docker socket is not permitted for security reasons. In these scenarios, performing the docker operations via Argo workflows is extremely difficult.

Proposal

It would be useful to have the capability for Argo workflow steps to be able to perform these docker operations without having to be executing in a privileged mode.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[alexec]

@srivathsanvc don’t forget to add to the “backlog” milestone.

[edlee2121]

For something like a build farm, one would want a scalable docker service.
One option is to run docker as a service/deployment, likely on the same cluster, and access it via HTTPS.
https://gist.github.com/kekru/4e6d49b4290a4eebc7b597c07eaf61f2

[alexec]

I have solved this privileged access aspect.