docs: Add docs for securityContext and emptyDir. Closes argoproj#…

…2239 (argoproj#4251)

docs/empty-dir.md

@@ -0,0 +1,34 @@
+# Empty Dir
+
+While by default, the Docker and PNS [workflow executors](workflow-executors.md) can get output artifacts from the base layer (e.g. `/tmp`), neither the Kubelet or K8SAPI exectuors can. Nor are you likely to be able to get output artifacts from the base layer if you run your workflo pods a [security context](workflow-pod-security-context.md). 
+
+You can work-around this constraint by mounting volumes onto your pod. The easiest way to do this is to use as `emptytDir` volume. 
+
+!!! Note 
+ This is only needed for output artifacts. Input artifacts are automatically mounted to an empty-dir if needed
+
+This example shows how to mount an output volume: 
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+metadata:
+ generateName: empty-dir-
+spec:
+ entrypoint: main
+ templates:
+ - name: main
+ container:
+ image: argoproj/argosay:v2
+ volumeMounts:
+ - name: out
+ mountPath: /mnt/out
+ volumes:
+ - name: out
+ emptyDir: { }
+ outputs:
+ artifacts:
+ - name: message
+ path: /mnt/out/message
+
+```

docs/security.md

@@ -12,4 +12,8 @@ Finally, you should configure the `argo-server` role and role binding with the c
 
 ### Read-Only
 
-You can achieve this by configuring the `argo-server` role ([example](https://github.com/argoproj/argo/blob/master/manifests/namespace-install/argo-server-rbac/argo-server-role.yaml) with only read access (i.e. only `get`/`list`/`watch` verbs). 
+You can achieve this by configuring the `argo-server` role ([example](https://github.com/argoproj/argo/blob/master/manifests/namespace-install/argo-server-rbac/argo-server-role.yaml) with only read access (i.e. only `get`/`list`/`watch` verbs).
+
+## Workflow Pod Security
+
+See [workflow pod security context](workflow-pod-security-context.md).

docs/workflow-executors.md

@@ -4,56 +4,74 @@ A workflow executor is a process that conforms to a specific interface that allo
 
 The executor to be used in your workflows can be changed in [the configmap](./workflow-controller-configmap.yaml) under the `containerRuntimeExecutor` key.
 
+
 ## Docker (docker)
 
 **default**
 
-### Pros
-
-* Most reliable and well-tested executor
-* Supports all workflow examples
-* Highly scalable as it communicates directly with the docker daemon for heavy lifting
-* Output artifacts can be located on the base layer (e.g. /tmp)
-
-### Cons
-
-* Least secure as it required `docker.sock` of the host to be mounted which is often rejected by OPA.
-
+* Reliability:
+ * Most well-tested
+ * Most popular
+* Least secure:
+ * It requires `priviledged` access to `docker.sock` of the host to be mounted which. Often rejected by Open Policy Agent (OPA) or your Pod Security Policy (PSP).
+ * It can escape the privileges of the pod's service account
+ * It cannot [`runAsNonRoot`](workflow-pod-security-context.md).
+* Most scalable:
+ * It communicates directly with the local Docker daemon.
+* Artifacts:
+ * Output artifacts can be located on the base layer (e.g. `/tmp`). 
+* Configuration:
+ * No additional configuration needed.
+
 ## Kubelet (kubelet)
 
-### Pros
-
-* Secure since you cannot escape the privileges of the pod's service account
-* Moderately scalable Log retrieval and container operations are performed against the kubelet
-
-### Cons
-
-* Additional kubelet configuration may be required
-* Output artifacts can only be saved on volumes (e.g. emptyDir) and not the base image layer (e.g. /tmp)
+* Reliability:
+ * Least well-tested 
+ * Least popular
+* Secure
+ * No `priviledged` access 
+ * Cannot escape the privileges of the pod's service account
+ * [`runAsNonRoot`](workflow-pod-security-context.md) - TBD, see [#4186](https://github.com/argoproj/argo/issues/4186)
+* Scalable:
+ * Operations performed against the local Kubelet
+* Artifacts:
+ * Output artifacts must be saved on volumes (e.g. [emptyDir](empty-dir.md)) and not the base image layer (e.g. `/tmp`)
+* Configuration:
+ * Additional Kubelet configuration maybe needed
 
 ## Kubernetes API (k8sapi)
 
-### Pros
-
-* Secure since you cannot escape the privileges of the pod's service account
-* No extra configuration is required
-
-### Cons
-
-* Least scalable since log retrieval and container operations are performed against the kubernetes api
-* Output artifacts can only be saved on volumes (e.g. emptyDir) and not the base image layer (e.g. /tmp)
+* Reliability:
+ * Well-tested
+ * Popular
+* Secure:
+ * No `priviledged` access 
+ * Cannot escape the privileges of the pod's service account
+ * Can [`runAsNonRoot`](workflow-pod-security-context.md)
+* Least scalable:
+ * Log retrieval and container operations performed against the remote Kubernetes API
+* Artifacts:
+ * Output artifacts must be saved on volumes (e.g. [emptyDir](empty-dir.md)) and not the base image layer (e.g. `/tmp`)
+* Configuration:
+ * No additional configuration needed.
 
 ## Process Namespace Sharing (pns)
-*PNS can't be used when running Windows containers within your workflow as sharing process namespaces [doesn't work on Windows](https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#v1-pod).*
-
-### Pros
 
-* Secure since you cannot escape the privileges of the pod's service account
-* Output artifacts can be located on the base layer (e.g. /tmp)
-* Highly scalable. Process polling is done over procfs rather than the Kubernetes/Kubelet API
+* Reliability:
+ * Well-tested 
+ * Popular
+* Secure:
+ * No `priviledged` access 
+ * cannot escape the privileges of the pod's service account
+ * Can [`runAsNonRoot`](workflow-pod-security-context.md), if you use volumes (e.g. [emptyDir](empty-dir.md)) for your output artifacts
+* Scalable:
+ * Most operations use local `procfs`.
+ * Log retrieval uses the remote Kubernetes API
+* Artifacts:
+ * Output artifacts can be located on the base layer (e.g. `/tmp`)
+ * Cannot capture artifacts from a base layer which has a volume mounted under it
+* Configuration:
+ * No additional configuration needed.
 * Process will no longer run with PID 1
+* [Doesn't work for Windows containers](https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#v1-pod).
 
-### Cons
-
-* Immature
-* Cannot capture artifact directories from base image layer which has a volume mounted under it

docs/workflow-pod-security-context.md

@@ -0,0 +1,27 @@
+# Workflow Pod Security Context
+
+By default, a workflow pods run as root. The Docker executor even requires `privileged: true`.
+
+For other [workflow executors](workflow-executors.md), you can run your workflow pods more securely by configuring the [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your workflow pod.
+
+This is likely to be necessary if you have a [pod security policy](https://kubernetes.io/docs/concepts/policy/pod-security-policy/). You probably can't use the Docker executor if you have a pod security policy. 
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+metadata:
+ generateName: security-context-
+spec:
+ securityContext:
+ runAsNonRoot: true
+ runAsUser: 8737 #; any non-root user
+ priviledged: false
+```
+
+You can configure this globally using [workflow defaults](default-workflow-specs.md).
+
+!!! Warning "It is easy to make a workflow need root unintentionally"
+ You may find that user's workflows have been written to require root with seemingly innocuous code. E.g. `mkdir /my-dir` would require root. 
+
+!!! Note "You must use volumes for output artifacts" 
+ If you use `runAsNonRoot` - you cannot have output artifacts on base layer (e.g. `/tmp`). You must use a volume (e.g. [empty dir](empty-dir.md)). 

mkdocs.yml

@@ -33,6 +33,7 @@ nav:
  - service-accounts.md
  - workflow-rbac.md
  - node-field-selector.md
+ - empty-dir.md
  - workflow-templates.md
  - cluster-workflow-templates.md
  - cron-workflows.md
@@ -42,6 +43,7 @@ nav:
  - artifact-repository-ref.md
  - resource-duration.md
  - estimated-duration.md
+ - workflow-pod-security-context.md
  - progress.md
  - workflow-creator.md
  - synchronization.md
@@ -117,7 +119,6 @@ nav:
  - installation.md
  - Configuration:
  - managed-namespace.md
- - service-accounts.md
  - configure-artifact-repository.md
  - workflow-controller-configmap.md
  - workflow-executors.md