Using driver in "docker" compatibility mode causes podman service to consume excessive CPU

[optiz0r]

I'm not sure if this is a problem with this driver or podman itself, but running a job which uses task { driver = "docker" } on a host with nomad-driver-podman, podman and podman-docker (compatibility shim which points /var/run/docker.sock at podman), nomad jobs cause the podman system service process to consume excessive amounts of CPU. Taking the same job and switching the task driver from docker to podman makes the podman system service consume negligible cpu once more.

In this example, nomad is running a single job with two tasks, and the system service is consuming a consistent 1.6 cores.

Taking the exact same job spec, and switching out the task driver (and replacing mount blocks with volumes) reduces the cpu load significantly

This was the effect when I switched all nomad jobs running on a 6 core E5-1650 vmware server, hosting 4 Nomad VMs running docker jobs to the native podman driver!

This is running on:

• AlmaLinux 8.5 (kernel 4.18.0-348.2.1.el8_5.x86_64)
• Nomad 1.2.3
• nomad-driver-podman 0.3
• Podman 3.3.1

[towe75]

@optiz0r thank you for your report. Nice that our driver performes well for you.

To your question: i do not see any relationship between driver="docker" and this issue tracker. The docker driver is for docker and the podman driver is for podman. Of course you can try to "bridge" between docker driver and podman backend to use some docker-driver-only feature but this is something that we will not cover or test in this project.

Are you missing some feature in podman driver?

[optiz0r]

D'oh, yes of course this was the wrong place to report, sorry to have bothered you with it!

Are you missing some feature in podman driver?

Yes I was using the compat shim because the podman driver does not yet handle volume mounts in the job spec (#142).