Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening

[L-U-C-K-Y]

Hi all

I have been using this library for a couple of weeks now and did not face any issues.

Since this morning, I run into the following error:

ERROR: Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded  driver=hetzner name=runner-xxx-xxx operation=create

Here the full log:

listen_address not defined, metrics & debug endpoints disabled  builds=0
[session_server].listen_address not defined, session endpoints disabled  builds=0
Checking for jobs... received                       job=899405106 repo_url=https://gitlab.com/...git runner=u_RKHSRB
Creating CA: /root/.docker/machine/certs/ca.pem     driver=hetzner name=runner-xxx-xxx operation=create
Creating client certificate: /root/.docker/machine/certs/cert.pem  driver=hetzner name=runner-xxx-xxx operation=create
Running pre-create checks...                        driver=hetzner name=runner-xxx-xxx operation=create
Creating machine...                                 driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                    Creating SSH key...  driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                    SSH key not found in Hetzner. Uploading...  driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                    Creating Hetzner server...  driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                     -> Creating server runner-xxx-xxx[x] in create_server[121954711]  driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                     -> Server runner-xxx-xxx[x]: Waiting to come up...  driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                    Using public network ...  driver=hetzner name=runner-xxx-xxx operation=create
(runner-xxx-xxx)                                     -> Server runner-xxx-xxx[x] ready. Ip x.x.x.x  driver=hetzner name=runner-xxx-xxx operation=create
Waiting for machine to be running, this may take a few minutes...  driver=hetzner name=runner-xxx-xxx operation=create
Detecting operating system of created instance...   driver=hetzner name=runner-xxx-xxx operation=create
Waiting for SSH to be available...                  driver=hetzner name=runner-xxx-xxx operation=create
Detecting the provisioner...                        driver=hetzner name=runner-xxx-xxx operation=create
Provisioning with ubuntu(systemd)...                driver=hetzner name=runner-xxx-xxx operation=create
Installing Docker...                                driver=hetzner name=runner-xxx-xxx operation=create
Copying certs to the local machine directory...     driver=hetzner name=runner-xxx-xxx operation=create
Copying certs to the remote machine...              driver=hetzner name=runner-xxx-xxx operation=create
Setting Docker configuration on the remote daemon...  driver=hetzner name=runner-xxx-xxx operation=create
ERROR: Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded  driver=hetzner name=runner-xxx-xxx operation=create
WARNING: Machine creation failed, trying to provision  error=exit status 1 name=runner-xxx-xxx
Waiting for SSH to be available...                  name=runner-xxx-xxx operation=provision
Detecting the provisioner...                        name=runner-xxx-xxx operation=provision
Installing Docker...                                name=runner-xxx-xxx operation=provision
Copying certs to the local machine directory...     name=runner-xxx-xxx operation=provision
Copying certs to the remote machine...              name=runner-xxx-xxx operation=provision

Anyone has any clues what I could try?

Edit: Found this related issue: docker/machine#4858 (comment).

[sokraflex]

I was running into exactly the same issue and can confirm that downgrading docker fixes the issue:

--engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh"

[JonasProgrammer]

Hi,

this seems to be a core docker-machine issue, as you have already confirmed by your edit. Could you verify @sokraflex's workaround and see if that fixes the issue for you please?

Even if it does not, I don't think there is anything the driver can do here. I'll close this, as it is out-of-scope, but feel free to use this thread for further discussion.

[szEvEz]

Ran into the exact same issue. Apparently it has something to do with the latest docker version 20.10 which was released yesterday. Downgrading docker via engine-install-urlsolved the issue for me.

[fschrempf]

Same issue here. Workaround with downgrading docker works.

[gerwinbrunner]

Is this still an issue?

[timothyallan]

Yes, I still get the same error with Docker version 20.10.4, build d3cb89e

[hectorm]

I was also suffering from this problem but I fixed it by restarting the Docker service after Docker Machine modifies its configuration.

[[runners]]
  [runners.machine]
    MachineOptions = [
      """hetzner-user-data=
        #cloud-config
        runcmd:
          - |
            while sleep 1; do
              if [ -e /etc/systemd/system/docker.service.d/10-machine.conf ]; then
                systemctl restart docker
                break
              fi
            done &
      """
    ]

[JanKoppe]

Had to modify the format of solution by @hectorm a bit, but with this it's working great now:

"hetzner-user-data=#!/bin/sh\nwhile sleep 1; do if [ -e /etc/systemd/system/docker.service.d/10-machine.conf ]; then systemctl restart docker ; break ; fi ; done &",

[alexandernst]

@hectorm 's workaround almost worked for me. Had to add a small delay:

[[runners]]
  [runners.machine]
    MachineOptions = [
      """hetzner-user-data=
        #cloud-config
        runcmd:
          - |
            while sleep 1; do
              if [ -e /etc/systemd/system/docker.service.d/10-machine.conf ]; then
                sleep 15
                systemctl restart docker
                break
              fi
            done &
      """
    ]