Can't start docker daemon.

[jarrettj]

Hi,

Good day.

Upgraded to Virtualbox 5.2.6 r120293 on a Mac. The dinghy up command gives the following error:

Starting the dinghy VM...
Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded
/usr/local/Cellar/dinghy/4.5.0/cli/dinghy/system.rb:18:in `system': Failure calling `docker-machine start dinghy` (System::Failure)
	from /usr/local/Cellar/dinghy/4.5.0/cli/dinghy/machine.rb:126:in `system'
	from /usr/local/Cellar/dinghy/4.5.0/cli/dinghy/machine.rb:25:in `up'
	from /usr/local/Cellar/dinghy/4.5.0/cli/cli.rb:208:in `start_services'
	from /usr/local/Cellar/dinghy/4.5.0/cli/cli.rb:79:in `up'
	from /usr/local/Cellar/dinghy/4.5.0/cli/cli.rb:134:in `restart'
	from /usr/local/Cellar/dinghy/4.5.0/cli/cli.rb:152:in `upgrade'
	from /usr/local/Cellar/dinghy/4.5.0/cli/thor/lib/thor/command.rb:27:in `run'
	from /usr/local/Cellar/dinghy/4.5.0/cli/thor/lib/thor/invocation.rb:126:in `invoke_command'
	from /usr/local/Cellar/dinghy/4.5.0/cli/thor/lib/thor.rb:359:in `dispatch'
	from /usr/local/Cellar/dinghy/4.5.0/cli/thor/lib/thor/base.rb:440:in `start'
	from /usr/local/bin/_dinghy_command:12:in `<main>'

Tried upgrading dinghy with upgrade command. It upgraded to:

Latest release for github.com/boot2docker/boot2docker is v18.02.0-ce

That didn't help. Does dinghy require the Virtualbox tools installed or something?

Also, dinghy status gives:

Jarretts-MacBook-Pro:legal thejj$ dinghy status
   VM: running
  NFS: stopped
 FSEV: stopped
  DNS: stopped
PROXY: stopped

NFS failed to run
details available in log file: /usr/local/Cellar/dinghy/4.5.0/local/var/dinghy-NFS.output

FsEvents failed to run
details available in log file: /usr/local/Cellar/dinghy/4.5.0/local/var/dinghy-FsEvents.output

Your environment variables are already set correctly.

Regards.
JJ

[codekitchen]

Looks like docker-machine is failing to bring up the VM. If you run docker-machine start dinghy to start up the VM and bypass Dinghy completely, you may get more details on the error. It might be worth checking the Virtualbox UI as well for any error displayed there.

In my experience, a lot of issues with Virtualbox tend to disappear after a reboot, sad as that is.

[jarrettj]

Hi,

Thanks for the reply and tips. :)

I tried docker-machine start dinghy, the output I got was:

$ docker-machine start dinghy
Starting "dinghy"...
(dinghy) Check network to re-create if needed...
(dinghy) Waiting for an IP...
Machine "dinghy" was started.
Waiting for SSH to be available...
Detecting the provisioner...
Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded

What is weird is the workaround.

1. execute dinghy up
2. while it loads open Virtualbox UI.
3. Select dinghy machine and click the Show button.
4. When booting is complete press Enter button in the Machine window.

It is as if dinghy waits for the Enter command.

Regards.
JJ

[chinthakagodawita]

Did you upgrade to boot2docker 18.02 recently as well? We're seeing increased problems with it and had to rollback to 17.12 till 18.03 is released (as per #270):

dinghy create --provider=virtualbox --boot2docker-url=https://github.com/boot2docker/boot2docker/releases/download/v17.12.1-ce/boot2docker.iso

[jarrettj]

Thanks @chinthakagodawita. I'll revert until 18.03 is released.

Regards.
JJ

[KamilKopaczyk]

@codekitchen

i ran into similar problem. In my case, it's "incorrectly" configured /etc/docker/daemon.json file.
As soon as i add "debug": false (or true - it doesn't matter) i'm unable to restart dinghy - machine is starting, but docker daemon process is dead.

Is there any way to get more info/logs from booting and process starting? Or start daemon process manually to get some output?
I tried to restart docker daemon manually, but failed:

Boot2Docker version 17.10.0-ce, build HEAD : 34fe485 - Wed Oct 18 17:16:34 UTC 2017
Docker version 17.10.0-ce, build f4ffd25
docker@dinghy:~$ sudo -i
Boot2Docker version 17.10.0-ce, build HEAD : 34fe485 - Wed Oct 18 17:16:34 UTC 2017
Docker version 17.10.0-ce, build f4ffd25
root@dinghy:~# /etc/init.d/docker start
Device "eth1" does not exist.
Need TLS certs for dinghy,127.0.0.1,172.16.26.129
-------------------
root@dinghy:~#

[codekitchen]

I'm looking at docker-machine create --help but I don't see any flags or anything that look likely to increase debug/logging output. I'm not an expert at debugging docker-machine, you might need to ask over at that project.

You can create and start the VM directly with docker-machine create and docker-machine start to take Dinghy out of the picture, that may help tracking down the issue. Essentially all that Dinghy does internally on these steps is docker-machine create -d virtualbox dinghy and docker-machine start dinghy with any other flags for CPU, mem, boot2docker URL, etc. It just waits for those commands to complete before starting its own Dinghy daemons.

If you're seeing an issue where you have to press Enter in the VM terminal before docker-machine start completes when using boot2docker 18.02, I bet they'd be interested in hearing about that.

[KamilKopaczyk]

My case is a little bit different - i'm trying to disable "debug" flag for docker daemon, because /var/lib/boot2docker/docker.log tends to grow to a size of couple GB's and i clear it manually from time to time.

Anyway, thanks to your comment i found what i was looking for in :)

[jarrettj]

How would you revert an existing docker-machine @chinthakagodawita?

[chinthakagodawita]

Unfortunately, I don't know a way apart from re-creating it:

dinghy destroy
dinghy create --provider=virtualbox --boot2docker-url=https://github.com/boot2docker/boot2docker/releases/download/v17.12.1-ce/boot2docker.iso

[jarrettj]

Good news, the dinghy upgrade command worked. It automatically downgraded to v17.12.1-ce. Wouldn't want to lose all my existing docker images.

But alas the problem still persists! :(

Still using the workaround.

Thanks though for the response.

JJ

[codekitchen]

It sounds like this has been fixed upstream now with the 18.03 release, closing this issue.