Bind-mount problem when building on Windows 10

[pk1972]

As a beginner in Docker i fear i'm missing some vital points here.
Followed instructions in 'Building.md' for making a dev-server.
Got the infamous 'gzip:stdin:not in gzip format ' error as well.
Found the post describing chmod detail, added world permissions to archive but still getting this:
'+ false

• DIR_EXTRACT=/tmp/mq
• mkdir -p /tmp/mq
• cd /tmp/mq
• curl -LO http:https://build:80/mqadv_dev911_ubuntu_x86-64.tar.gz
  % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  100 153 100 153 0 0 25563 0 --:--:-- --:--:-- --:--:-- 30600
• tar -zxvf ./mqadv_dev911_ubuntu_x86-64.tar.gz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c chmod u+x /usr/local/bin/install-mq.sh && sleep 1 && install-mq.sh && rm -rf /var/mqm && /opt/mqm/bin/crtmqdir -f -s' returned a non-zero code: 2
2019/01/30 07:25:03 copy from pipe to stdout failed: write /dev/stdout: The specified network name is no longer available.
build-server
build
docker build --build-arg BASE_IMAGE=mq-sdk:9.1.1.0-x86_64-ubuntu-16.04 -t mq-golang-sdk:9.1.1.0-x86_64-ubuntu-16.04 -f incubating/mq-golang-sdk/Dockerfile .
Sending build context to Docker daemon 10.9MB
Step 1/7 : ARG BASE_IMAGE=mq-sdk:9.1.1.0-x86_64-ubuntu-16.04
Step 2/7 : FROM $BASE_IMAGE
pull access denied for mq-sdk, repository does not exist or may require 'docker login'
Makefile-UBUNTU:267: recipe for target 'build-golang-sdk-ex' failed
make[1]: *** [build-golang-sdk-ex] Error 1
make[1]: Leaving directory '/root/mq-container'
Makefile:73: recipe for target 'build-devserver-ubuntu' failed
make: *** [build-devserver-ubuntu] Error 2'

Propably missing something obvious, but i'm blind to what obviously.

[arthurbarr]

I wonder if the tar.gz hasn't downloaded properly. My copy of mqadv_dev911_ubuntu_x86-64.tar.gz is 839,931,811 bytes. Could you perhaps check your copy, and if it isn't that size, just delete it, and the build will download it again.

If that's not it, it could be related to something on your machine, so some details of how you're running Docker would be useful.

[pk1972]

Tar.gz downloaded ok, checked that .
Running Docker on WSL.
My Docker info:
:~/mq-container# docker info
Containers: 85
Running: 30
Paused: 0
Stopped: 55
Images: 44
Server Version: 18.09.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 96ec2177ae841256168fcf76954f7177af9446eb
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.125-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.934GiB
Name: docker-desktop
ID: LY26:FDZA:CL56:ONZH:NX7G:B7KE:WCRR:G53L:IS3D:PGEK:LPSS:KHCD
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 121
Goroutines: 124
System Time: 2019-01-30T13:29:50.7090629Z
EventsListeners: 1
Username: pk1972
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

[pk1972]

Tried setting $MQ_ARCHIVE_DEV to the downloaded mqadv_dev911_ubuntu_x86-64.tar.gz also but same error happens.
"+ false

• DIR_EXTRACT=/tmp/mq
• mkdir -p /tmp/mq
• cd /tmp/mq
• curl -LO http:https://build:80/root/mq-container/downloads/mqadv_dev911_ubuntu_x86-64.tar.gz
  % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  100 153 100 153 0 0 57346 0 --:--:-- --:--:-- --:--:-- 76500
• tar -zxvf ./mqadv_dev911_ubuntu_x86-64.tar.gz

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now"__

[arthurbarr]

If you're using WSL, you need to look at this guide by Microsoft to get bind-mounts working. There's also a link to this in our doc.

[pk1972]

@arthurbarr I tried that way first. But since that produced the same result i went for the method posted in here:https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly
Which is what i run now. So for me it fails on both methods.

[pk1972]

Wiped my WSL installation, built it from scratch according to instructions provided in the docs
Running Docker works fine, but to build a queuemanager with 9.1.1 still fails with the same error.
I must be missing something fundamental here.

[arthurbarr]

Can you use a bind-mounted volume with Docker? For example:

echo "Hello world" > /tmp/host.file
docker run --rm -it --volume /tmp/host.file:/var/container.file ubuntu:16.04 cat /var/container.file

That should print "Hello world" if bind-mounting volumes is working OK.

The build bind-mounts the tar file into a Docker container, then hosts that tar.gz on an HTTP server, on a local network. The main build container runs (as part of docker build), which uses curl to download the tar file. It does this to ensure the tar file doesn't get committed into a Docker image layer. The trouble is that the curl command appears to only be downloading 153 bytes, instead of the expected 801 MB. So I suspect that the tar file isn't making it into the HTTP server container.

[pk1972]

I think i have an issue with the bind-mounting.
Ran your sample and the result was:
cat: /var/container.file: Is a directory

[arthurbarr]

I think that might happen if Docker can't find the file on the host. The volume in the bind-mount needs to be a fully qualified path.

[pk1972]

@arthurbarr Can you expand on that, "The volume in the bind-mount needs to be fully qualified path" ?
I tried setting volume to a folder on mnt/c (which is accessible from docker, and writable) but i still get the error 'cat: /mnt/c/test/container.file: Is a directory'

[arthurbarr]

I mean that the host side of the bind mount directory (before the ":") needs to be a fully qualified path (not a relative path). It would help if you could supply the full command you are using. Based on your example file name there, you might try running this:

echo "Hello world" > /mnt/c/test/host.file
docker run --rm -it --volume /mnt/c/test/host.file:/var/container.file ubuntu:16.04 cat /var/container.file

[pk1972]

Well, got an update for my Windows 10 (updated to 1809 build) and had to reinstall WSL completely.
Did the WSL setup linked in Build Docs and still fails with the TAR archive.
Ran your sample and here is the result:
pk1972@SEPC4234:~/mq-container/downloads$ echo "Hello world" > /mnt/c/test/host.file

pk1972@SEPC4234:~/mq-container/downloads$ docker run --rm -it --volume /mnt/c/test/host.file:/var/container.file ubuntu:16.04 cat /var/container.file
cat: /var/container.file: Is a directory

Maybe i should just do VMWare Linux and use docker from there instead ?

[arthurbarr]

I'm afraid that I don't know enough about WSL to be able to help you get that going. However, for all the apparent complexity of the mq-container build, it all boils down to a couple of regular docker build commands. We have extra wrapper code to set up an HTTP server to host the MQ tar.gz file, but if you're happy doing that yourself, you could give this a go.

Firstly, start an HTTP server to host the tar.gz file. Here is a way of doing this with Python, which serves up the contents of the current directory on port 8000:

python -m SimpleHTTPServer 8000 .

Next, build the MQ Go SDK image, which is used to build the Go programs with the MQ client:

docker build -t mq-golang-sdk -f incubating/mq-golang-sdk/Dockerfile .

Next, get your TCP/IP address for the host you're running that HTTP server on (i.e. your Windows machine), then plug it into this command (or just set MY_IP_ADDRESS):

docker build \
  --tag mq \
  --file Dockerfile-server \
  --build-arg MQ_URL=http:https://${MY_IP_ADDRESS}:8000/downloads/mqadv_dev911_ubuntu_x86-64.tar.gz \
  --build-arg BASE_IMAGE=ubuntu:16.04 \
  --build-arg BUILDER_IMAGE=mq-golang-sdk \