logspout build.sh not respecting proxy environment variables #465
Comments
Loriden
changed the title
|
@Loriden the problem is not the script itself (imagine if every script or application had to be configured manually when running in containers!), you have to configure your Docker client to pass those variables automatically for you: https://docs.docker.com/network/proxy/ {
"proxies":
{
"default":
{
"httpProxy": "http:https://proxy.example.com:3001",
"httpsProxy": "http:https://proxy.example.com:3001",
"noProxy": "*.test.example.com,.example2.com"
}
}
} |
@antoineco You absolutely do, I have configured Docker to do this and it works for my other hosted containers. I suspect this is related to the go program requiring a bash profile which Docker may or may not be configuring. I think this issue can be closed out with some documentation referencing an updating of bash.sh. |
|
Alright let me find out the root cause. Could you share at what stage the script fails? Already at the |
|
It was failing the image pull from alpinelinux.org due to my corporate proxy. I have CNTLM set up on another ubuntu box and i was monitoring its logs to see what the issue was (we have service accounts with highly restricted filtering for this very use case - and initially i put it down to this) but then noted that the requests weren't even appearing in the CNTLM log. See below output for the docker-compose command: I'll have to take back my go comment, it never even got that far - first time i encountered go was yesterday and i investigated as a link the chain. To be fair i think this is more a logspout issue than docker-elk, but i encountered it while spinning up a proof of concept for the rest of my team. I can replicate this issue by deleting the logspout container and the export lines and rerunning compose, so if you need more please ask. |
|
It might be worth adding the following command at the beginning of the script to troubleshoot whether Docker actually configured the proxy variables when it created the container: |
|
@Loriden I just ran the experiment above, and the build script ran flawlessly behind my proxy. Let's compare our results. 📄 {
"auths": {
"https://index.docker.io/v1/": {
"auth": "<redacted>",
"email": "<redacted>"
}
},
"proxies":
{
"default":
{
"httpProxy": "http:https://proxy.example.com:3001",
"httpsProxy": "http:https://proxy.example.com:3001",
"noProxy": "*.test.example.com,.example2.com"
}
}
}📄 diff --git a/extensions/logspout/build.sh b/extensions/logspout/build.sh
index eef0822..4f27a82 100755
--- a/extensions/logspout/build.sh
+++ b/extensions/logspout/build.sh
@@ -3,6 +3,8 @@
# unmodified from:
# https://github.com/gliderlabs/logspout/blob/67ee3831cbd0594361bb3381380c65bdbeb3c20f/custom/build.sh
+env
+
set -e
apk add --update go git mercurial build-base
mkdir -p /go/src/github.com/gliderlabs$ docker-compose -f docker-compose.yml -f extensions/logspout/logspout-compose.yml up
Building logspout
Step 1/2 : FROM gliderlabs/logspout:master
\# Executing 3 build triggers
---> Using cache
---> Using cache
---> Running in 95cd563707cc
# NOTE: env printed below. Notice the *_PROXY env vars
HTTPS_PROXY=http:https://proxy.example.com:3001
no_proxy=*.test.example.com,.example2.com
HOSTNAME=95cd563707cc
SHLVL=2
HOME=/root
OLDPWD=/
NO_PROXY=*.test.example.com,.example2.com
https_proxy=http:https://proxy.example.com:3001
http_proxy=http:https://proxy.example.com:3001
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/src
HTTP_PROXY=http:https://proxy.example.com:3001
# NOTE: rest of the logs below
fetch http:https://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http:https://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/34) Installing binutils (2.32-r0)
(2/34) Installing libmagic (5.37-r1)
(3/34) Installing file (5.37-r1)
(4/34) Installing gmp (6.1.2-r1)
(5/34) Installing isl (0.18-r0)
(6/34) Installing libgomp (8.3.0-r0)
(7/34) Installing libatomic (8.3.0-r0)
(8/34) Installing libgcc (8.3.0-r0)
(9/34) Installing mpfr3 (3.1.5-r1)
(10/34) Installing mpc1 (1.1.0-r0)
(11/34) Installing libstdc++ (8.3.0-r0)
(12/34) Installing gcc (8.3.0-r0)
(13/34) Installing musl-dev (1.1.22-r3)
(14/34) Installing libc-dev (0.7.1-r0)
(15/34) Installing g++ (8.3.0-r0)
(16/34) Installing make (4.2.1-r2)
(17/34) Installing fortify-headers (1.1-r0)
(18/34) Installing build-base (0.5-r1)
(19/34) Installing nghttp2-libs (1.39.2-r0)
(20/34) Installing libcurl (7.66.0-r0) (21/34) Installing expat (2.2.8-r0)
(22/34) Installing pcre2 (10.33-r0)
(23/34) Installing git (2.22.2-r0)
(24/34) Installing go (1.12.12-r0)
(25/34) Installing libbz2 (1.0.6-r7)
(26/34) Installing libffi (3.2.1-r6)
(27/34) Installing gdbm (1.13-r1)
(28/34) Installing ncurses-terminfo-base (6.1_p20190518-r0)
(29/34) Installing ncurses-terminfo (6.1_p20190518-r0) (30/34) Installing ncurses-libs (6.1_p20190518-r0)
(31/34) Installing readline (8.0.0-r0)
(32/34) Installing sqlite-libs (3.28.0-r2)
(33/34) Installing python2 (2.7.16-r2)
(34/34) Installing mercurial (4.9.1-r0) Executing busybox-1.30.1-r2.trigger
OK: 531 MiB in 49 packages go get: warning: modules disabled by GO111MODULE=auto in GOPATH/src;
ignoring go.mod;
see 'go help modules'
(1/34) Purging build-base (0.5-r1)
(2/34) Purging file (5.37-r1)
(3/34) Purging g++ (8.3.0-r0)
(4/34) Purging make (4.2.1-r2)
(5/34) Purging libc-dev (0.7.1-r0)
(6/34) Purging musl-dev (1.1.22-r3) (7/34) Purging fortify-headers (1.1-r0) (8/34) Purging git (2.22.2-r0)
(9/34) Purging go (1.12.12-r0) (10/34) Purging gcc (8.3.0-r0)
(11/34) Purging binutils (2.32-r0) (12/34) Purging libatomic (8.3.0-r0)
(13/34) Purging libgomp (8.3.0-r0) (14/34) Purging mercurial (4.9.1-r0) (15/34) Purging libmagic (5.37-r1)
(16/34) Purging mpc1 (1.1.0-r0) (17/34) Purging mpfr3 (3.1.5-r1)
(18/34) Purging isl (0.18-r0) (19/34) Purging gmp (6.1.2-r1) (20/34) Purging libstdc++ (8.3.0-r0) (21/34) Purging libgcc (8.3.0-r0) (22/34) Purging libcurl (7.66.0-r0) (23/34) Purging nghttp2-libs (1.39.2-r0) (24/34) Purging python2 (2.7.16-r2) (25/34) Purging expat (2.2.8-r0)
(26/34) Purging pcre2 (10.33-r0)
(27/34) Purging libbz2 (1.0.6-r7)
(28/34) Purging libffi (3.2.1-r6)
(29/34) Purging gdbm (1.13-r1)
(30/34) Purging readline (8.0.0-r0)
(31/34) Purging ncurses-libs (6.1_p20190518-r0)
(32/34) Purging ncurses-terminfo (6.1_p20190518-r0)
(33/34) Purging ncurses-terminfo-base (6.1_p20190518-r0)
(34/34) Purging sqlite-libs (3.28.0-r2)
Executing busybox-1.30.1-r2.trigger
OK: 6 MiB in 15 packages
Removing intermediate container 95cd563707cc
---> 171c3a7da29e
Step 2/2 : ENV SYSLOG_FORMAT rfc3164
---> Running in 7f81eaf217c8
Removing intermediate container 7f81eaf217c8
---> 29368a6777d7
Successfully built 29368a6777d7
Successfully tagged docker-elk_logspout:latest
WARNING: Image for service logspout was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Recreating docker-elk_elasticsearch_1 ... done
Recreating docker-elk_kibana_1 ... done
Recreating docker-elk_logstash_1 ... done
Creating docker-elk_logspout_1 ... done
Attaching to docker-elk_elasticsearch_1, docker-elk_logstash_1, docker-elk_kibana_1, docker-elk_logspout_1
logspout_1 | 2020/02/11 13:54:41 # logspout v3.2.6-custom by gliderlabs
logspout_1 | 2020/02/11 13:54:41 # adapters: logstash raw udp tcp
logspout_1 | 2020/02/11 13:54:41 # options :
logspout_1 | 2020/02/11 13:54:41 persist:/mnt/routes
logspout_1 | 2020/02/11 13:54:41 # jobs : http[]:80 pump routes
[...] |
|
Hi @antoineco - no worries, see mine below. /.docker/config.json build.sh Output from docker-compose -f docker-compose.yml -f extensions/logspout/logspout-compose.yml up #here is the env spit out #back to output It is interesting that we have different results. The one thing i can suggest is that i have an authenticated NTLM proxy that is supposedly auto detected, but never does, with settings pushed out by DHCP option 252. Which is why i've configured a CTNLM middleman to work with all my package managers and various other services that aren't proxy-aware or NTLM compatible. |
|
Thanks for double checking. As you can see, Docker seems to ignore your proxy configuration altogether, it doesn't export any I can't tell what is wrong by just looking at your configuration, but I have 2 theories:
If none of these are the culprits, I can only recommend to retry on a fresh installation, with default Docker settings, or to search/ask for a solution in the Docker forums. |
antoineco
removed
the
help wanted
Hi @antoineco - suggestion #2 is a copy and paste error, its in the config file, and suggestion #1 is in the correct location (i accessed it via relative path). Here's docker info confirming the proxies are importing Id have to spin up a new docker box to test that as we're running production containers in the current one, so thats probably gonna be shelved for a while. I think if we can add a caveat to the readme or wiki for this scenario to help future google'rs, that'd be sufficient to close this out. |
|
Could you please redo the same experiment with a manual
I'm not against adding caveats for things related to the software we run in the ELK stack, but when it comes to Docker we need to know where to draw the line to ensure this repo doesn't get out of control and become a Docker FAQ. To document a caveat, we first need to understand what specific software version(s) the problem occurs with, and how it can be reproduced. |
|
@Loriden did you find the time to run |
|
@Loriden I managed to reproduce the issue on a fresh machine with the following software versions: $ docker -v
Docker version 19.03.3, build a872fc2f86$ docker-compose -v
docker-compose version 1.17.1, build 6d101fb$ lsb_release -d
Description: Ubuntu 19.10
✅ Does pass *_PROXY env vars $ docker build ./extensions/logspout/
Sending build context to Docker daemon 7.168kB
Step 1/2 : FROM gliderlabs/logspout:master
# Executing 3 build triggers
---> Running in b29c63523ab5
HTTPS_PROXY=http:https://proxy.example.com:3001
NO_PROXY=*.test.example.com,.example2.com
HTTP_PROXY=http:https://proxy.example.com:3001
fetch http:https://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
# ...
❌ Does not pass *_PROXY env vars $ docker-compose -f docker-compose.yml -f extensions/logspout/logspout-compose.yml build logspout
Building logspout
Step 1/2 : FROM gliderlabs/logspout:master
# Executing 3 build triggers
---> Using cache
---> Using cache
---> Running in 8074a02ee35f
fetch http:https://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
# ...I looked at Compose's changelog and found the following entry for v1.20.0
I'm gonna go ahead and close this issue by bumping the minimum supported Compose version. Thanks for catching this! 🙌 |
Versions below 1.20.0 do not pass proxy settings to containers created by Compose. Therefore, we discourage users from using those versions. Closes #465
Versions below 1.20.0 do not pass proxy settings to containers created by Compose. Therefore, we discourage users from using those versions. Closes #465
Versions below 1.20.0 do not pass proxy settings to containers created by Compose. Therefore, we discourage users from using those versions. Closes deviantony#465
Problem description
running the logspout extension fails due to the build.sh script not using docker/docker host environment variables
docker-compose -f docker-compose.yml -f extensions/logspout/logspout-compose.yml up -d
Can be resolved by adding proxy variables at the top of build.sh, like so
Extra information
Stack configuration
Default, except logstash running on tcp:5001
Docker setup
Default
The text was updated successfully, but these errors were encountered: