dump1090 coredumps in amd64 image #79

wollew · 2021-04-12T08:11:56Z

I am trying to run the latest image in relay mode, but I am running into the following issue:

piaware               | [piaware] 2021/04/12 07:59:13 piaware version 5.0 is running, process ID 333
piaware               | [piaware] 2021/04/12 07:59:13 your system info is: Linux 0628bd400100 5.9.0-0.bpo.2-amd64 #1 SMP Debian 5.9.6-1~bpo10+1 (2020-11-19) x86_64 GNU/Linux
piaware               | ./run: line 48:   354 Illegal instruction     (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
piaware               |        355 Done                    | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'

My relevant part of docker-compose.yml:

piaware:
    #image: mikenye/piaware:latest_i386
    image: mikenye/piaware:latest
    tty: true
    container_name: piaware
    restart: always
    networks:
      - adsbnet
    environment:
      - TZ="Europe/Berlin"
      - LAT=xxx
      - LONG=xxx
      - RECEIVER_TYPE=relay
      - BEASTHOST=readsb
      - BEASTPORT=30005
      - FEEDER_ID=xxx
    tmpfs:
      - /run:exec,size=64M
      - /var/log

It works flawlessly if I change to the i386 image.

My system is

Linux 5.9.0-0.bpo.2-amd64 #1 SMP Debian 5.9.6-1~bpo10+1 (2020-11-19) x86_64 GNU/Linux

The text was updated successfully, but these errors were encountered:

mikenye · 2021-04-12T10:41:45Z

Hi @wollew, sorry you’re having problems.

Can you please try mikenye/piaware:latest_amd64?

Can you please also post the output of lscpu on your host?

Thanks.

wollew · 2021-04-12T11:39:04Z

Ok, I tried, unfortunately it doesn't make a difference:

$ docker-compose up piaware
Creating piaware ... done
Attaching to piaware
piaware               | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
piaware               | [s6-init] ensuring user provided files have correct perms...exited 0.
piaware               | [fix-attrs.d] applying ownership & permissions fixes...
piaware               | [fix-attrs.d] done.
piaware               | [cont-init.d] executing container initialization scripts...
piaware               | [cont-init.d] 01-piaware: executing...
piaware               | Set feeder-id to XXX in /etc/piaware.conf:1
piaware               | Set allow-auto-updates to no in /etc/piaware.conf:2
piaware               | Set allow-manual-updates to no in /etc/piaware.conf:3
piaware               | Set allow-mlat to yes in /etc/piaware.conf:4
piaware               | Set mlat-results to yes in /etc/piaware.conf:5
piaware               | Set receiver-type to relay in /etc/piaware.conf:6
piaware               | Set receiver-host to readsb in /etc/piaware.conf:7
piaware               | Set receiver-port to 30005 in /etc/piaware.conf:8
piaware               | [cont-init.d] 01-piaware: exited 0.
piaware               | [cont-init.d] done.
piaware               | [services.d] starting services
piaware               | [services.d] done.
piaware               | ./run: line 48:   325 Illegal instruction     (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
piaware               |        326 Done                    | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
piaware               | [skyaware] 2021/04/12 11:36:55 2021-04-12 11:36:55: (plugin.c.190) Cannot load plugin mod_setenv more than once, please fix your config (lighttpd may not accept such configs in future releases)
piaware               | [skyaware] 2021/04/12 11:36:55 2021-04-12 11:36:55: (server.c.1464) server started (lighttpd/1.4.53)
piaware               | [beast-splitter] 2021/04/12 11:36:55 127.0.0.1:30004: connection to 127.0.0.1:30004 failed: Connection refused
piaware               | [beast-splitter] 2021/04/12 11:36:55 127.0.0.1:30004: reconnecting in 60 seconds
piaware               | [beast-splitter] 2021/04/12 11:36:55 net(readsb:30005): connected to 172.19.0.3:30005
piaware               | [beast-splitter] 2021/04/12 11:36:55 net(readsb:30005): configured with settings: BCDfgIjk
piaware               | [piaware] 2021/04/12 11:36:55 ****************************************************
piaware               | [piaware] 2021/04/12 11:36:55 piaware version 5.0 is running, process ID 334
piaware               | [piaware] 2021/04/12 11:36:55 your system info is: Linux ad7660cc31d0 5.9.0-0.bpo.2-amd64 #1 SMP Debian 5.9.6-1~bpo10+1 (2020-11-19) x86_64 GNU/Linux
piaware               | ./run: line 48:   354 Illegal instruction     (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
piaware               |        355 Done                    | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
piaware               | [piaware] 2021/04/12 11:36:56 Connecting to FlightAware adept server at piaware.flightaware.com/1200
piaware               | [piaware] 2021/04/12 11:36:56 Connection with adept server at piaware.flightaware.com/1200 established
piaware               | ./run: line 48:   360 Illegal instruction     (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
piaware               |        361 Done                    | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
...


$ docker-compose images  | grep piaware
piaware                               mikenye/piaware           latest_amd64   926b05a42d64   346 MB

Here's my CPU info:

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               60
Model name:          Intel(R) Pentium(R) CPU G3220 @ 3.00GHz
Stepping:            3
CPU MHz:             2993.359
CPU max MHz:         3000.0000
CPU min MHz:         800.0000
BogoMIPS:            5986.72
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            3072K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt xsave rdrand lahf_lm abm cpuid_fault epb invpcid_single pti tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust erms invpcid xsaveopt dtherm arat pln pts

fabriziofiorucci · 2021-04-17T18:10:49Z

Hi, same problem here: it started misbehaving a few days ago after restarting and re-pulling the docker image. What I have in my logs is:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-piaware: executing...
Set feeder-id to XXXXXXXXXXXXXXXXXXXXXXXX in /etc/piaware.conf:1
Set allow-auto-updates to no in /etc/piaware.conf:2
Set allow-manual-updates to no in /etc/piaware.conf:3
Set allow-mlat to yes in /etc/piaware.conf:4
Set mlat-results to yes in /etc/piaware.conf:5
Set receiver-type to relay in /etc/piaware.conf:6
Set receiver-host to rpi-ads-b.lan.xyz in /etc/piaware.conf:7
Set receiver-port to 30005 in /etc/piaware.conf:8
[cont-init.d] 01-piaware: exited 0.
[cont-init.d] done.
[services.d] starting services
[beast-splitter] 2021/04/17 20:04:53 127.0.0.1:30004: connection to 127.0.0.1:30004 failed: Connection refused
[beast-splitter] 2021/04/17 20:04:53 127.0.0.1:30004: reconnecting in 60 seconds
[skyaware] 2021/04/17 20:04:53 2021-04-17 20:04:53: (plugin.c.190) Cannot load plugin mod_setenv more than once, please fix your config (lighttpd may not accept such configs in future releases)
[skyaware] 2021/04/17 20:04:53 2021-04-17 20:04:53: (server.c.1464) server started (lighttpd/1.4.53)
[beast-splitter] 2021/04/17 20:04:53 net(rpi-ads-b.lan.xyz:30005): connected to 192.168.1.28:30005
[beast-splitter] 2021/04/17 20:04:53 net(rpi-ads-b.lan.xyz:30005): configured with settings: BCDfgIjk
[services.d] done.
[piaware] 2021/04/17 20:04:53 ****************************************************
[piaware] 2021/04/17 20:04:53 piaware version 5.0 is running, process ID 334
[piaware] 2021/04/17 20:04:53 your system info is: Linux piaware-847cb5968b-zpp4b 4.15.0-141-generic #145-Ubuntu SMP Wed Mar 24 18:08:07 UTC 2021 x86_64 GNU/Linux
./run: line 48: 329 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
330 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 356 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
357 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
[piaware] 2021/04/17 20:04:54 Connecting to FlightAware adept server at piaware.flightaware.com/1200
[piaware] 2021/04/17 20:04:55 Connection with adept server at piaware.flightaware.com/1200 established
./run: line 48: 362 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
363 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
[piaware] 2021/04/17 20:04:55 TLS handshake with adept server at piaware.flightaware.com/1200 completed
[piaware] 2021/04/17 20:04:55 FlightAware server certificate validated
[piaware] 2021/04/17 20:04:55 encrypted session established with FlightAware
[piaware] 2021/04/17 20:04:55 adept reported location: XXXX, YYYY, ZZZft AMSL
[piaware] 2021/04/17 20:04:55 Receiver location changed, restarting dump1090 and skyaware978
[piaware] 2021/04/17 20:04:55 attempting to restart dump1090..
[piaware] 2021/04/17 20:04:55 invoke-rc.d: could not determine current runlevel
[piaware] 2021/04/17 20:04:55 invoke-rc.d: policy-rc.d denied execution of restart.
[piaware] 2021/04/17 20:04:55 can't restart dump1090, no services that look like dump1090 found
[piaware] 2021/04/17 20:04:55 attempting to restart skyaware978..
[piaware] 2021/04/17 20:04:55 can't restart skyaware978, no services that look like skyaware978 found
[piaware] 2021/04/17 20:04:55 logged in to FlightAware as user XXXXXXXX
[piaware] 2021/04/17 20:04:55 my feeder ID is XXXXXXXXXXXXXXXXXXXXXXXXXXXX
[piaware] 2021/04/17 20:04:55 site statistics URL: https://flightaware.com/adsb/stats/user/XXXXXXXXXXXXXXXXXXXXXX
[piaware] 2021/04/17 20:04:55 multilateration data requested
[piaware] 2021/04/17 20:04:55 no ADS-B data program is serving on port 30005, not starting multilateration client yet
[piaware] 2021/04/17 20:04:56 no ADS-B data program seen listening on port 30005 for 3 seconds, next check in 60s
[piaware] 2021/04/17 20:04:56 UAT support disabled by local configuration setting: uat-receiver-type
./run: line 48: 382 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
383 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
[beast-splitter] 2021/04/17 20:04:56 net(rpi-ads-b.lan.xyz:30005): connected to a Beast-style receiver
./run: line 48: 389 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
390 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 395 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
396 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 401 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
402 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 407 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
408 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 413 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
414 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 419 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
420 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
[statsurlupdater] Updating SkyAware link to: https://flightaware.com/adsb/stats/user/XXXXXXXXXXXXXXXXXXXXXX
./run: line 48: 437 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
438 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'
./run: line 48: 443 Illegal instruction (core dumped) "${DUMP1090_BIN}" "${DUMP1090_CMD[@]}" 2>&1
444 Done | stdbuf -o0 awk '{print "[dump1090] " strftime("%Y/%m/%d %H:%M:%S", systime()) " " $0}'

It's running on a Kubernetes cluster:

    NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
    k8s-master Ready control-plane,master 295d v1.20.4 192.168.1.40 Ubuntu 18.04.5 LTS 4.15.0-140-generic docker:https://20.10.5
    k8s-node1 Ready 295d v1.20.4 192.168.1.41 Ubuntu 18.04.5 LTS 4.15.0-141-generic docker:https://20.10.5
    k8s-node2 Ready 295d v1.20.4 192.168.1.42 Ubuntu 18.04.5 LTS 4.15.0-141-generic docker:https://20.10.5

My k8s is running with plenty of CPUs and RAM, all other containers are working as usual.

Images that fail this way are:

    image: mikenye/piaware
    image: mikenye/piaware:latest_amd64
    image: mikenye/piaware:latest_nohealthcheck

Any clue? Thank you.

mikenye · 2021-04-19T07:05:57Z

Could I please get you to try building the image on the destination system, to see if this still happens? This will tell me whether the problem is with compiler optimisation or not.

The steps to do this are as follows:

Clone the github repo

Inside your /opt/adsb directory, run the command git clone https://github.com/mikenye/docker-piaware.git.

Update your docker-compose.yml file

Change the image: mikenye/piaware line in your docker-compose.yml file to build: /opt/adsb/docker-piaware

Build the Image

Run the command docker-compose build in your /opt/adsb directory.

Start the Image

Run the command docker-compose up -d in your /opt/adsb directory.

This should start the container using the locally built image.

Can you let me know if this works as expected, or whether you still end up with the core dumps?

Thanks.

wollew · 2021-04-19T08:06:56Z

Building the image locally works for me.

mikenye · 2021-04-19T08:19:45Z

Thanks for letting me know. I'll take a look into the build process of dump1090 and see if I can reduce the optimisation.

wollew · 2021-04-19T08:21:30Z

Let me know if I can help you with anything.

mikenye · 2021-04-19T08:33:47Z

@wollew any chance you could get me the core dump file from the broken version of the container?

wollew · 2021-04-19T09:51:50Z

Yes, I just extracted a core file, I'd prefer to not upload it here, I'll contact you via Discord.

mikenye · 2021-05-21T03:28:21Z

Hi @wollew, I just wanted to check in to see if you're still having issues with this container. It's been a month or so since we last spoke on Discord, and I haven't had much time to spend looking into this due to my day job. If still a problem, I'll spend some time on it. If not, cool. :)

wollew · 2021-05-22T09:18:43Z

Does the latest image work for me? No. Is it a problem for me? Also no, I'll just use the i386 image :-)

mikenye · 2021-05-24T01:32:03Z

Awesome, thanks <3

luciodaou · 2021-05-28T03:21:11Z

I'm having the exact same issue, also with a amd64 processor. The tags amd64, arm-v7, arm-v6 didn't help. Thanks for the tip of using i386. I was using the v4.0 tag to make it work.

tezhrec · 2021-10-28T22:59:48Z

FYI - I've been running your stack with FR24 and ADSBx for months now without issue. Decided to add Flightaware to the stack and ran into this issue. The AMD64 image did not work, however building locally is working perfectly. Glad I found this thread and thank you much for the awesome work!

mikenye · 2022-01-18T07:08:59Z

I deployed a version of the container that built dump1090 on first run. Upon reflection (and some discourse in Discord), this was not the correct approach and I will roll back.

In the meantime, dump1090 has been updated to version 7.1.

The updated container should be available in the next few hours. I'll post back then, and ask that anyone that had this issue try testing with the new container with pre-built dump1090 v7.1.

Thanks and apologies to anyone that's had trouble due to the change.

mikenye · 2022-01-19T00:57:04Z

@wollew, @fabriziofiorucci, @luciodaou, @tezhrec: Could I please trouble you to test with the pre-built image mikenye/piaware:latest? Please be sure to pull the container to ensure you get the updated version. Please let me know if you're still getting segfaults. Thanks.

wollew · 2022-01-19T07:59:58Z

The local tag doesn't exist. If you meant latest, I pulled that and it works without core dumps.

EDIT: I should've looked here instead of just the email notification and then would've seen that you already edited your comment. So to me, this looks good now.

mikenye · 2022-01-19T09:39:26Z

Yep sorry, typo'd!

Thanks for letting me know. Hopefully it is also resolved for the others too.

mikenye · 2022-01-21T02:19:58Z

Closing this, can re-open if anyone else has issues.

luciodaou · 2022-01-22T01:02:33Z

@wollew, @fabriziofiorucci, @luciodaou, @tezhrec: Could I please trouble you to test with the pre-built image mikenye/piaware:latest? Please be sure to pull the container to ensure you get the updated version. Please let me know if you're still getting segfaults. Thanks.

Great. Will check on the computer probably on early Monday and then give you feedback.

mikenye · 2022-01-24T01:42:49Z

Thanks @luciodaou, let me know how you go.

mikenye self-assigned this Apr 12, 2021

mikenye closed this as completed May 24, 2021

mikenye reopened this May 28, 2021

mikenye mentioned this issue Jan 18, 2022

Build dump1090 on firstrun #100

Merged

mikenye closed this as completed in #100 Jan 18, 2022

mikenye mentioned this issue Jan 18, 2022

Revert to pre-built dump1090 #101

Merged

mikenye reopened this Jan 18, 2022

mikenye closed this as completed Jan 21, 2022

mikenye mentioned this issue Apr 1, 2022

Docker container does not start dump1090 after upgrade to 7.2 #123

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dump1090 coredumps in amd64 image #79

dump1090 coredumps in amd64 image #79

wollew commented Apr 12, 2021 •

edited

Loading

mikenye commented Apr 12, 2021

wollew commented Apr 12, 2021

fabriziofiorucci commented Apr 17, 2021 •

edited by mikenye

Loading

mikenye commented Apr 19, 2021

wollew commented Apr 19, 2021

mikenye commented Apr 19, 2021

wollew commented Apr 19, 2021

mikenye commented Apr 19, 2021

wollew commented Apr 19, 2021 •

edited

Loading

mikenye commented May 21, 2021

wollew commented May 22, 2021

mikenye commented May 24, 2021

luciodaou commented May 28, 2021

tezhrec commented Oct 28, 2021

mikenye commented Jan 18, 2022

mikenye commented Jan 19, 2022 •

edited

Loading

wollew commented Jan 19, 2022 •

edited

Loading

mikenye commented Jan 19, 2022

mikenye commented Jan 21, 2022

luciodaou commented Jan 22, 2022

mikenye commented Jan 24, 2022

dump1090 coredumps in amd64 image #79

dump1090 coredumps in amd64 image #79

Comments

wollew commented Apr 12, 2021 • edited Loading

mikenye commented Apr 12, 2021

wollew commented Apr 12, 2021

fabriziofiorucci commented Apr 17, 2021 • edited by mikenye Loading

mikenye commented Apr 19, 2021

wollew commented Apr 19, 2021

mikenye commented Apr 19, 2021

wollew commented Apr 19, 2021

mikenye commented Apr 19, 2021

wollew commented Apr 19, 2021 • edited Loading

mikenye commented May 21, 2021

wollew commented May 22, 2021

mikenye commented May 24, 2021

luciodaou commented May 28, 2021

tezhrec commented Oct 28, 2021

mikenye commented Jan 18, 2022

mikenye commented Jan 19, 2022 • edited Loading

wollew commented Jan 19, 2022 • edited Loading

mikenye commented Jan 19, 2022

mikenye commented Jan 21, 2022

luciodaou commented Jan 22, 2022

mikenye commented Jan 24, 2022

wollew commented Apr 12, 2021 •

edited

Loading

fabriziofiorucci commented Apr 17, 2021 •

edited by mikenye

Loading

wollew commented Apr 19, 2021 •

edited

Loading

mikenye commented Jan 19, 2022 •

edited

Loading

wollew commented Jan 19, 2022 •

edited

Loading