Garbage interfaces and port when comparing to local interfaces

[Galati1337]

I'm using Ubuntu 18.04
OpenSSL 1.1.1g
mosquitto v1.6.9
libwebsockets 4.1.0

This is my configuration:
listener 9001
protocol websockets
log_type all
websockets_log_level 1023
connection_messages true
bind_interface eno1

This is what i get:

1600638792: mosquitto version 1.6.9 starting
1600638792: Config loaded from /etc/mosquitto/mosquitto.conf.
1600638792: Opening websockets listen socket on port 9001.
1600638792: Initial logging level 1023
1600638792: Libwebsockets version: 2.0.3 unknown-build-hash
1600638792: IPV6 not compiled in
1600638792: libev support compiled in but disabled
1600638792: libuv support compiled in but disabled
1600638792: LWS_DEF_HEADER_LEN : 1024
1600638792: LWS_MAX_PROTOCOLS : 5
1600638792: LWS_MAX_SMP : 32
1600638792: SPEC_LATEST_SUPPORTED : 13
1600638792: sizeof (*info) : 280
1600638792: SYSTEM_RANDOM_FILEPATH: '/dev/urandom'
1600638792: default timeout (secs): 20
1600638792: Threads: 1 each 1024 fds
1600638792: mem: context: 71488 bytes (67392 ctx + (1 thr x 4096))
1600638792: mem: http hdr rsvd: 68224 bytes (1 thr x (1024 + 3240) x 16))
1600638792: mem: pollfd map: 8192
1600638792: mem: platform fd map: 8192 bytes
1600638792: Compiled with OpenSSL support
1600638792: SSL disabled: no LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT
1600638792: Creating Vhost 'default' port 0, 1 protocols
1600638792: interface lo vs B?_U
1600638792: interface eno1 vs B?_U
1600638792: interface wlp58s0 vs B?_U
1600638792: interface lo vs B?_U
1600638792: interface eno1 vs B?_U
1600638792: interface lo vs B?_U
1600638792: interface eno1 vs B?_U
1600638792: Unable to find interface B?_U
1600638792: Failed to create default vhost
1600638792: Error: Unable to create websockets listener on port 9001.

Keep in mind that i get various random data when comparing eno1 vs XXX and so on.
I just want to create a basic libwebsocket server using mosquitto.
I've initially opened a ticket on libwebsockets side, but it seems is ok on their side: warmcat/libwebsockets#2060

Thanks.

[karlp]

Regardless of what LWS says, does it actually work with an older LWS? 4.1.0 only just came out, and 4.0.x was broken on many target architectures. does LWS 3.1 work? or 2.4? Those are known to be functional LWS releases.

[Galati1337]

Using LWS 3.1.0 and mosquitto 1.6.7 ( or 1.4.10 ) i get this:

Starting program: /usr/local/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
1600680838: mosquitto version 1.6.7 starting
1600680838: Config loaded from /etc/mosquitto/mosquitto.conf.
1600680838: Opening websockets listen socket on port 9001.
1600680838: Initial logging level 1023
1600680838: Libwebsockets version: 2.0.3 unknown-build-hash
1600680838: IPV6 not compiled in
1600680838: libev support compiled in but disabled
1600680838: libuv support compiled in but disabled
1600680838: LWS_DEF_HEADER_LEN : 1024
1600680838: LWS_MAX_PROTOCOLS : 5
1600680838: LWS_MAX_SMP : 32
1600680838: SPEC_LATEST_SUPPORTED : 13
1600680838: sizeof (*info) : 280
1600680838: SYSTEM_RANDOM_FILEPATH: '/dev/urandom'
1600680838: default timeout (secs): 20
1600680838: Threads: 1 each 1024 fds
1600680838: mem: context: 71488 bytes (67392 ctx + (1 thr x 4096))
1600680838: mem: http hdr rsvd: 68224 bytes (1 thr x (1024 + 3240) x 16))
1600680838: mem: pollfd map: 8192
1600680838: mem: platform fd map: 8192 bytes
1600680838: Compiled with OpenSSL support
1600680838: Creating Vhost 'default' port 9001, 2 protocols
1600680838: Using non-SSL mode
1600680838: interface lo vs eno1
1600680838: interface eno1 vs eno1
1600680838: interface wlp58s0 vs eno1
1600680838: interface lo vs eno1
1600680838: interface eno1 vs eno1
1600680838: insert_wsi_socket_into_fds: 0x5555557dd600: tsi=0, sock=7, pos-in-fds=1
1600680838: Listening on port 9001
1600680838: LWS_MAX_EXTENSIONS_ACTIVE: 2
1600680838: mem: per-conn: 920 bytes + protocol rx buf
1600680838: canonical_hostname = blabla.com
1600680838: lws_protocol_init

Program received signal SIGSEGV, Segmentation fault.
0x0000555555583841 in ?? ()
(gdb) bt
#0 0x0000555555583841 in ?? ()
#1 0x00007ffff6c96a8f in ?? () from /usr/lib/x86_64-linux-gnu/libwebsockets.so.8
#2 0x00007ffff6c91ed5 in lws_service_fd_tsi () from /usr/lib/x86_64-linux-gnu/libwebsockets.so.8
#3 0x00007ffff6c9d6a3 in ?? () from /usr/lib/x86_64-linux-gnu/libwebsockets.so.8
#4 0x0000555555569d33 in mosquitto_main_loop (db=0x5555557902a0 <int_db>, listensock=0x0, listensock_count=0) at loop.c:623
#5 0x0000555555559b84 in main (argc=, argv=) at mosquitto.c:367

[ralight]

libwebsockets8 is 2.0.0 to 2.0.3. It looks like you have that version installed in your OS and it is getting picked up before 3.2.0. The earliest good version was about 2.4.

libwebsockets is very unstable from version to version with frequent breaking changes in behaviour, API, and ABI. Even a change in bugfix versions can introduce significant incompatibilities, e.g. from 3.0.0 to 3.0.1, so it is important that you use only the exact binary that you compiled against.

[Galati1337]

Yup. I've removed ubuntu's libws-dev and ran ldconfig after.
Now i'm trying with mosquitto 1.6.7 and lws 2.4.0

The app remains like this:
/localdisk/aoana/mosquitto/mosquitto$ sudo /usr/local/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf
1600685238: mosquitto version 1.6.7 starting
1600685238: Config loaded from /etc/mosquitto/mosquitto.conf.
1600685238: Opening websockets listen socket on port 9001.
1600685238: libuv support not compiled in
1600685238: Creating Vhost 'default' port 9001, 3 protocols, IPv6 off
1600685238: Using non-SSL mode
1600685238: created client ssl context for default

The output of:
sudo lsof | grep 9001
Output information may be incomplete.
mosquitto 26441 mosquitto 7u IPv4 26756180 0t0 TCP *:9001 (LISTEN)

I've added
http_dir /var/www/html with chmod 777

I've tried to access from my browser: localhost:9001 but nothing.

Any suggestions ?

[ralight]

What does "nothing" mean? It doesn't respond, a 404, or something else?

Try using an mqtt websockets client to connect to that port and subscribe to $SYS/#.

[Galati1337]

I'm getting this:

1600687733:
1600687733: 0000: 10 23 00 04 4D 51 54 54 04 02 00 3C 00 17 6D 6F .#..MQTT...<..mo
1600687733: 0010: 73 71 2D 31 73 46 74 73 78 4D 43 37 4D 6C 61 69 sq-1sFtsxMC7Mlai
1600687733: 0020: 32 65 52 54 36 2eRT6
1600687733:
1600687733:
1600687733: 0000: 10 23 00 04 4D 51 54 54 04 02 00 3C 00 17 6D 6F .#..MQTT...<..mo
1600687733: 0010: 73 71 2D 31 73 46 74 73 78 4D 43 37 4D 6C 61 69 sq-1sFtsxMC7Mlai
1600687733: 0020: 32 65 52 54 36 2eRT6
1600687733:
1600687733: forbidding on uri sanitation
1600687733: lws_return_http_status: return
1600687733: error on reading from skt : 104

When running this:
mosquitto_sub -v -p 9001 -t '$SYS/#'

From the web it actually didn't responded.

[ralight]

mosquitto_sub doesn't support websockets, you'll have to use something else. MQTT Explorer is worth a look.

[Galati1337]

Is it normal for a second process to be spawned after i start mine manually as sudo ?

root 4681 0.0 0.0 66396 4460 pts/3 S+ 18:18 0:00 sudo /usr/local/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf
mosquit+ 4682 0.0 0.0 35996 6360 pts/3 S+ 18:18 0:00 /usr/local/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf

[ralight]

The first command is your sudo call, it isn't the mosquitto broker itself.

[mrdc]

@ralight

The earliest good version was about 2.4.

Should we use this version?
I've tried libwebsockets-4.2.0 and there is the same issue: Unable to create websockets listener on port 9001.
Can't make mosquitto work with websockets for several days.

In mosquitto log:

2021-04-26|19:10:52: Opening websockets listen socket on port 9001.
2021-04-26|19:10:52: Initial logging level 255
2021-04-26|19:10:52: Libwebsockets version: 2.0.3 unknown-build-hash
2021-04-26|19:10:52: IPV6 not compiled in
2021-04-26|19:10:52: libev support compiled in but disabled
2021-04-26|19:10:52: libuv support compiled in but disabled
2021-04-26|19:10:52:  LWS_DEF_HEADER_LEN    : 1024
2021-04-26|19:10:52:  LWS_MAX_PROTOCOLS     : 5
2021-04-26|19:10:52:  LWS_MAX_SMP           : 32
2021-04-26|19:10:52:  SPEC_LATEST_SUPPORTED : 13
2021-04-26|19:10:52:  sizeof (*info)        : 160
2021-04-26|19:10:52:  SYSTEM_RANDOM_FILEPATH: '/dev/urandom'
2021-04-26|19:10:52:  default timeout (secs): 20
2021-04-26|19:10:52:  Threads: 1 each 1024 fds
2021-04-26|19:10:52:  mem: context:         40456 bytes (36360 ctx + (1 thr x 4096))
2021-04-26|19:10:52:  mem: http hdr rsvd:   67968 bytes (1 thr x (1024 + 3224) x 16))
2021-04-26|19:10:52:  mem: pollfd map:       8192
2021-04-26|19:10:52:  mem: platform fd map:  4096 bytes
2021-04-26|19:10:52:  Compiled with OpenSSL support
2021-04-26|19:10:52:  SSL disabled: no LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT
2021-04-26|19:10:52: Creating Vhost 'default' port 0, 1 protocols
2021-04-26|19:10:52:  interface lo vs  f^D
2021-04-26|19:10:52:  interface eth0 vs  f^D
2021-04-26|19:10:52:  interface wlan0 vs  f^D
2021-04-26|19:10:52:  interface docker0 vs  f^D
2021-04-26|19:10:52:  interface vethd5d0318 vs  f^D
2021-04-26|19:10:52:  interface vethdcc72fc vs  f^D
2021-04-26|19:10:52:  interface lo vs  f^D
2021-04-26|19:10:52:  interface eth0 vs  f^D
2021-04-26|19:10:52:  interface tun0 vs  f^D
2021-04-26|19:10:52:  interface docker0 vs  f^D
2021-04-26|19:10:52:  interface vethd5d0318 vs  f^D
2021-04-26|19:10:52:  interface vethdcc72fc vs  f^D
2021-04-26|19:10:52:  interface lo vs  f^D
2021-04-26|19:10:52:  interface eth0 vs  f^D
2021-04-26|19:10:52:  interface tun0 vs  f^D
2021-04-26|19:10:52:  interface docker0 vs  f^D
2021-04-26|19:10:52:  interface vethd5d0318 vs  f^D
2021-04-26|19:10:52:  interface vethdcc72fc vs  f^D
2021-04-26|19:10:52: Unable to find interface  f^D
2021-04-26|19:10:52: Failed to create default vhost
2021-04-26|19:10:52: Error: Unable to create websockets listener on port 9001.

Debian Buster
mosquitto 2.0.10
libwebsockets-4.2.0
libwebsockets-dev/stable,now 2.0.3-3 armhf [installed] (the latest available in official repo)

I've recompiled mosquitto 2.0.10 using libwebsockets-2.4.2 as you suggested here: #2060 (comment)
And mosquitto process just dies with Illegal instruction when I include

listener 9001
protocol websockets
socket_domain ipv4

With libwebsockets-4.2.0 mosquitto exited with this error: Unable to create websockets listener on port 9001.

Using libwebsockets-2.4.2

2021-04-26|19:26:57: Initial logging level 255
2021-04-26|19:26:57: Libwebsockets version: 2.0.3 unknown-build-hash
2021-04-26|19:26:57: IPV6 not compiled in
2021-04-26|19:26:57: libev support compiled in but disabled
2021-04-26|19:26:57: libuv support compiled in but disabled
2021-04-26|19:26:57:  LWS_DEF_HEADER_LEN    : 1024
2021-04-26|19:26:57:  LWS_MAX_PROTOCOLS     : 5
2021-04-26|19:26:57:  LWS_MAX_SMP           : 32
2021-04-26|19:26:57:  SPEC_LATEST_SUPPORTED : 13
2021-04-26|19:26:57:  sizeof (*info)        : 160
2021-04-26|19:26:57:  SYSTEM_RANDOM_FILEPATH: '/dev/urandom'
2021-04-26|19:26:57:  default timeout (secs): 20
2021-04-26|19:26:57:  Threads: 1 each 1024 fds
2021-04-26|19:26:57:  mem: context:         40456 bytes (36360 ctx + (1 thr x 4096))
2021-04-26|19:26:57:  mem: http hdr rsvd:   67968 bytes (1 thr x (1024 + 3224) x 16))
2021-04-26|19:26:57:  mem: pollfd map:       8192
2021-04-26|19:26:57:  mem: platform fd map:  4096 bytes
2021-04-26|19:26:57:  Compiled with OpenSSL support
2021-04-26|19:26:57: Creating Vhost 'default' port 9001, 2 protocols
2021-04-26|19:26:57:  Using non-SSL mode
2021-04-26|19:26:57: insert_wsi_socket_into_fds: 0x1847070: tsi=0, sock=11, pos-in-fds=1
2021-04-26|19:26:57:  Listening on port 9001
2021-04-26|19:26:57:  LWS_MAX_EXTENSIONS_ACTIVE: 2
2021-04-26|19:26:57:  mem: per-conn:          600 bytes + protocol rx buf
2021-04-26|19:26:57:  canonical_hostname = raspberrypi
2021-04-26|19:26:57: Opening ipv4 listen socket on port 1883.
2021-04-26|19:26:57: Opening ipv6 listen socket on port 1883.
2021-04-26|19:26:57: mosquitto version 2.0.10 running
2021-04-26|19:26:57: lws_protocol_init

[ralight]

@mrdc have you tried the packages from repo.mosquitto.org?

[mrdc]

have you tried the packages from repo.mosquitto.org?

@ralight Not yet - I've read somewhere that pre-build binaries are without websockets...

[ralight]

Those packages have websockets support.

[mrdc]

@ralight

Version from repo.mosquitto.org works fine: no issues with websockets. Not clear what can be wrong with compiling.
I suspect some issues caused by outdated libwebsockets: I've had 2021-04-26|19:26:57: Libwebsockets version: 2.0.3 unknown-build-hash
Updating mosquitto from repo.mosquitto.org also updated libwebsockets to libwebsockets 2.4.2-0mosquitto1~buster1

[ralight]

Great, I'm glad that worked. I'm closing this issue because it's solved for you and the original reporter was using a much older version. If it's still a problem elsewhere, please reopen and add more details.