Hang on DISCONNECT with tlsv1.{1,2}

[titouanc]

Short description

When using mosquitto as client library (on an embedded device) with a TLSv1.1 or TLSv1.2 connection, the call to mosquitto_disconnect(mosq) never returns, and the program uses lots of CPU. The problem does not occur with TLSv1.

Details

Using strace, I observe the following message being continuously repeated until the program is killed:

futex(0x24418, 0x80 /* FUTEX_??? */, 2) = -1 EAGAIN (Resource temporarily unavailable)

I have traced down the problem to the call to SSL_write for the DISCONNECT packet https://github.com/eclipse/mosquitto/blob/master/lib/net_mosq.c#L820 which never returns. Using the following modification:

@@ -819,7 +826,9 @@ ssize_t _mosquitto_net_write(struct mosquitto *mosq, void *buf, size_t count)
 #ifdef WITH_TLS
        if(mosq->ssl){
                mosq->want_write = false;
+               printf("@@@ SSL_write(%d) ... >> %p\n", (int) count, mosq->ssl);
                ret = SSL_write(mosq->ssl, buf, count);
+               printf("@@@ SSL_write(%d) -> %d @@@\n", (int) count, ret);
                if(ret < 0){
                        err = SSL_get_error(mosq->ssl, ret);
                        if(err == SSL_ERROR_WANT_READ){
@@ -880,8 +889,10 @@ int _mosquitto_packet_write(struct mosquitto *mosq)
 
        while(mosq->current_out_packet){
                packet = mosq->current_out_packet;
+               printf(">>>> ... PKT cmd=%02X pos=%d <<<<\n", packet->command, packet->pos);
 
                while(packet->to_process > 0){
+                       printf("--- Gonna write %ld bytes\n", packet->to_process);
                        write_length = _mosquitto_net_write(mosq, &(packet->payload[packet->pos]), packet->to_process);
                        if(write_length > 0){
 #if defined(WITH_BROKER) && defined(WITH_SYS_TREE)
@@ -889,6 +900,7 @@ int _mosquitto_packet_write(struct mosquitto *mosq)
 #endif
                                packet->to_process -= write_length;
                                packet->pos += write_length;
+                               printf("--- packet write ... %ld bytes\n", packet->pos);
                        }else{
 #ifdef WIN32
                                errno = WSAGetLastError();
@@ -908,6 +920,8 @@ int _mosquitto_packet_write(struct mosquitto *mosq)
                        }
                }
 
+               printf(">>>> OK PKT cmd=%02X <<<<\n", packet->command);
+
 #ifdef WITH_BROKER
 #  ifdef WITH_SYS_TREE
                g_msgs_sent++;

Using TLSv1

>>>> ... PKT cmd=E0 pos=0 <<<<
--- Gonna write 2 bytes
@@@ SSL_write(2) ... >> 0x28028
@@@ SSL_write(2) -> 2 @@@
--- packet write ... 2 bytes
>>>> OK PKT cmd=E0 <<<<

Using TLSv1.1 or TLSv1.2

I obtain the following output, and I see no network activity with Wireshark:

>>>> ... PKT cmd=E0 pos=0 <<<<
--- Gonna write 2 bytes
@@@ SSL_write(2) ... >> 0x28028

It only happens for the DISCONNECT packet, all other messages are exchanged without trouble.

Software versions

• I am using libmosquitto 1.4.11 and OpenSSL 1.1.0g on armv5.
• The same problem happens with the latest libmosquitto 1.4.15.
• I tested with two different MQTT brokers with the same results
  • mosquitto 1.4.14
  • RabbitMQ 3.7 with MQTT plugin

On the same device, a TLSv1.2 connection with libcurl is properly terminated.

---

EDIT: added output for TLSv1