QUIC Interop: some testcases fail on LibreSSL (server), picoquic (client)

[chipitsine]

Detailed Description of the Problem

I ran QUIC Interop suite on picoquic client, servers were HAProxy/LibreSSL and HAProxy/QUICTLS

picoquic / libressl

+----------+----------------------+
|          |       haproxy        |
+----------+----------------------+
| picoquic | HDCLRMS3BAL1L2C1C2V2 |
|          |          E           |
|          |       C20RZU6        |
+----------+----------------------+
+----------+----------------------+
|          |       haproxy        |
+----------+----------------------+
| picoquic | G: 9023 (± 47) kbps  |
|          | C: 4467 (± 219) kbps |
+----------+----------------------+

picoquic / quictls

+----------+-------------------------+
|          |         haproxy         |
+----------+-------------------------+
| picoquic | HDCLRC20MSRZ3BAL2C1C2V2 |
|          |            E            |
|          |           UL16          |
+----------+-------------------------+
+----------+----------------------+
|          |       haproxy        |
+----------+----------------------+
| picoquic | G: 9078 (± 113) kbps |
|          | C: 4636 (± 250) kbps |
+----------+----------------------+

haproxy commit used: 7acdc3f

Expected Behavior

not to fail

Steps to Reproduce the Behavior

run QUIC Interop suite on LibreSSL

Do you have any idea what may have caused this?

No response

Do you have an idea how to solve the issue?

No response

What is your configuration?

usual QUIC Interop

Output of haproxy -vv

no

Last Outputs and Backtraces

No response

Additional Information

No response

[chipitsine]

logs.zip

attached logs for "chacha20" and "resumption" testcases

@a-denoyelle fyi

[chipitsine]

I guess chacha20 may be disabled on purpose due to c7335d5

[a-denoyelle]

Thanks for your information. So here are the tests which are currently failing under LibreSSL :

• C20 : chacha20. Your analysis seems right. We should probably disabled the testcase when using LibreSSL. This can be done by modifying run_endpoint.sh.
• R : resumption
• Z : zerortt

Maybe @haproxyFred can tell us if those two last tests are supposed or not to work with LibreSSL ?

[chipitsine]

zerortt is not supposed to run on LibreSSL. thus I did not include logs

[chipitsine]

LibreSSL does not support TLS sessions. maybe "resumption" testcase is not supposed to work either

[chipitsine]

btw, I ran the same tests on current master ...

@a-denoyelle , what do you think, should we enable weekly scheduled quic interop for libressl ?
we know testcases that work, and we can catch if them got broken

[a-denoyelle]

I think it could be useful to run it on a weekly basis. However I expect it to be mostly similar to interop.seeman.io with QuicTLS as QUIC SSL parts do not evolve frequently these days...

[chipitsine]

no, it will be more "automatic". as QUIC Interop just produce a matrix of results, we'll make it more restrictive, since we know which testcases we want to be green, we'll check for them and we'll fail once them are red

[chipitsine]

@a-denoyelle , what do you think ?

we can enable weekly workflows like that:

https://github.com/chipitsine/haproxy-quic-interop-libressl/blob/main/.github/workflows/quic-interop-libressl.yml#L44-L45

(example run: https://github.com/chipitsine/haproxy-quic-interop-libressl/actions/runs/9784433890 )

it might be useful, since we know which tests are supposed to pass, we enable only those tests.
test will be run in automated mode, if everything is green, no notification will be issued.

we are not bound to particular repository, in theory we can enable those workflows either in https://github.com/haproxy/haproxy or https://github.com/haproxytech/quic-dev (or any arbitrary repo)

[a-denoyelle]

It would be nice for us to have our own set of weekly interop tests. We could also go further by enabling non-default QUIC features which are rarely tested, such as tune.quic.socket-owner listener. For this, we could only run a small subset of tests as it's not useful to rerun everything. The only downside of running QUIC interop test is that it will consume CI credits and I don't know if we are limited or not ?

[chipitsine]

It would be nice for us to have our own set of weekly interop tests. We could also go further by enabling non-default QUIC features which are rarely tested, such as tune.quic.socket-owner listener. For this, we could only run a small subset of tests as it's not useful to rerun everything. The only downside of running QUIC interop test is that it will consume CI credits and I don't know if we are limited or not ?

my own github account shows me that I have 2000 minutes monthly, but I could not find usage report (and I've no idea what happens when all minutes are spent)

also, I found useful to run QUIC Interop with ASAN enabled, it discovered several issues in the past.
I'll add ASAN to docker images later

[chipitsine]

@a-denoyelle , can you please try my workflow in your fork (in order to keep project limits safe) ?

once we are fine with details we'll enable on project repo

[a-denoyelle]

We already had notice in the past about CI usage for haproxytech repositories. As such, after discussing with Willy it is probably better to integrate your CI scripts directly in this repository.

[chipitsine]

"in this" refers to https://github.com/haproxy/haproxy ?

or we can setup dedicated repo, for example, https://github.com/haproxy/<quic-interop-conformance>

[chipitsine]

well, I think it should not be hard to scrap github api to calculate GHA minutes ))
I'll try

[a-denoyelle]

Indeed I meant https://github.com/haproxy/haproxy. I prefer this solution as it will be more visible, by opposition to creating a dedicated new repository.

[chipitsine]

here we go...
https://www.mail-archive.com/[email protected]/msg45081.html