Much more elaborate test suite

[dtaht]

Leveraging the resources we've garnered via #151 producing a repeatable test suite for comparisons of not just our stuff, but that of others, would be nice. I'm not going to lay out too much of what that looks like here as yet...

[dtaht]

https://www.linkedin.com/feed/update/urn:li:activity:6996837189460328449/ CI bootcamp

[thebracket]

I absolutely agree - CI and a test suite would be awesome. It's also a really big job, so it may take a while. There are some parts of the xdp code that I'm not sure how to unit test (I don't know how to run unit-tests on the BPF side when you have moving parts that require data transit from ingress XDP to egress TC BPF code. It's probably possible, I just haven't figured it out, yet). So on that side, some integration testing to quickly pull the latest XDP code and run some tests against it will be required. It looks like the orchestration system can do it, it'll take me a while to figure out exactly how to constrain the various moving parts. One thing that's really handy (I use it all the time in my Rust libraries) is to have a standard benchmark and persistent test results. REALLY handy to see that not just "the code works", but "performance has changed by X%".

…

On Mon, Nov 14, 2022 at 4:25 AM Dave Täht ***@***.***> wrote: https://www.linkedin.com/feed/update/urn:li:activity:6996837189460328449/ CI bootcamp — Reply to this email directly, view it on GitHub <#153 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADRU43223ALC3MEFWPVBJJLWIIHSVANCNFSM6AAAAAAR7DUBA4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[dtaht]

Maybe gathering requirements is a thing for another form of github, so we can all edit one page?

Traffic generators:

https://www.spirent.com/products/testcenter-hardware

https://www.candelatech.com/datasheet_fire.php

Correctness tests

https://www.iwl.com/network-protocol-testing

[dtaht]

I added some stuff for simulation, and the CI is a great start (tho I would like it to retain artifacts and have those get installed into the testbed automatically). I'm not sure how to use checkboxes. Still open here are:

[ ] Unit testing stuff (I'd wanted unit tests, for example, of structure size)
[ ] Test suite (notably of 10k ips)

[dtaht]

What we have now, is rather hacky. I have a string of shell scripts, mostly, that help manage the testbed. Trying to find the right methods AND things to emulate has been a problem, and I diverted to writing out some ideas in C rather than any other language, just to better express myself for a while. I have an increasing grip on what I would like to do (containerizing a lot), and what we have has been good enough for v1.4 so far... mostly.