[WIRE136] Network redesign

[andreagilardoni]

As the title states the objective of this PR is the redesign of the network stack of the portenta C33. This PR aims to solve multiple issues that affected reliability, performance and usability of networking.

The main issues addressed are the follwing:

• Unreliable TCP streams over both WiFi and Ethernet
• Bad performances of Ethernet drivers
• Bad Performances of WiFi drivers
• Memory leaks in the code on different levels, starting from missing destructors of Client classes
• incorrect management of lwip pbufs that lead to missing chunks in TCP communication

This issue needs arduino/ArduinoCore-API#218 in order to completely fix memory leaking when using a class that is derived from arduino::Client.

Since this work started as a task to address Ethernet bad performances and behavior it was mainly focused on the ethernet improvement. We were able to achieve a speed of ~80Mbit/s in Rx with some fine tuning of the libraries (this require high ram usage) by means of Rx zero copy capabilities of Ra6m5 controller. Wifi drivers were not taken into consideration that deeply, but there is room for improvement, i.e. by reducing the number of memcpy performed on buffers.

Furthermore some aspects of the integration of lwip have been improved:

• now a WiFi Client runs exclusively on WiFi interface, the same is valid with Ethernet
• the dhcp client interface has been simplified to leverage more on lwip implementation
• the dns integration has been simplified
• the code has been designed to run mostly in a non-blocking way, in order to avoid starvation in a system that doesn't support threads

This issue is currently on draft, since there are still some imperfections, which are addressed in TODO and FIXME comments. Some will be addressed before this PR is merged and others are going to provide a path to follow for future improvements.

[andreagilardoni]

@Eduardo-Lopes I think I solved the issues you were describing, if you can confirm me that.

@JAndrassy I will for sure take a look at your work and take it into consideration for further deveplment, thanks for your work!

[Eduardo-Lopes]

Hi @andreagilardoni,
unfortunately, the problem persists.
Also, the problem is on Ethernet and DHCP, not in EthernetServer or EthernetClient.

[andreagilardoni]

@Eduardo-Lopes sorry, I forgot to add a part of the fix

[Eduardo-Lopes]

@andreagilardoni the DHCP is solved, but still there is connection loss when I stress test.

[andreagilardoni]

@Eduardo-Lopes how are you performing stress tests? So that can better address the issue

[Eduardo-Lopes]

I just call a Python script that loops requesting the WebServer on C33.
I added the code on the issue: #240 (comment)

[andreagilardoni]

This will take some more time to debug, I think there is a memory leak somewhere I am missing.