IPv6 on esp8266-nonos-sdk and arduino

[d-a-v]

IPv6 for esp8266-nonos-sdk and Arduino

Testers and feedback are welcome

This PR will be kept up-to-date with current master for testing.

update 2019-09-18:

• ping, tcp, udp are working on link- and global-scope.
• dhcpv6 stateless is working
• addresses are based on MAC address for now

update 2019-10-10: Ready for review:

• main change is IPAddress
• addition: IfList / IfListClass
• other notable changes: sdk's ip_info and ip_addr renamed to ipv4_info and ipv4_addr.
• the rest is adaptation:
  extensive use of IPAddress everywhere where an uint32_t was used for an IP address.
• not everything is tested yet (notably ssdp and llmnr)

What about changes

There is nothing to do in your own sketches:

• (exception:) if you are dealing with uint32_t addresses, use IPAddress instead.
• even though IPAddress now understands IPv6, you don't need to spell addresses,
  IPv6 relies on DNS and mDNS
• sketches will (hopefully) just work on IPv6 along with IPv4
• tcp/udp servers will be accessible from the entire internet6
• dns requests will result in an IPv6 address when no IPv4 is available
  (IPv4 is the compiled-in default when both are available)
• this sketch shows how to display your address
• IPv6 really comes from lwIP-2.1.0
• IPv6 is not enabled by default. Select it in lwIP Variant menu
• IPAddress reworked to be compatible with IPv4 and IPv6, except toString() but printTo() is ok
• IPv6 autoconfiguration, DHCP6-stateless enabled
• SDK's ip_addr renamed to ipv4_addr (was in the way)

This PR needs polishing and feedback before integration.

How to try

git fetch origin pull/5136/head:ipv6
git checkout ipv6

Revert to master:

git checkout master

in case of updates in the PR, do this before restarting the above.

git checkout master
git branch -D ipv6

IPv6 notes

IPv6 is a strange new beast for the majority of us.
I am very far from being an IPv6 expert, but here are some tips about it.

• We don't need anymore WiFi.config(<static address>) (well, we can still use it for IPv4)
  IPv6 auto-configures itself in various ways. IPv6 always has a valid local address (like 192.168. or 10. intranets). This address is always fe80:0:0:0:xxxx:xxxx:xxxx:xxxx and is called the "scope:link" address. It always work even on a not IPv6-capable network. The lower part of the address is left to the stack (64 bits only for your intranet, where IPv4 internet is only 32 bits), and can be (here: currently is) based on the mac address. Not yet: It could be randomly changed every few hours / days. It could be setup by program.

• To be routable on internet, we need a second network address on the same interface. This one is called "scope:global" address. It can have (here: it has) the same lower part (x:x:x:x). The higher/network part is given by your ISP through your router with DHCP6. It always start with a 2 - ex: 2a00:1450:4002:807:xxxx:xxxx:xxxx:xxxx (/64 prefix).

• Consecutive 0s in the address can be omitted (fe80::xxxx:xxxx:xxxx:xxxx). ::1 is self/local-interface - we do not use it here - and :: is the equivalent of 0.0.0.0 (= any address / unconfigured).

• We generally do not need to write those addresses down, all is done with DNS and mDNS.

• My ISP provides IPv6. When enabled, every single IPv6-capable device at home is routable on the internet, and the modem/router provided by my ISP does not offer any kind of firewall. With IPv4, NAT is a kind of minimal firewall. This is good to know.

[tablatronix]

What is the point reasoning of this? Lots of strange behavior reported, Not sure where to start investigating, but this is showing up in places

[owendelong]

First, a huge THANK YOU!! This is fantastic progress, but it appears that the documentation has not been updated to reflect IPv6. While IPv6 is not a strange new beast to me and I am an IPv6 expert, I'm not a stack development expert (or even a stack developer at all), so this implementation is a great help to me. That said, I'm happy to help answer any questions or collaborate in any useful way with the people working on this if additional IPv6 expertise would be useful.

While I appreciate the ideal of "IPv6 and IPv4 should just work without code updates as long as you use the IPaddress class", the realities are a bit different...

e.g. What is the expected behavior of LocalIP on a dual stack network? What happens if the local network is dual stack, but the remote host is IPv6 only?

Some errata for the "IPv6 Notes above in the PR description:
fe80::/10 is not like RFC-1918 addresses in IPv4 (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16). It is, in fact a link local address (akin to IPv4 link local 169.254.0.0/16 RFC-3927, but more useful and required).

If you want an IPv6 analog to RFC-1918, you want to look at Unique Local Addresses (Random) (RFC-4193). Those addresses are in the range fd00::/8 and are technically part of fc00::/7, but the Internet Draft for ULA (Registered) never gained consensus in the IETF, so while fc00::/7 is set aside for ULA and fc00::/8 is supposed to be ULA (Registered), the reality is this space is in limbo and only fd00::/8 is available for legitimate use.

fe80::/10 being link local has some distinctions from ULA and GUA (Global Unicast Addresses, or, "normal" addresses). Technically, ULA can be routed anywhere the parties involved agree to route it. The intent is that it doesn't cross organizational boundaries, but that's strictly by convention. OTOH, link local addresses CANNOT BE ROUTED. Not to the subnet next door, not to the internet, not to your partner. They are strictly limited to the local link. A local link is defined as the set of hosts which can be reached without traversing a router functioning at layer 3 (basically the local subnet or broadcast domain, but there are subtle highly technical distinctions that prevent an exact match to that in certain corner-case environments).

All IPv6 capable interfaces MUST have a link local address. The link local address is used for bootstrapping the automatic configuration of any other addresses. There are multiple ways to generate a link local address, but by far the simplest is to take your local MAC address, convert it to EUI-64 (I'll explain the formula for this below) and append it to fe80::. Technically fe80:: link locals are supposed to be in a /10 subnet, but many implementations erroneously use a /64. In practice, since only the last 64 bits differ from fe80:: and there is no off-link routing of link local addresses, these net mask errors have little, if any, consequence, but /10 is the correct value.

To convert an EUI-48 MAC address to EUI-64, we perform the following steps:

1. The first octet of the EUI-48 address is logically or'd with 0x02. This converts the identifier from OUI to Company ID (CID).
2. The first three octets (OUI) are separated from the last three octets (Extension).
3. The address is reassembled with 0xfffe placed between the OUI and Extension.

Example: An EUI-48 global MAC address on an interface is 68:fe:f7:07:11:6f
The resulting EUI-64 IPv6 suffix would be 6afe:f7ff:fe07:116f
The resulting IPv6 link local address could be fe80::6afe:f7ff:fe07:116f

68->6a is the result of or'ing 0x68 with 0x02 as described in step 1.

There are some privacy considerations with use of MAC addresses as the basis for EUI-based IPv6 addresses, so there are other cryptographic and randomized mechanisms available for the autonomous creation of the host portion of IPv6 addresses.

Any home gateway which implements IPv6 and does not implement a minimal firewall with default-deny-all inbound policy is not fully RFC compliant and should be reported to its manufacturer as defective. (See RFC-6092).

There are rules for zero- elimination in IPv6 addresses that warrant some cation and are important to understand...

First, any sequence of multiple quartets (groups of 4 hex digits) that are all 0 bits can be reduced to :: with the limitation that no more than one such sequence may be thus reduced in any address.

The most significant bits of any quartet that are 0 can be eliminated from display of an address.

There is ambiguity in how addresses can be represented as a result of these rules. For example, consider the following address:

2620:0000:0930:0001:0000:0000:0000:0102

The shortest representation and the intuitive one most humans would use is:
2620:0:930:1::102

Many software implementations, however, would present the following:
2620::930:1:0:0:0:102

Some people new to IPv6 would attempt:
2620::930:1::102 which is wrong because it then becomes ambiguous as to how many 0 bits are represented by each ::. This address could be expanded to 2620:0:0:930:1:0:0:102 or several other permutations.

If one needs to compare IPv6 addresses, one must exercise caution and the comparison of the textual representation of IPv6 addresses should be avoided because of these ambiguities. Instead, the IPv6 address(es) should be converted to a 128 bit binary value and compared logically (and, or, not, etc.) or mathematically (==, +, -, etc.).

Similarly, it is recommended that when addresses are stored in databases or other files intended for machine parsing, that they be stored as binary values.

For convenience, IPv4 addresses can be represented as if they were IPv6 addresses in one of two forms:

• ::ffff:10.1.2.3
• ::ffff:0a01:0203
  Both of these forms represent the same IPv4 address. These representations are for internal use only and will not be present in IPv6 datagrams on the wire. Host implementations are required to recognize these as IPv4 protocol addresses packed in an IPv6 format for convenience. (Allows software to be written for IPv6 only and still handle both protocols). I don't know if this representation is supported in the code implemented here.

A reference for the MAC address manipulation can be found here:
https://standards.ieee.org/content/dam/ieee-standards/standards/web/documents/tutorials/eui.pdf
Also in appendix A of https://tools.ietf.org/html/rfc4291 (starting on Page 19)

[d-a-v]

the documentation has not been updated to reflect IPv6.

Any help with this is much appreciated (especially with your long and very instructive summary on IPv6 addressing definitions/rules/policies)

First, a huge THANK YOU!! This is fantastic progress.

This is appreciated, thanks :)
My contribution here was on IPAddress.
Once again, lwIP authors deserve these acknowledgements.

While I appreciate the ideal of "IPv6 and IPv4 should just work without code updates as long as you use the IPaddress class", the realities are a bit different...
e.g. What is the expected behavior of LocalIP on a dual stack network? What happens if the local network is dual stack, but the remote host is IPv6 only?

LocalIP is the address of the interface. When IPv6 is enabled, this interface once fully initialized has an IPv4 address and some IPv6 addresses. That, is on a dual stack network (or at least dual stack DHCP server).
I have personally not tested with an IPv6 only network.

As for now I can only guess what would happen when there is no DHCPv4 answer. I think the waiting-for-a-connection loop described in the IPv6 example would succeed but the usual test (WiFi.status() == WL_CONNECTED) would stay false. It wouldn't prevent communications.
Of courses sketches that regularly test this legacy function would always think there is no network.

In order to fulfill the holy "transparent backward compatibility" precept that we are always trying to follow, we may need to answer WL_CONNECTED when IPv6 is enabled thanks a new external function / user callback called from WiFi.status().

[plinioseniore]

Fantastic @d-a-v! Is my understanding is correct, the implementation is a dual stack where the node work at same time with its IPv4 address as well as IPv6 ones.

So it would be possible to communicate at same time to IPv6 devices as well as legacy (think about Arduino Wiznet based ones) IPv4 ones?

Thanks,
Dario.

[d-a-v]

Yes