version of tcpreplay-edit with innermost ip randomization

[jasonlue]

tcpreplay-edit is a great tool to replay traffic from a simple seed pcap file, especially with --unique-ip feature. This works great with simply structured pcaps. By simple I mean each packet is NOT encapsulated with layers such as vlan,mpls, vxlan, etc. In reality, especially in data center, encapsulations abound.

Current version of tcpreplay-edit makes effort to handle these encapsulations. It handles vlan (including QinQ, 802.1q, 802.1ad) and mpls (with additional support on EoMPLS). However, this is just the tip of the iceberg. The typical vxlan, for example, is not supported.

Improper handling of the stripping of encapsulation results in parse_metadata() function results in
l2offset: the first appeared ethernet header
vlan_offset: the first appeared vlan header.
l2len stops at the first ip header.

When --unique-ip is applied, it's actually applied on the outermost IP header, which is the first encapsulation layer when the packet is encapsulated in another protocol such as vxlan. This makes --unique-ip almost useless for performance/loadbalance tests.

This MR makes an effort to handle most encapsulation protocols. It also creates a recursive programming structure to extend to more protocols easily.

With the new parse_metadata() function,
l2offset: the last appeared ethernet header
vlan_offset: the first appeared vlan header after the last appeared ethernet header.
l2len stops at the last ip header.

The protocols it covers are below:

Over Ethernet:

vlan: 802.1q, 802.1ad, QinQ
mpls: multiple labels.
PPPoE

Over IPv4/IPv6:

GRE over IP
GRE over IPv6
MPLS over IP
MPLS over IPv6
IPinIP
IP6 in IP6
IP4 in IP6
IP6 in IP4

over MPLS:

EoMPLS: wrap the whole ethernet frame inside MPLS

overPPP

MPLS over PPP
IPv4 over PPP
IPv6 over PPP

over GRE

EoGRE
ERSPAN I
ERSPAN II
ERSPAN III
PPPoGRE

over UDP

VXLAN over UDP
GENEVE over UDP
MPLS over UDP

[fklassen]

Moving to staging branch to resolve conflicts, test and review for possibility of pushing into 4.5.0.

[fklassen]

I merged into a staging branch after resolving merge conflicts. I ran tests, and they didn't go well. I inspected the differences and it appears that there are some fundamental errors. For example the portmap test should convert packet 13 from port 80 to 8080. That isn't happening.

I also am concerned that many tests for packet overruns are not implemented. This will almost certain result in a flood of CVE's.

Finally, I would like to see the "todo" addressed.

I really like this feature, but am concerned about targeting for 4.5.0. I will target for the following feature release.

echo Making test in ./test
Making test in ./test
cd ./test && make test
make[1]: Entering directory '/home/fklassen/git/tcpreplay/build/test'
rm: cannot remove 'test.log': No such file or directory
NOTICE: Tests must be run as root
Sending traffic on 'enp0s5' and 'enp0s5'
[tcpprep] Auto/Router mode test: make[1]: *** [Makefile:701: auto_router] Error 1
[tcpprep] Auto/Bridge mode test: make[1]: *** [Makefile:708: auto_bridge] Error 1
[tcpprep] Auto/Client mode test: make[1]: *** [Makefile:715: auto_client] Error 1
[tcpprep] Auto/Server mode test: make[1]: *** [Makefile:722: auto_server] Error 1
[tcpprep] Auto/First mode test: make[1]: *** [Makefile:729: auto_first] Error 1
[tcpprep] CIDR mode test: make[1]: *** [Makefile:743: cidr] Error 1
[tcpprep] Regex mode test: make[1]: *** [Makefile:736: regex] Error 1
[tcpprep] Port mode test: make[1]: *** [Makefile:750: port] Error 1
[tcpprep] MAC mode test: 			OK
[tcpprep] Comment mode test: 			OK
[tcpprep] Print info mode test: 		OK
[tcpprep] Print comment mode test: 		OK
[tcpprep] Config mode test: make[1]: *** [Makefile:694: prep_config] Error 1
[tcpprep] MAC reverse mode test: 		OK
[tcpprep] CIDR reverse mode test: make[1]: *** [Makefile:789: cidr_reverse] Error 1
[tcpprep] Regex reverse mode test: 		OK
[tcpprep] exclude packets test: make[1]: *** [Makefile:803: exclude_packets] Error 1
[tcpprep] include packets test: 		OK
[tcpprep] include source test: make[1]: *** [Makefile:817: include_source] Error 1
[tcpprep] include destination test: make[1]: *** [Makefile:824: include_dest] Error 1
[tcpreplay] Basic test: 			OK
[tcpreplay] Nano timer test: 			OK
[tcpreplay] Cache test: 			OK
[tcpreplay] Packets/sec test: 			OK
[tcpreplay] Mbps test: 				OK
[tcpreplay] Topspeed test: 			OK
[tcpreplay] Config file/VLAN add test: 		OK
[tcpreplay] Multiplier test: 			OK
[tcpreplay] Packets/sec Multiplier test: 	OK
[tcpreplay] Precache test: 			OK
[tcpreplay] Statistics test: 			OK
[tcpreplay] Dual file test: 			OK
[tcpreplay] Maximum sleep test: 		OK
[tcprewrite] Portmap test: make[1]: *** [Makefile:861: rewrite_portmap] Error 1
[tcprewrite] Portmap range test: make[1]: *** [Makefile:869: rewrite_range_portmap] Error 1
[tcprewrite] Endpoint test: make[1]: *** [Makefile:885: rewrite_endpoint] Error 1
[tcprewrite] Pseudo NAT test: make[1]: *** [Makefile:894: rewrite_pnat] Error 1
[tcprewrite] Truncate test: make[1]: *** [Makefile:941: rewrite_trunc] Error 1
[tcprewrite] Pad test: make[1]: *** [Makefile:949: rewrite_pad] Error 1
[tcprewrite] Seed IP test: make[1]: *** [Makefile:957: rewrite_seed] Error 1
[tcprewrite] Src/Dst MAC test: make[1]: *** [Makefile:903: rewrite_mac] Error 1
[tcprewrite] Layer2 test: make[1]: *** [Makefile:965: rewrite_layer2] Error 1
[tcprewrite] Config/VLAN Add test: make[1]: *** [Makefile:974: rewrite_config] Error 1
[tcprewrite] Skip bcast test: make[1]: *** [Makefile:982: rewrite_skip] Error 1
[tcprewrite] DLT User test: make[1]: *** [Makefile:993: rewrite_dltuser] Error 1
[tcprewrite] DLT Cisco HDLC test: make[1]: *** [Makefile:1002: rewrite_dlthdlc] Error 1
[tcprewrite] VLAN 802.1ad test: make[1]: *** [Makefile:1011: rewrite_vlan802.1ad] Error 1
[tcprewrite] VLAN Delete test: make[1]: *** [Makefile:1021: rewrite_vlandel] Error 1
[tcprewrite] Remove EFCS: make[1]: *** [Makefile:1030: rewrite_efcs] Error 1
[tcprewrite] Force TTL: make[1]: *** [Makefile:1038: rewrite_1ttl] Error 1
[tcprewrite] Increase TTL: make[1]: *** [Makefile:1046: rewrite_2ttl] Error 1
[tcprewrite] Reduce TTL: make[1]: *** [Makefile:1054: rewrite_3ttl] Error 1
[tcprewrite] Force TTL with header fix: make[1]: *** [Makefile:1062: rewrite_1ttl-hdrfix] Error 1
[tcprewrite] Increase TTL with header fix:make[1]: *** [Makefile:1070: rewrite_2ttl-hdrfix] Error 1
[tcprewrite] Reduce TTL with header fix: make[1]: *** [Makefile:1078: rewrite_3ttl-hdrfix] Error 1
[tcprewrite] TOS test: make[1]: *** [Makefile:1086: rewrite_tos] Error 1
[tcprewrite] MTU Truncate test: make[1]: *** [Makefile:1094: rewrite_mtutrunc] Error 1
[tcprewrite] Substitute Src/Dst MAC test: make[1]: *** [Makefile:913: rewrite_enet_subsmac] Error 1
[tcprewrite] Seeded MAC test: make[1]: *** [Makefile:923: rewrite_mac_seed] Error 1
[tcprewrite] Seeded Keep MAC test: make[1]: *** [Makefile:932: rewrite_mac_seed_keep] Error 1
[tcprewrite] L7 fuzzing test: make[1]: *** [Makefile:1102: rewrite_l7fuzzing] Error 1
[tcprewrite] TCP sequence test: make[1]: *** [Makefile:877: rewrite_sequence] Error 1
[tcprewrite] Fix checksum test: make[1]: *** [Makefile:1110: rewrite_fixcsum] Error 1
[tcprewrite] Fix length and pad test: make[1]: *** [Makefile:1118: rewrite_fixlen_pad] Error 1
[tcprewrite] Fix length and truncate test: make[1]: *** [Makefile:1126: rewrite_fixlen_trunc] Error 1
[tcprewrite] Fix length and delete test: make[1]: *** [Makefile:1134: rewrite_fixlen_del] Error 1
make[1]: Target 'test' not remade because of errors.
make[1]: Leaving directory '/home/fklassen/git/tcpreplay/build/test'
make: *** [Makefile:903: test] Error 2

test.log