Demonstrations of tcpdrop, the Linux BPF/bcc version. tcpdrop prints details of TCP packets or segments that were dropped by the kernel, including the kernel stack trace that led to the drop: # ./tcpdrop.py TIME PID IP SADDR:SPORT > DADDR:DPORT STATE (FLAGS) 20:49:06 0 4 10.32.119.56:443 > 10.66.65.252:22912 CLOSE (ACK) tcp_drop+0x1 tcp_v4_do_rcv+0x135 tcp_v4_rcv+0x9c7 ip_local_deliver_finish+0x62 ip_local_deliver+0x6f ip_rcv_finish+0x129 ip_rcv+0x28f __netif_receive_skb_core+0x432 __netif_receive_skb+0x18 netif_receive_skb_internal+0x37 napi_gro_receive+0xc5 ena_clean_rx_irq+0x3c3 ena_io_poll+0x33f net_rx_action+0x140 __softirqentry_text_start+0xdf irq_exit+0xb6 do_IRQ+0x82 ret_from_intr+0x0 native_safe_halt+0x6 default_idle+0x20 arch_cpu_idle+0x15 default_idle_call+0x23 do_idle+0x17f cpu_startup_entry+0x73 rest_init+0xae start_kernel+0x4dc x86_64_start_reservations+0x24 x86_64_start_kernel+0x74 secondary_startup_64+0xa5 20:49:50 12431 4 127.0.0.1:8198 > 127.0.0.1:48280 CLOSE (RST|ACK) tcp_drop+0x1 tcp_v4_do_rcv+0x135 __release_sock+0x88 release_sock+0x30 inet_stream_connect+0x47 SYSC_connect+0x9e sys_connect+0xe do_syscall_64+0x73 entry_SYSCALL_64_after_hwframe+0x3d [...] The last two columns show the state of the TCP session, and the TCP flags. These two examples show packets arriving for a session in the closed state, that were dropped by the kernel. This tool is useful for debugging high rates of drops, which can cause the remote end to do timer-based retransmits, hurting performance. USAGE: # ./tcpdrop.py -h usage: tcpdrop.py [-4 | -6] [-h] Trace TCP drops by the kernel optional arguments: -4, --ipv4 trace IPv4 family only -6, --ipv6 trace IPv6 family only -h, --help show this help message and exit examples: ./tcpdrop # trace kernel TCP drops ./tcpdrop -4 # trace IPv4 family only ./tcpdrop -6 # trace IPv6 family only