tcpdump and Packet Capture: Reading Traffic

When network problems elude every ping and traceroute, the packet capture becomes definitive evidence. tcpdump is the classic Linux tool for capturing and analyzing raw network traffic. It shows you exactly what bytes crossed the wire, in what order, and toward which destination. This turns network debugging from speculation into precise diagnosis.

Getting Started with tcpdump

tcpdump runs as root or with the CAP_NET_RAW capability. The most common first invocation captures everything on an interface, but that quickly overwhelms the terminal on a busy server. You nearly always want a filter to reduce the noise.

sudo tcpdump -i eth0
sudo tcpdump -i eth0 -c 50
sudo tcpdump -i eth0 port 443

The -c 50 flag stops capture after fifty packets, which prevents runaway output while you learn. The port filter narrows the capture to a single service. Filters use a rich Berkeley Packet Filter (BPF) syntax that lets you match on protocols, ports, hostnames, addresses, and even packet contents.

Expression Syntax

The killer feature of tcpdump is its filter expression language. You can combine primitives with and, or, and not to target exactly the traffic you care about.

sudo tcpdump -i eth0 host 10.0.0.10
sudo tcpdump -i eth0 tcp port 22 and not src net 10.0.0.0/24
sudo tcpdump -i eth0 'icmp[icmptype] = icmp-echo'
sudo tcpdump -i eth0 'tcp[tcpflags] & (tcp-syn) != 0'

The last two examples inspect packet bytes at specific offsets. They match ICMP echo requests and any packet with the SYN flag set, respectively. This level of control is what makes tcpdump suitable for protocol-level debugging.

Interpreting the Output

tcpdump output looks cryptic at first, but each field is meaningful. A TCP capture line shows the timestamp, source and destination addresses with ports, sequence and acknowledgment numbers, TCP flags, window size, and the payload length.

10:15:01.223344 IP 10.0.0.5.54022 > 93.184.216.34.443: Flags [S], seq 123456, win 64240

This is a SYN packet (the [S] flag) from 10.0.0.5 port 54022 to port 443 on the remote host, with an initial sequence number and a window of 64240 bytes. Knowing how to read these lines reveals handshake issues, retransmissions, and window scaling problems.

The -v flag increases verbosity, showing TTL values, IP options, and TCP options. The -nn flag stops address and port resolution, which keeps output fast and unambiguous. Always use -nn when debugging, because reverse DNS lookups can hang and corrupt your timing analysis.

Saving and Analyzing Captures

Beyond live inspection, tcpdump saves captures to files for later analysis or for sharing.

sudo tcpdump -i eth0 -w capture.pcap -c 10000
sudo tcpdump -r capture.pcap
sudo tcpdump -r capture.pcap -nn -c 20

The -w flag writes raw packets to a pcap file. The -r flag reads them back. The pcap format is portable, so you can open the same file in Wireshark on your laptop for graphical analysis, or inspect it directly on the server with tcpdump.

For long-running captures, limit the stored bytes with -s (snaplen). A value of 96 bytes captures header information for TCP analysis without storing full packet payloads, which reduces disk usage and privacy exposure.

sudo tcpdump -i eth0 -s 96 -w headers.pcap

The Detective Workflow

Diagnosing a symptom typically follows a repeatable pattern. Suppose a web client cannot reach your service. Capture on the server's interface while reproducing the problem, then look for the syn packets reaching the interface and whether the SYN-ACK leaves.

sudo tcpdump -i eth0 -nn port 443 and 'tcp[tcpflags] & (tcp-syn) != 0'

If SYN packets arrive but no SYN-ACK leaves, the kernel is dropping them, pointing to a firewall or a service bound to the wrong address. If SYN packets never arrive, the problem is upstream, a routing or filtering issue elsewhere. Each capture narrows the search by hard evidence.

DNS Debugging

DNS problems produce clear traces. Query packets go to port 53 and responses come back. Watch for timeouts, unexpected response sources, or answers that take far too long.

sudo tcpdump -i eth0 -nn udp port 53
dig example.com

Run the capture in one terminal, then trigger the lookup with dig. Compare the query line with the response line. If the response arrives from the wrong server or not at all, your resolv.conf or upstream DNS is suspect. tcpdump reveals exactly which DNS server answered, at what time, and with what content.

Capturing on Virtual Interfaces and Tunnels

On a VPS, packets may traverse multiple interfaces. Traffic inside a VPN tunnel might appear only on the tunnel interface. Bridges have per-port captures. Use -i any to capture on all interfaces at once, and add vlan filter expressions to isolate tagged traffic.

sudo tcpdump -i any -nn port 443
sudo tcpdump -i eth0.100 -nn -e
sudo tcpdump -i br0 -nn arp

The -e flag on a VLAN interface reveals the 802.1Q tag in the Ethernet header. The br0 capture example filters ARP traffic on a bridge, useful for diagnosing address resolution problems in virtualized networks.

Packet capture is a superpower for sysadmins. You can practice on a fresh VPS from Netbay and capture real production traffic safely at netbayhosts.in.