MTU, TCP Tuning, and the net.* sysctl Knobs

Modern TCP stacks tune themselves remarkably well, so aggressive network tuning on a VPS is usually unnecessary and often counterproductive. But there are a handful of sysctl knobs that genuinely matter for throughput and latency, and understanding MTU still resolves real problems. This post covers the settings that are safe to adjust and the reasoning behind them.

Understanding MTU and Path MTU Discovery

The Maximum Transmission Unit is the largest packet a link can carry. Ethernet typically uses 1500 bytes. Every layer between you and a destination has its own MTU, and the smallest one on the path limits the largest packet that travels without fragmentation. Path MTU Discovery (PMTUD) lets a host learn that limit by sending packets with the DF (don't fragment) bit set and listening for ICMP fragmentation-needed messages.

ip link show
ping -M do -s 1472 -c 3 93.184.216.34

The ping with -M do tests a 1500-byte path (1472 payload plus 28 bytes of IP and ICMP headers). If it succeeds, the path supports 1500. If it fails with fragmentation needed, lower the size to find the ceiling. This is the definitive way to determine your effective MTU.

Common MTU Fixes

The most common MTU problem on servers appears in tunnels, VPNs, and encapsulation. When you wrap traffic in another header, the effective MTU shrinks. Setting the interface MTU down to match prevents fragmentation storms.

ip link set eth0 mtu 1400
ip link set wg0 mtu 1420

A related issue is the kernel ignoring ICMP fragmentation-needed messages. If packets get dropped and you see no fragmentation, check the relevant sysctls. On some stacks, net.ipv4.tcp_mtu_probing is off by default. Enabling it makes TCP actively probe for a working MTU.

sysctl -w net.ipv4.tcp_mtu_probing=1
sysctl -w net.ipv4.tcp_base_mss=1024

These two settings help in networks where ICMP is filtered and PMTUD is unreliable, which happens on some cloud and firewall-heavy paths.

The net.core Queue and Socket Knobs

The net.core group controls global networking resources: the receive and send socket buffers, the maximum queue lengths, and the automatic tuning of those buffers.

sysctl net.core.rmem_max
sysctl net.core.wmem_max
sysctl net.core.netdev_max_backlog
sysctl net.core.somaxconn

The rmem_max and wmem_max set the maximum the kernel allows for receive and send buffers per socket. Raising them only matters if you are moving very large amounts of data and the auto-tuning is capping out. The netdev_max_backlog is the maximum packets queued in the receive path when the CPU is busy, which matters on loaded servers.

The somaxconn value limits the pending connection backlog for listening sockets. A burst of connections that exceeds this gets refused. Raising it helps servers that accept many simultaneous new connections.

The net.ipv4 TCP Knobs

The net.ipv4 group carries per-protocol parameters. The congestion control, window settings, and automatic tuning knobs live here.

sysctl net.ipv4.tcp_congestion_control
sysctl net.ipv4.tcp_sack
sysctl net.ipv4.tcp_window_scaling
sysctl net.ipv4.tcp_timestamps

The default congestion control on Ubuntu is usually cubic, which is fine for most internet paths. For high-bandwidth long-latency links, bbr often improves throughput dramatically. You can enable it and reload connections.

sysctl net.ipv4.tcp_available_congestion_control
sysctl -w net.ipv4.tcp_congestion_control=bbr
sysctl net.ipv4.tcp_notsent_lowat

TCP SACK and window scaling should be left on for high-throughput traffic. TCP timestamps help correctness and RTT measurement. Disabling any of these to save a few bytes of header is rarely worth the loss of robustness.

Making Persisted Tuning Changes

All sysctl tunings live in /proc at runtime, so they reset on reboot. Persist them in /etc/sysctl.conf or a file under /etc/sysctl.d/.

net.core.rmem_max = 26214400
net.core.wmem_max = 26214400
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_slow_start_after_idle = 0
net.core.somaxconn = 4096

After editing, apply with sysctl --system or sysctl -p. The slow_start_after_idle setting keeps TCP from shrinking its congestion window after a quiet period, which helps bursty request patterns re-establish full throughput faster.

A Realistic Tuning Recipe for a VPS

For a typical VPS serving web traffic, most tuning is unnecessary. If you want modest, safe improvements, start here.

sysctl -w net.core.somaxconn=4096
sysctl -w net.ipv4.tcp_fastopen=3
sysctl -w net.ipv4.tcp_slow_start_after_idle=0
sysctl -w net.core.netdev_max_backlog=30000

TCP FastOpen (tcp_fastopen=3) lets new connections skip part of the handshake, which measurably reduces latency for repeated short connections. The other values raise headroom without changing protocol behavior. Apply them and measure before and after with a benchmark. If you cannot measure improvement, revert.

Measuring the Impact

Never tune blindly. Measure the actual effect with tools that show real throughput and latency.

iperf3 -c <server> -t 30
curl -o /dev/null -sS -w 'time_total: %{time_total}s
' https://netbayhosts.in
ss -ti

iperf3 gives you raw throughput over a single TCP stream. curl reports real-world connection latency. The ss -ti command shows per-socket TCP details including congestion control and round-trip time, letting you verify a knob took effect on live connections.

The right attitude to sysctl tuning is skeptical empiricism: change one variable, measure, and keep the change only if it helped. Netbay VPSes give you root access to apply and test these knobs anytime at netbayhosts.in.