TCP and Kernel Tuning for Latency-Sensitive Servers

Network latency on a modern VPS is rarely the wire; it is the queues. The kernel buffers a finite amount of unread data per connection, and when a fast sender meets a slow reader, packets back up, buffers overflow, and retransmissions add tens of milliseconds that feel like seconds to a user. Tuning TCP and the scheduler is about shrinking those queues, and it is one of the highest-leverage changes available without rewriting the application.

The few sysctls that matter

There are hundreds of knobs under net.ipv4, but latency work concentrates on a handful. Socket buffers set how much unread data the kernel holds before it starts dropping; the send queue controls how much it lets a slow peer back-pressure the writer. For an interactive API server you want just enough buffering to absorb micro-bursts, not so much that latency balloons under load.

# /etc/sysctl.d/99-latency.conf
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_max = 4194304
net.ipv4.tcp_rmem = 4096 262144 4194304
net.ipv4.tcp_wmem = 4096 262144 4194304
net.ipv4.tcp_fastopen = 3
net.ipv4.tcp_slow_start_after_idle = 0

Reload with sysctl -p /etc/sysctl.d/99-latency.conf and verify the values stuck. The fastopen and slow-start settings matter for short-lived connections, where the handshake and the slow-start ramp add a full round trip to every request. tcp_slow_start_after_idle = 0 stops the kernel from resetting the congestion window on idle, which keeps long polling and keep-alive connections walking fast.

Backlog and the accept queue

An overloaded accept queue silently drops connections or, worse, sends resets to the application. The backlog is a per-listener limit set by the app, but the kernel enforces a ceiling via net.core.somaxconn. If your web server listens with a high backlog and the kernel caps it, you never get the depth you asked for.

sysctl net.core.somaxconn
# confirm the listener actually bound the requested depth
ss -ltn

Raise somaxconn to match what your server asks for, and raise it in the application config too. On a busy front end, a too-small backlog shows up as connection refused spikes exactly when traffic peaks.

Connection tracking and keep-alive

A second, quieter performance ceiling on Linux is the connection-tracking table that the netfilter stack uses to remember in-flight flows. Each connection occupies a table entry, and when the table fills, the kernel starts rejecting new connections regardless of how much memory or CPU you have free. High concurrency APIs with chatty clients hit this before they hit the CPU.

# where the limit stands and how close you are
sysctl net.netfilter.nf_conntrack_max
cat /proc/sys/net/netfilter/nf_conntrack_count

If the count sits near the max, either raise the limit with headroom for memory, or reduce concurrent tracking by enabling a keep-alive policy so clients reuse a small number of long-lived connections instead of hammering the table with new ones. The same keep-alive setting that reduces handshake latency also shrinks the tracking footprint, so it is a two-for-one latency lever on an API-heavy server. Verify the change by watching the count under the same traffic you profiled before.

When the scheduler adds latency

The network scheduler decides who transmits next when several connections compete. The default pfifo_fast is fair but not latency-friendly; a bulk download can push a small interactive request behind it. For latency-sensitive traffic, the cake or fq scheduler keeps interactive flows moving ahead of bulk ones.

# view current queues on the outbound interface
tc -s qdisc show dev eth0
# change scheduler to fq for interactive traffic (tune to your NIC)
sudo tc qdisc replace dev eth0 root fq

The change is not persistent, so wire it through a systemd unit or netplan hook if you keep it. The benefit is a visible cut in p99 latency for short requests sharing a link with streaming or transfer traffic.

Measure before and after

Tuning without measurement is guesswork. Capture p50, p95, and p99 request latency before the change, apply settings, and re-run the same test under the same load. Latency work is intensely before/after; a single sysctl can look great in isolation and hurt real traffic because it moved a different queue.

Takeaway

Latency is queue depth, and the kernel lets you shrink the queues that matter. Apply a small, reversible set of sysctls, verify the backlog and scheduler, and judge every change by its before/after p99. To test tuning against production-shaped traffic, spin up a second Ubuntu instance on Netbay in under 60 seconds and compare both sides from netbayhosts.in.