Proxy Performance Knobs: Timeouts, Buffers and Limits

Most proxy pain is not architecture — it is defaults. Nginx and HAProxy ship with conservative timeouts and modest buffer sizes so nothing explodes on first install, but those same defaults punish uploads, slow clients, and servers that genuinely need a few extra seconds. Understanding the handful of knobs below turns a proxy that times out mysteriously into one you can reason about.

The three timers you will actually tune

A proxied request has three phases, each with its own timer.

• **proxy_connect_timeout:** how long the proxy waits to open a TCP connection to the upstream. This should stay small; a healthy localhost backend connects in milliseconds. Leave the default around the five-second mark.
• **proxy_read_timeout:** the gap between two successive reads of the response body. For a normal API this is small, but for a slow report or a chat endpoint you must raise it or the proxy kills the request mid-stream.
• **proxy_send_timeout:** the gap between two writes back to the client. Slow mobile users on flaky networks are the reason this exists.

Buffers decide who carries big payloads

Two directives divide the work when a response is larger than memory.

• **proxy_buffering on** lets the proxy spool the response and drip it to a slow client, freeing the upstream socket early. Enable it unless your app depends on streaming semantics.
• **proxy_buffers, proxy_buffer_size** control how many memory buffers Nginx allocates per connection and the size of the header buffer. A large header from an oversized cookie is the usual reason responses 502 on a mis-sized deployment.

A Nginx config with sane production values

server {
    listen 80;
    server_name api.example.com;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_connect_timeout 5s;
        proxy_read_timeout  30s;
        proxy_send_timeout  30s;
        proxy_buffering on;
        proxy_buffer_size 8k;
        proxy_buffers 8 8k;
        proxy_max_temp_file_size 0;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

proxy_max_temp_file_size 0 tells Nginx to stop spilling responses to disk and keep them in RAM, which is right when your responses fit comfortably in memory and you care about speed over disk conservation. The emptied Connection header plus HTTP/1.1 enables keepalives to the upstream, so every request does not pay for a fresh TCP handshake.

Upstream keepalives and worker limits

Keepalives are the biggest single lever for latency. A reused upstream socket skips TCP slow-start, which on a localhost hop saves little, but the same idea matters at the process level: fewer connection churns means fewer TIME_WAIT sockets and lower load.

upstream api_up {
    server 127.0.0.1:8000;
    keepalive 32;
}

server {
    listen 80;
    server_name api.example.com;
    keepalive_timeout 65;

    location / {
        proxy_pass http://api_up;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

The keepalive value in the upstream block is the number of idle sockets Nginx maintains per worker process. Worker counts and Firewall connection tracking come next: on a busy box raise worker_connections and check your kernel's conntrack table, because an exhausted connection table looks like random timeouts.

sysctl net.netfilter.nf_conntrack_max
ss -s | grep -i tcp

Test with real payloads

Do not trust a single curl for tuning. Fire concurrent uploads and slow-drain clients, then watch the numbers move.

seq 1 50 | xargs -P 10 -I {} curl -o /dev/null -s   -w "%{http_code} %{time_total}
" http://localhost/  

Watch for the tail of the distribution — the slowest 5% of requests — because that is what proxy timeouts punish most. Apply one change at a time, measure, and move on.

Takeaway

Defaults are a starting point, not a target. Raise read and send windows for slow endpoints, keep connect tight, enable upstream keepalives, and size buffers to your response bodies. Measure the tail latency after every change, and only then move on. This kind of tuning assumes a box you control end to end — the argument for a plain VPS — and that is easy at netbayhosts.in.