Put nginx TLS in Front of a Local Model API

llama-server and Ollama should listen on 127.0.0.1. TLS, HTTP/2, and a domain name belong on nginx. That split keeps the model process simple and gives you a place for headers, timeouts, and a basic secret check. Netbay VPS instances are DDoS Protected at L3/L4, which is not an application firewall. You still need TLS and you still need to not publish port 8080. The datacenter is Lucknow; point your DNS A record at the VPS IPv4 and obtain a Let's Encrypt certificate as usual.

This is not a managed inference gateway. It is a reverse proxy in front of a CPU GGUF process.

Listen on loopback, proxy on 443

Install nginx and certbot. Obtain a certificate for the hostname before you flip the server block to SSL if you prefer the nginx plugin, or use certbot certonly and paste the paths. The upstream must be HTTP to 127.0.0.1:8080.

upstream llama_cpu {
    server 127.0.0.1:8080;
    keepalive 8;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name llm.example.com;

    ssl_certificate     /etc/letsencrypt/live/llm.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/llm.example.com/privkey.pem;

    client_max_body_size 1m;
    proxy_read_timeout 180s;
    proxy_send_timeout 180s;

    location /v1/ {
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header Connection "";
        proxy_buffering off;
        proxy_pass http://llama_cpu;
    }
}

proxy_buffering off matters for streamed completions. A 180 second read timeout covers a slow 7B Q4 turn on Intel Xeon Platinum; raise it only if you allow large max_tokens. client_max_body_size 1m stops someone pasting a novel into a CPU box that cannot hold the context anyway.

Auth is your problem

The model server may accept any Bearer token. Add a map or a simple header check in nginx so the public hostname is not an open proxy to your GGUF. A shared secret in Authorization is enough for a private tool. For a product, put a real app in front that already knows users.

location /v1/ {
    if ($http_authorization != "Bearer s3cret-rotate-me") {
        return 401;
    }
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_buffering off;
    proxy_pass http://llama_cpu;
}

if in nginx has caveats, but a single exact Bearer match is fine for a private endpoint. Rotate the secret. Do not commit it. HTTP/2 from clients still becomes HTTP/1.1 to llama-server, which is what you want.

Redirect 80 to 443. Close UFW or iptables for 8080 and 11434. ss -lntp should show those ports on 127.0.0.1 only. If Ollama or llama-server bound 0.0.0.0, fix the service environment and restart.

Health, limits, and what not to proxy

Proxy /health if you use it for uptime checks, but do not require the Bearer secret on that path if your monitor cannot store it; use a separate location from 127.0.0.1 or a firewall-restricted GET. Do not proxy the llama-server HTML UI to the world. Limit /v1/chat/completions. Return 413 on oversize bodies before they hit RAM.

Rate limiting with limit_req can stop a retry storm from filling your one-slot queue. A small burst is enough. Remember that each admitted request may hold a vCPU stack for a long time, so a "100 rps" limit is meaningless; think in concurrent generates, which should be one.

Streaming and SDKs

OpenAI clients that stream expect text/event-stream. nginx must not wait for the full body. proxy_buffering off is the main switch. gzip on streamed tokens is more trouble than it is worth; leave it off for /v1/. Idle connections should not be longer than your generate timeout.

DNS TTL can stay low until the certificate works. The VPS lives in Lucknow; pick a hostname you own. There is no second Netbay region to fail over to, so your HA story is a remote API backup, not a second city.

Takeaway

nginx terminates TLS and hides the model port. llama-server stays on loopback with a Q4 GGUF that fits RAM. Timeouts, body size, and a Bearer check are the whole security baseline. This is DIY serving, not a managed GPU gateway.

You can spin up Ubuntu 24.04 on Netbay in under 60 seconds, install nginx, and proxy a local model API — netbayhosts.in.