Store Model API Keys as Secrets on a Linux VPS

A production LLM app is usually a small HTTP service that holds a handful of third-party keys: OpenAI, Anthropic, an embeddings vendor, maybe a search API. Those keys are worth more than the application source. Anyone who copies OPENAI_API_KEY from a world-readable dotenv file can spend your quota, scrape your prompts, and leave a billing surprise. On a VPS you operate, secrets belong in a root-owned file that systemd injects into a locked-down unit. They do not belong in git, in a compose file committed to the repo, or in a paste that journald will replay for the next operator.

This post is about model API keys specifically. The same pattern works for any secret, but model keys have a habit of showing up in prompt logs, in debug dumps of the vendor client, and in error messages that include the Authorization header. You will put the key on disk once, load it as an environment variable, never print it, and rotate it without rebuilding the app.

Why a dotenv next to the app fails

A dotenv file beside server.js is convenient on a laptop. On a Lucknow VPS it is a leak waiting to happen. Deploy scripts rsync the tree. Backups copy /srv/ai-app. A mis-set umask leaves the file mode 0644. journald captures stdout from a process that printed process.env on boot just to confirm it loaded. CI that SSHes in and cats the file to verify the deploy has now copied the key into another log.

Keep application config in the repo: model names, max tokens, timeouts, which vendor is primary. Keep credentials in /etc/ai-app.env owned by root, mode 0640, group matching the service user. systemd EnvironmentFile reads that file before ExecStart. The app process sees OPENAI_API_KEY in its environment and nowhere else.

Do not source the file in a shell wrapper. A wrapper that does set -a, dots the env file, then exec node looks the key in process listings if xtrace is on, and it requires the service user to read the file. EnvironmentFile is parsed by PID 1 as root, then the variables are passed into the exec. The usual production choice is 0640 root:aiapp so operators in that group can rotate the file without becoming root, and the unit still lists EnvironmentFile.

Netbay instances in Lucknow DC01 boot Ubuntu in under a minute with Intel Xeon Platinum cores, High-Speed SSD, and L3/L4 DDoS filtering on the public NIC. None of that helps if the first git push includes sk-live. Treat the VPS as a place that will be snapshotted, copied, and tailed.

Write the env file and the unit

Create a system user, the env file, and a unit that refuses to start if the key is missing.

sudo useradd --system --home-dir /srv/ai-app --shell /usr/sbin/nologin aiapp
sudo mkdir -p /srv/ai-app /srv/ai-app/var
sudo chown -R aiapp:aiapp /srv/ai-app
sudo install -o root -g aiapp -m 0640 /dev/null /etc/ai-app.env
printf '%s\n' 'OPENAI_API_KEY=sk-replace-me' 'ANTHROPIC_API_KEY=sk-ant-replace-me' 'MODEL_ROUTER_TOKEN=replace-me' | sudo tee /etc/ai-app.env
sudo chmod 0640 /etc/ai-app.env
sudo chown root:aiapp /etc/ai-app.env
stat /etc/ai-app.env

Then the unit. There is no Environment line with a real key. EnvironmentFile is the only path that carries secrets.

[Unit]
Description=AI app HTTP frontend
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=aiapp
Group=aiapp
WorkingDirectory=/srv/ai-app
EnvironmentFile=/etc/ai-app.env
Environment=NODE_ENV=production
ExecStart=/usr/bin/node /srv/ai-app/server.js
Restart=on-failure
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/srv/ai-app/var
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6

[Install]
WantedBy=multi-user.target

ProtectHome stops a compromised process from reading /root history or an operator laptop key dumped under /home. RestrictAddressFamilies still allows outbound HTTPS to the model vendor. If the app only talks to loopback plus 443, that is enough.

Stop the key from landing in logs

Most leaks are not /proc tricks. They are log lines. Vendor SDKs include request headers in some error objects. A debug logger dumps the outgoing fetch options. nginx access logs will not have the key if you never send it to nginx; keep Authorization on the server-to-vendor hop only.

In the app, read the key once at boot, fail closed if it is missing or still the placeholder, and never interpolate it into a string that might be logged.

const key = process.env.OPENAI_API_KEY || '';
if (key.length < 20 || key.indexOf('replace-me') !== -1) {
  console.error('OPENAI_API_KEY is missing or still a placeholder');
  process.exit(1);
}

function redact(value) {
  if (!value) return '';
  if (value.length < 8) return '***';
  return value.slice(0, 4) + '...' + value.slice(-3);
}

async function complete(prompt) {
  const res = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: 'Bearer ' + key,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: prompt }],
    }),
  });
  if (!res.ok) {
    console.error('vendor status', res.status, 'key', redact(key));
    throw new Error('upstream model failed');
  }
  return res.json();
}

Never log the Authorization header. Never log the full environment. If you dump config at startup, print redact(key) so an on-call engineer can confirm which key prefix is live without copying the secret into chat.

Rotate without a rebuild

Keys get leaked. The vendor dashboard is the source of truth: create a new key, put it in /etc/ai-app.env, run systemctl restart ai-app, confirm completions still work, then revoke the old key. Do not revoke first unless you like a ten-minute outage. Keep a paper trail of the prefix and the date in a ticket, not the secret itself.

If two processes share the file, restart both. A worker and an HTTP frontend both read EnvironmentFile. Restarting only nginx will not pick up the new key because nginx never had it. Restart the units that consume the file.

The takeaway: treat model API keys as production credentials, not config. Root-owned EnvironmentFile, a nologin user, no key in git, no key in logs, and a rotation that is a file edit plus a restart.

You can spin up an Ubuntu 24.04 instance on Netbay in under 60 seconds and follow along — netbayhosts.in.