Throttling, Budgets, and Abuse Detection for Public APIs

A public API is an open door, and open doors attract both curiosity and abuse. You do not want to over-block customers, but you do want to stop a single caller from exhausting your capacity or scraping your data. The answer is a combination of throttling, spend budgets, and abuse detection tuned to distinguish traffic from attacks.

Throttling: the blunt instrument

Rate limiting caps requests per window — say 60 per minute per key or per IP. A fixed-window counter is easy but lets bursts at the window edge double a caller's rate. Sliding-window or token-bucket algorithms smooth that out and are still cheap to evaluate. When a caller crosses the limit, return '429 Too Many Requests' with a 'Retry-After' header so well-behaved clients back off instead of hammering.

Throttling stops the *volume* problem. It does not, on its own, distinguish a legitimate heavy customer from an attacker — that is the job of detection.

Budgets: throttle in cost, not just count

Some calls are far more expensive than others — a search across millions of rows is not the same cost as a health check. Fixed request counts treat them identically, letting a caller drain your spend through the expensive path even under a per-request limit. A **budget** assigns each operation a cost and meters cumulative cost per key per window.

To build one, give every route a unit cost proportional to the real work it does — a cheap lookup might cost 1 unit, a heavy analytics query 50, an export job 200 — and charge that cost when the request is served. Then decide whether the budget resets on a fixed clock, which is easiest to communicate, or uses a sliding window, which is smoother under spiky but legitimate traffic. Send the current remaining budget back in a response header so well-behaved clients can pace themselves, and be transparent about what happens when the budget depletes rather than leaving callers to discover the 429 by accident.

A token-bucket limiter, in Python

import time, threading

class TokenBucket:
    def __init__(self, rate, capacity):
        self.rate = rate
        self.capacity = capacity
        self.tokens = capacity
        self.updated = time.monotonic()
        self.lock = threading.Lock()

    def consume(self, cost=1):
        with self.lock:
            now = time.monotonic()
            self.tokens = min(
                self.capacity,
                self.tokens + (now - self.updated) * self.rate,
            )
            self.updated = now
            if self.tokens < cost:
                return False
            self.tokens -= cost
            return True

Detecting the differences between traffic and abuse

Budgets stop the bleeding; detection finds *who* is the problem. Watch for signals that batch rate limits cannot see:

• One account walking every endpoint at near-maximum speed (scraping).
• Many keys from one network or many logins from one IP (credential stuffing).
• A client ignoring '429' and 'Retry-After' (uncooperative, often automated).
• Requests concentrating on a single expensive endpoint.

Route suspicious callers to a stricter tier or a challenge rather than instantly banning a real customer. The goal is to slow abuse, not to break a customer who just had a busy hour.

Whatever signals you choose, keep the detection loop tight: observe a window of events, score them, act on the score, and re-score after your action so you can confirm the behavior actually changed. Deliberately tune against your own normal traffic before you take it live, or you will spend your first weeks throttling your most enthusiastic legitimate customers while the real attackers sail through under the noise. And always pair a score with a human review path — a threshold that fires an alert is worth more than one that silently bans, because the former teaches you something and the latter hides the failure until a customer complains.

An abuse-scoring stride

function score(events) {
  let s = 0;
  const byIp = new Map();
  for (const e of events) {
    byIp.set(e.ip, (byIp.get(e.ip) || 0) + 1);
    if (e.status === 429) s += 4;         // ignored throttle
    if (e.path === "/v1/expensive") s += 3;
  }
  const maxPerIp = Math.max(...byIp.values());
  if (maxPerIp > 500) s += 5;              // burst from one source
  return s;
}

Takeaway

Rate limits stop floods, cost budgets stop expensive-path drains, and scoring separates the repeat offender from the merely busy customer. Combine all three so legitimate traffic stays smooth while abuse gets throttled into irrelevance. Run and tune this stack on a Netbay VPS, from DC01 in Lucknow — netbayhosts.in.