Token and Cost Budgets for Long-Running Agents

Long-running agents do not fail loudly. They fail as a line item. A loop that retries a flaky health check every twenty seconds will keep paying for a full transcript plus tools until someone notices the invoice. Budgets are not a finance nicety. They are a stop condition equal to "the model emitted a final answer".

You need three meters: steps, tokens, and money. Any one of them should halt the run. A daily cap should halt the service. The code lives next to the loop on your VPS. The model API will not protect you; it is happy to keep answering.

Count What the Provider Already Returns

Most Chat Completions responses include usage.prompt_tokens and usage.completion_tokens. Trust that when present. Fall back to a local estimator if a provider omits it. Add the tool observation length to the next prompt yourself; that is usually the bulk of a long run.

Store counters in a file, not only in RAM, so a crash-restart cannot reset the daily cap. systemd Restart=on-failure is otherwise a way to launder a budget.

import json
import os
import time

STATE_PATH = "/var/lib/ops-agent/budget.json"
MAX_STEPS = 8
MAX_TOKENS_PER_RUN = 24000
MAX_USD_PER_DAY = 5.0
USD_PER_1K_IN = 0.15
USD_PER_1K_OUT = 0.60

def load_state():
    if not os.path.exists(STATE_PATH):
        return {"day": "", "usd": 0.0, "runs": 0}
    return json.loads(open(STATE_PATH, encoding="utf-8").read())

def save_state(state):
    tmp = STATE_PATH + ".tmp"
    with open(tmp, "w", encoding="utf-8") as handle:
        json.dump(state, handle)
    os.replace(tmp, STATE_PATH)

def day_ok(state):
    today = time.strftime("%Y-%m-%d", time.gmtime())
    if state.get("day") != today:
        state["day"] = today
        state["usd"] = 0.0
        state["runs"] = 0
    if state["usd"] >= MAX_USD_PER_DAY:
        raise RuntimeError("daily USD cap reached")
    return state

def add_usage(state, prompt_tokens, completion_tokens):
    usd = (prompt_tokens / 1000.0) * USD_PER_1K_IN
    usd += (completion_tokens / 1000.0) * USD_PER_1K_OUT
    state["usd"] = state.get("usd", 0.0) + usd
    save_state(state)
    return usd

Put your own rates in EnvironmentFile. The numbers above are placeholders so the code is obvious. Read them from env so a price change is not a commit. Round up. Under-counting is how you blow the cap "by accident".

Truncate Before You Hit the Wall

A budget that only errors after the 24k token call has already spent the money. Truncate observations to a few kilobytes. Drop old turns in favor of the summary. Refuse to attach a journal dump. If a tool returns more than 4 KB, the tool is wrong.

def gated_chat(messages, tools, run):
    if run["steps"] >= MAX_STEPS:
        raise RuntimeError("step budget exceeded")
    if run["tokens"] >= MAX_TOKENS_PER_RUN:
        raise RuntimeError("token budget exceeded")
    data = chat(messages, tools)
    usage = data.get("usage") or {}
    prompt_tokens = int(usage.get("prompt_tokens") or 0)
    completion_tokens = int(usage.get("completion_tokens") or 0)
    run["steps"] += 1
    run["tokens"] += prompt_tokens + completion_tokens
    run["usd"] += add_usage(run["day_state"], prompt_tokens, completion_tokens)
    return data["choices"][0]["message"]

def clip_observation(text, limit=4000):
    if text is None:
        return ""
    if len(text) <= limit:
        return text
    return text[:limit] + "\n...[truncated %s chars]..." % (len(text) - limit)

When the step budget fires, return the last observation to the operator, not a fake success. A silent cap looks like the bot finished the work.

Long-Running Does Not Mean Always-On Chat

A bot that polls every minute should not send the full transcript each poll. Send a tiny heartbeat prompt: the latest metric and the state card. Most polls should exit after one model call with no tools. If nothing is wrong, spend a few hundred tokens, not twenty thousand.

Use a systemd timer, not a tight Python sleep, so you can stop the unit without killing a wedged child. Log usd and tokens to journald with a structured prefix you can grep. Intel Xeon Platinum will not be your bottleneck; the API latency will. High-Speed SSD holds the budget file. Lucknow DC01 plus L3/L4 DDoS filtering on any public webhook keeps the trigger path boring.

If you need more headroom, raise MAX_USD_PER_DAY after you look at a week of traces. Do not remove the cap. An uncapped agent is a process with a credit card.

Alert when you hit 50 percent of the daily cap, not only when you hit 100. A quiet morning of retries is cheaper to stop than an afternoon of "the bot is still thinking." Ship the budget.json fields to journald every run: usd, tokens, steps, finish reason. A one-line grep then tells you whether yesterday was a model regression or a hung tool.

Heartbeat polls should use a cheaper model if your provider allows routing. Investigation runs can use a stronger one under a tighter MAX_STEPS. That split is a budget control, not a quality cult. Either way the numbers live on the VPS in Lucknow, not in a vendor dashboard you remember to open after the invoice.

Takeaway

Steps, tokens, and a durable daily USD cap are stop conditions. Truncate first so the cap is not a surprise invoice. Run the meter on a Netbay VPS in Lucknow and grep the journal when the number jumps — netbayhosts.in.