Structured JSON Output That Agents Can Parse

Tool-calling loops die on the second step more often than the first: the model returned almost-JSON, your parser threw, and the agent either retried the entire plan or executed nothing. "Please respond in JSON" is not a parser. You need a schema, a decode path that strips accidental fences, a validator that rejects extra keys, and a nack that does not re-prompt forever.

This post is the decode-and-validate layer you put between the HTTP response and any tool. It runs on the VPS next to the worker. No special Netbay LLM product is required — there is not one — only a model API that can be asked for JSON and a few dozen lines of Python. Lucknow DC01 High-Speed SSD is where you log the raw text when validation fails, because that log is how you tighten the schema.

Ask for JSON, then distrust it

Use the provider's JSON mode or response schema if it exists. Still validate locally. Providers guarantee an object-shaped string, not your enum, not your maxLength, and not the absence of a trailing "hope this helps." Local validation is the contract; the provider flag is a hint that reduces retries.

# /opt/agents/jsonout.py
import json, re
from jsonschema import Draft202012Validator

TICK = chr(96)
FENCE = re.compile("^" + TICK * 3 + r"(?:json)?s*|s*" + TICK * 3 + "$", re.I | re.M)

def decode(text):
    text = text.strip()
    text = FENCE.sub("", text).strip()
    start, end = text.find("{"), text.rfind("}")
    if start < 0 or end <= start:
        raise ValueError("no object")
    return json.loads(text[start:end + 1])

def validate(obj, schema):
    Draft202012Validator(schema).validate(obj)
    return obj

The fence strip is there because models still wrap objects even in JSON mode. The find/rfind pair drops a leading sentence. json.loads still fails on single quotes and trailing commas; treat those as nacks, not as an excuse to eval.

Set additionalProperties to false on every object. If you do not, the model will add "note" and your worker will ignore it until "note" contains an instruction you accidentally concatenate into the next prompt.

Schema first, prompt second

Write the schema as if a hostile intern will fill it. Enums instead of free strings for anything that routes. maxLength on summaries. integer for cents, never a dollar string. No optional fields that change control flow; if a field matters, require it.

# /opt/agents/schemas/triage.py
TICKET = {
    "type": "object",
    "additionalProperties": False,
    "required": ["label", "severity", "summary", "citations"],
    "properties": {
        "label": {
            "type": "string",
            "enum": ["billing", "outage", "how_to", "bug", "abuse", "human"],
        },
        "severity": {"type": "string", "enum": ["p0", "p1", "p2", "p3"]},
        "summary": {"type": "string", "minLength": 20, "maxLength": 800},
        "citations": {
            "type": "array",
            "maxItems": 5,
            "items": {"type": "string", "maxLength": 120},
        },
    },
}

The prompt then names the fields once: "Return one JSON object that matches the ticket schema. citations are SOURCE ids from the blocks, or empty." Do not paste the schema twice and the policy three times. Redundant instructions are how models invent a sixth label.

Retry policy for bad JSON

One repair retry is enough. The repair prompt gets the validator error string and the original text, and is told to return only the object. If the second decode fails, dead-letter the job with both raw strings. A third retry trains you to accept slop and trains the model to ramble.

Do not repair by walking the AST and "fixing" types. Coercing "p0" from "P0!!!" is fine in your own adapter if you document it. Coercing a missing citations array into [] hides a retrieval miss. Fail that case.

Log three artefacts on every nack: request id, raw text (capped at 8 KiB), and the exception message. That JSONL on disk is more useful than a dashboard of "JSON errors." After a week you will know whether the model is fencing, adding comments, or ignoring enums — three different prompt bugs.

Typed outputs are how workers stay narrow

The supervisor in the triage post only works if label is an enum. The handoff queue only works if kind is an enum. Structured output is not a nicety for pretty logs; it is the type system of an agent that is otherwise a string blender. If you cannot write a schema for a step, that step is not ready to be a worker.

On the box, keep schemas in /opt/agents/schemas and version them. When you add a label, you ship a schema change, a routing table change, and a prompt change together. A prompt-only change that mentions a new label is a bug.

Takeaway: provider JSON mode is a hint, local schema is the law, and one repair retry is charity. Log the raw failures. You can drop this decoder next to your workers on Ubuntu 24.04 in Lucknow DC01 from Netbay — netbayhosts.in.