Support Classifier Agent That Routes Tickets

Support inboxes rot when every message lands in one folder and a human tags it three hours later. A classifier agent is not a chatbot. It reads a ticket, emits a queue name, a handful of extracted fields, and a confidence score, then a router moves the ticket or leaves it for a person. Wrong routing is worse than slow routing, so the useful product is the fallback: below 0.7 confidence, do nothing except tag needs-human. For a VPS provider style inbox you will see billing, SSH access, DNS, abuse, and app-bug over and over. Those five queues plus other are enough. Do not invent a sixth because the model felt creative.

Queues are a closed enum

Give the model a fixed list: billing, ssh, dns, abuse, app-bug, other. Map those to your ticketing labels or IMAP folders. If the model returns capacity-planning or refund-special, reject the payload and tag needs-human. Closed enums are how you keep SLA dashboards stable. Describe each queue in one sentence in the prompt: ssh is cannot log in, key, two-factor, lockout; dns is records, rDNS, hostname not resolving; abuse is inbound reports about a VPS you host; billing is invoices and plan changes; app-bug is software the customer runs on the VPS.

You are classifying mail about hosts in Lucknow DC01. Do not route based on imagined other regions. A ticket that says the site is slow may be app-bug, ssh (cannot even get in), or a disk-full watchdog alert forwarded by a customer. If the body contains both an invoice ID and an SSH snippet, prefer the reason they wrote in the first paragraph, then lower confidence.

Extract fields with regex, then let the model fill gaps

Hostname, IPv4, and invoice-looking tokens should be pulled with regex before the model runs. The agent may add a plan hint or a unit name, but it must not overwrite a regex hit. Redact passwords, private keys, and cookie headers before the prompt. If someone pasted an ed25519 private key, drop the ticket into ssh with confidence 1.0, strip the key from the stored body, and page a human to rotate.

import re, json

QUEUES = {"billing", "ssh", "dns", "abuse", "app-bug", "other"}
HOST = re.compile(r"(?:[a-z0-9-]+.)+[a-z]{2,}", re.I)
IPV4 = re.compile(r"(?:d{1,3}.){3}d{1,3}")
INVOICE = re.compile(r"INV-?d{4,}", re.I)

def pre_extract(text):
    return {
        "hostnames": HOST.findall(text)[:5],
        "ipv4": [ip for ip in IPV4.findall(text) if not ip.startswith("127.")][:5],
        "invoices": INVOICE.findall(text)[:3],
    }

def accept(doc, extracted):
    if doc.get("queue") not in QUEUES:
        return {"queue": "other", "confidence": 0.0, "reason": "bad enum"}
    conf = float(doc.get("confidence") or 0)
    if conf < 0.7:
        return {"queue": "needs-human", "confidence": conf, "fields": extracted}
    fields = dict(extracted)
    fields["plan_hint"] = doc.get("plan_hint")
    return {"queue": doc["queue"], "confidence": conf, "fields": fields}

The accept() function is the router contract. Everything else is decoration. Persist the raw model JSON next to the accepted route so you can audit a misfile without re-running the model.

Route, do not reply

Do not let this agent send mail to customers. Auto-replies from a classifier are how you tell an abuse reporter "thanks for your billing question." The only side effect is a label, an assignee, and maybe a Slack ping. Abuse always pings, even at 0.95 confidence. SSH lockouts can ping if the customer is on a paid plan and the ticket is during business hours in India. Billing can wait in the queue. App-bug should include the extracted hostname so the engineer SSHs to the right Lucknow VPS, not a guess.

# /etc/support-classifier/queues.yaml
queues:
  billing:
    label: billing
    ping: false
  ssh:
    label: ssh-access
    ping: true
    ping_hours: "09:00-21:00 Asia/Kolkata"
  dns:
    label: dns
    ping: false
  abuse:
    label: abuse
    ping: true
    ping_always: true
  app-bug:
    label: app-bug
    ping: false
  other:
    label: needs-human
    ping: false
confidence_min: 0.7

Keep that YAML in git. Changing a ping flag is an ops change, not a prompt change. Measure precision per queue weekly from a 50-ticket sample. If ssh precision drops below 90 percent, raise the threshold rather than adding adjectives to the prompt.

Fixtures are old tickets with the label you wish you had

Export fifty real tickets, strip PII, and store expected queue plus extracted hostname. Run them on every prompt edit. Include traps: a billing thread that quotes an old SSH error, an abuse report that mentions DNS, a customer who pastes journalctl and an invoice in one mail. The classifier should hold those. Hardware facts belong in runbooks, not here: Xeon Platinum, High-Speed SSD, L3/L4 DDoS filtering do not change the queue unless the customer is asking about a DDoS event, which is app-bug or abuse depending on who wrote.

Takeaway

A support classifier emits a closed queue, extracted hosts, and a confidence gate. It labels; it does not talk to customers. Run the worker on a Netbay Lucknow VPS beside your mail fetcher — netbayhosts.in will have Ubuntu 24.04 up in under 60 seconds.