Agent Memory: Transcripts, Summaries, Vectors

Agents without memory repeat themselves. Agents with unbounded memory spend a dollar rereading a 40-turn transcript to answer a one-line question. Memory is a storage and truncation problem, not a personality. On a Linux VPS you have three layers that actually work: the live transcript for the current run, a rolling summary for the session, and an optional local index for older facts. You do not need a hosted vector product. SQLite on High-Speed SSD is enough for a single-box ops bot.

The failure mode to avoid is stuffing every journal line into the next prompt. Context windows are large now. Bills and attention are not. Models still get lost in the middle of a long trace.

Layer 1: The Transcript Is the Source of Truth

Write every message, tool call, and truncated observation to a JSONL file keyed by run_id. That file is how you debug. It is also how you resume after a process crash. Do not treat the in-memory messages array as durable. systemd can SIGKILL you.

Cap each observation. Cap the array you send to the API. A useful rule: send the system prompt, the user goal, the latest summary, and the last N turns (start with N=6). Everything else lives on disk.

import json
import os
import time

LOG_DIR = "/var/lib/ops-agent/runs"

def append_turn(run_id, event):
    os.makedirs(LOG_DIR, exist_ok=True)
    path = os.path.join(LOG_DIR, run_id + ".jsonl")
    event = dict(event)
    event["ts"] = time.time()
    with open(path, "a", encoding="utf-8") as handle:
        handle.write(json.dumps(event, ensure_ascii=False) + "\n")

def load_recent(run_id, n=6):
    path = os.path.join(LOG_DIR, run_id + ".jsonl")
    lines = open(path, encoding="utf-8").read().splitlines()
    events = [json.loads(line) for line in lines[-n:]]
    return events

The directory belongs to the agent user, mode 700. Rotate files with a timer, not with hope. A week of JSONL for a quiet bot is small. A verbose bot that tails logs without truncation will fill the disk; treat that as a bug in the tool, not in memory.

Layer 2: Rolling Summaries Beat Full Recall

After every K steps, ask the model to rewrite a 200-word state card: what the user wanted, what you already checked, what is still unknown, what you must not retry. Replace the previous summary. Do not append summaries forever; that recreates the original problem.

Keep the summary in a separate file so a new process can start without replaying tools. Include hashes of commands already run so the next loop does not restart nginx a second time.

SUMMARY_PROMPT = (
    "Rewrite the state card in under 200 words. Include goal, checks done, "
    "open questions, and tool names already used. No new plans."
)

def roll_summary(messages, old_summary):
    packed = [
        {"role": "system", "content": SUMMARY_PROMPT},
        {"role": "user", "content": "OLD: " + (old_summary or "(none)")},
        {"role": "user", "content": json.dumps(messages[-8:])[:12000]},
    ]
    data = chat(packed, tools=[])
    text = data["choices"][0]["message"].get("content") or ""
    return text[:1500]

def prompt_messages(goal, summary, recent):
    out = [
        {"role": "system", "content": "You operate one Linux VPS. Use tools. Do not repeat done work."},
        {"role": "user", "content": "GOAL: " + goal},
        {"role": "user", "content": "STATE: " + (summary or "(empty)")},
    ]
    out.extend(recent)
    return out

Summaries hallucinate. That is expected. Mitigate by putting hard facts in a structured side file the summarizer does not rewrite: units restarted, last HTTP status, last error string. The model can paraphrase the narrative. It must not be the only copy of the integers.

Layer 3: Local Vectors Are Optional

Keyword search over JSONL handles most ops recall: unit names, HTTP paths, error substrings. Add embeddings only when you have many run files and operators ask "what did we do last time nginx 502'd". Even then, generate embeddings via the same remote model API and store them in SQLite. Do not invent an object store. Do not stand up a second cluster.

A row is: run_id, ts, text, embedding blob. Query with cosine in Python for a few thousand rows. When that is too slow, you have a research problem, not an ops-bot problem.

Watch privacy. Transcripts contain hostnames and maybe customer URLs. Do not ship them to a third-party memory SaaS. The VPS in Lucknow, sitting on Intel Xeon Platinum with High-Speed SSD, is the memory system. Encrypt the disk, restrict the user, and grep the JSONL when you are unsure.

Rotation is part of memory. A timer that gzip's JSONL older than seven days and deletes embeddings older than thirty is operations, not ML. If you cannot say where last month's traces live, you will not be able to evaluate the agent either. Keep the current run's transcript hot, the summary hot, and everything else cold and greppable. Keyword search over gzip is still faster than a confused 80k-token prompt.

Takeaway

Memory is transcript, summary, and maybe SQLite. Keep facts out of the prose card, and never paste the whole run into the next call. Store the files on a Netbay VPS in Lucknow and inspect them with ordinary Unix tools at netbayhosts.in.