Code-Review Agent That Posts PR Comments

Most review bots die because they nag about commas. A useful code-review agent for ops-heavy repos looks at systemd units, firewall rules, deploy scripts, and secret-shaped strings, then posts one GitHub review with a handful of line comments. It does not approve. It does not request changes on formatting. It fails closed if the diff is huge. The audience is the same people who SSH to a Lucknow VPS after merge; the comments should sound like that person, not like a linter with an API key.

Scope the files or the bot will review CSS

Walk the PR files and keep systemd units, nginx and Caddy configs, shell under scripts/, Dockerfiles, GitHub workflows, SQL, and anything named authorized_keys or sudoers. Drop lockfiles, generated CSS, and vendor trees. Cap the remaining diff at 200 KB. If the cap trips, post a single issue comment that says the diff was too large and skip line comments. That is better than half-reviewing a 2,000-file refactor.

Give the model a checklist, not a personality: private keys in the diff, AWS-style access keys, passwords in unit Environment=, ExecStart as root when the app does not need it, Binding to 0.0.0.0 when localhost would do, missing After=network-online.target, curl | sh, unpinned actions tags, GRANT ALL, and rm -rf without a dry-run. Each finding needs path, line, severity (block or note), and a one-sentence why. Max eight findings. If the model returns twenty nits, take the eight highest severity and drop the rest.

Post one review, never a comment storm

GitHub's pull request review API accepts a list of comments and an event. Use COMMENT, not APPROVE, not REQUEST_CHANGES. REQUEST_CHANGES from a bot trains people to ignore the red X. APPROVE from a bot trains people to skip humans. A single review with inline comments is readable. Ten separate issue comments are noise.

import json, urllib.request

def post_review(token, repo, pr, commit_id, findings):
    comments = []
    for f in findings[:8]:
        comments.append({
            "path": f["path"],
            "line": int(f["line"]),
            "body": f["severity"].upper() + ": " + f["why"],
        })
    body = {
        "commit_id": commit_id,
        "event": "COMMENT",
        "body": "Ops review agent: secrets, units, firewall, SQL grants. Not a style bot.",
        "comments": comments,
    }
    req = urllib.request.Request(
        "https://api.github.com/repos/" + repo + "/pulls/" + str(pr) + "/reviews",
        data=json.dumps(body).encode(),
        headers={
            "Authorization": "Bearer " + token,
            "Accept": "application/vnd.github+json",
            "X-GitHub-Api-Version": "2022-11-28",
        },
        method="POST",
    )
    with urllib.request.urlopen(req) as resp:
        print(resp.status)

The token is a fine-grained PAT or a GitHub App installation token with pull-requests write, stored in Actions secrets, never in the prompt. Deduplicate: if the bot already reviewed this commit SHA, do not review again. On synchronize, review the new commit only.

Deterministic scanners go first

Run gitleaks or a regex for BEGIN OPENSSH PRIVATE KEY and for AKIA-style prefixes before the model. Those hits are block severity and do not need an LLM. The model then looks at the rest: systemd sandboxing, UMask, ProtectSystem, whether a unit after a binary rename still points at the old path, whether a firewall script drops SSH before allowing it. That last one takes down a Lucknow VPS until you use the console. Say so in the comment.

# CI step before the model
set -euo pipefail
git diff origin/main...HEAD -- "*.service" "*.timer" "scripts/"   "Caddyfile" "nginx.conf" "*.sql" ".github/workflows/" > /tmp/ops.diff
wc -c /tmp/ops.diff
if grep -n "BEGIN OPENSSH PRIVATE KEY" /tmp/ops.diff; then
  echo "block: private key in diff"
  exit 2
fi
python3 tools/review_agent.py /tmp/ops.diff

Keep the agent on the CI runner, not on the production VPS. The VPS still runs Intel Xeon Platinum with High-Speed SSD and L3/L4 DDoS filtering; it does not need a GitHub token. If a finding is about opening port 5432 to the world, that is a block even when the rest of the PR is a docs fix.

Measure silence, not volume

A good week of this bot is three comments, all correct. Track thumbs-down reactions and "not useful" replies. If more than 20 percent of comments are downvoted, shrink the checklist. Add fixtures: a unit missing NoNewPrivileges, a workflow using a floating action tag, a SQL file with GRANT ALL ON ALL TABLES. Expected output is the path and severity, not the essay. Humans still own merge, including on Ubuntu 24.04 deploy repos that ship to Netbay.

Takeaway

A review agent is a scoped diff, a short ops checklist, and one GitHub review with event COMMENT. It catches secrets and systemd traps; it does not approve. Run it in Actions and keep the token off the VPS. When you want a host to deploy those PRs onto, Netbay Lucknow Ubuntu 24.04 is up in under 60 seconds at netbayhosts.in.