Logging and Log Rotation for Self-Hosted Apps

The apps in your homelab are quietly producing logs all day, and left alone they will eventually fill the disk and stop everything. Logging done well has two halves: rotation keeps storage bounded, and centralization means you can actually read logs across services in one place when you are debugging. For a self-hoster the pragmatic combo is Docker's log rotation options plus a log driver or a simple aggregator so you query one place instead of hopping between container consoles.

Rotation Before It Is a Problem

Docker containers write to the JSON-file log driver by default, and unbounded that file grows forever. The fix is declarative rotation on the daemon or per-container. Set both a maximum file size and a file count so the on-disk cost is predictable — the classic silent-killer fix for a busy host.

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "20m",
    "max-file": "5"
  }
}

That daemon.json caps each container's log history at five files of 20 MB, so the worst case is 100 MB per container no matter how noisy the app. The cleanup is automatic and requires no cron or manual truncation.

Choose the Driver That Matches

The json-file driver is fine for rotation alone, but if you want to search across services you move to either a driver that forwards logs or an aggregator container you point them all at. A common lightweight aggregator in the homelab world is a tool that ingests structured logs and exposes a search API. Rotate at the source so the aggregator never receives an unbounded stream.

services:
  loki:
    image: grafana/loki:latest
    restart: unless-stopped
    volumes:
      - loki-data:/loki
    ports:
      - "3100:3100"
  app:
    image: some/web-app:latest
    logging:
      driver: loki
      options:
        loki-url: "http://loki:3100/loki/api/v1/push"
        max-size: "10m"
        max-file: "3"

Grafana Loki is a log aggregator designed to be cheap and beginner-friendly, and its Docker driver forwards each container's lines to it, while rotation still bounds each side of the pipeline. You then browse every service's logs from one Grafana or Loki UI.

Query Like You Grep

The payoff of aggregation is queries that span the whole stack. Loki's LogQL borrows a grep-like feel: pick a label or container and add a string filter, exactly the way you already mentally answer "what was that error, and in which service."

{container="web"} |= "connection refused"

That query pulls every line mentioning connection refused from the web container — and swapping the label lets you search any other service without opening a separate console. When the whole stack logs to one place, a "post-mortem" becomes a single search instead of an archaeology dig across terminals.

Set a Retention Policy, Not a Fire-and-Forget

Aggregation has a cost: the longer you keep logs, the more disk they consume, and an unbounded retention slowly turns a tidy setup into the very disk-pressure problem rotation was meant to solve. Decide deliberately how much history matters. For debugging you almost certainly care about the last day or two intensely and the last month loosely; a year of verbose application logs is rarely worth the disk. Encode that decision so growth stays flat.

limits_config:
  retention_period: 336h   # two weeks of searchable history

A two-week retention window keeps recent debugging history sharp while bounding the long-term footprint. Long before that window, the per-source rotation your Docker driver already enforces keeps the live stream from ballooning. Between the two, logging has a ceiling at every stage of the pipeline instead of an accidental one at the point of disk-full.

Make Logs Structured at the Source

Searching plain prose works, but searching key-value logs is better. When your own application is emitting logs, prefer structured output — one log line per event with fields the aggregator can index — over human-sentence fragments. It turns a vague logql string search into a precise field filter, which is dramatically faster to reason about when you are chasing a bug across three containers. Most language logging libraries emit JSON with a one-line change; the effort pays off the first real incident.

Takeaway

Logging is boring until it saves you. Rotate at the source so the disk stays bounded, then aggregate so you search one surface when it matters, and you will thank yourself at 2 a.m. during an incident. A tidy logging setup runs comfortably on a Netbay Lucknow VPS — set up your host and logging stack at netbayhosts.in.