TLDR: Every background daemon you ship needs a
RotatingFileHandler. One line. Five minutes. Your disk will thank you.
The Thing I Was Building
I've been running a background Python service I call the Apollo scanner — it triages incoming items with Claude, files tasks into Things 3 (my task manager), and keeps my calendar briefing sane.
It runs on a loop. All day. Every day.
And for a while, it logged to a plain file.
The Problem (That I Never Noticed)
Here's the thing about unbounded log files: they're completely fine in development.
You run the script, check the output, done. The file's a few kilobytes. No issue.
In production — on a daemon that's been running for weeks — that same file just… keeps growing. Forever. No ceiling. No cleanup. Until one day it isn't fine anymore.
I hadn't hit the wall yet. But I was heading toward it.
How I Actually Found This
My autonomous council (a set of sub-agents that does a rolling architectural review of my codebases) flagged it during a scanner hardening pass.
It caught the boring operational stuff. The kind of thing I'd never prioritize on my own because the scanner was working and I had real features to ship.
That's the part that actually matters to me. The fix was trivial. The fact that I had a system that would notice it before it became a 3am incident — that's the real win.
The Fix (One Line)
In scanner/src/main.py, I swapped the file handler:
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
log_path,
maxBytes=5 * 1024 * 1024, # 5MB
backupCount=3
)
That's it.
5MB per file, 3 backups. Total disk footprint: ~15MB, bounded forever. When the active log hits 5MB, Python rolls it to scanner.log.1, pushes .1 to .2, drops .3. Clean, automatic, zero maintenance.
Why This Is My Default Now
After I shipped the fix to the scanner, I looked at a separate RAG indexing pipeline and found the exact same problem waiting for me. Same fix: 5MB rotating logs × 3 backups.
Two services. Same quiet time bomb. Same one-liner defuse.
It's my default now. Every long-running Python service I touch gets RotatingFileHandler on day one. Not as an afterthought. Not when the disk alert fires.
The lesson isn't the log rotation. The lesson is what kind of review catches the thing you'd never prioritize yourself.