TLDR: Any "learn from me" feature needs two layers — an instant deterministic one and a slow LLM one — both reading the same persisted store. Build both. Never conflate them.

the setup

I built Apollo Dashboard (my personal command-center, a local Python app that runs entirely offline on 127.0.0.1) to triage my inbox every hour using Claude Haiku.

Action, FYI, or noise. Three buckets. Simple enough.

But I wanted it to learn.

If I 👎 a newsletter, I don't want to keep seeing it. If I 👍 a client, I want them surfaced every time. So I wired up thumbs on every email row and wired those thumbs to /api/email-feedback.

The first version worked. Technically.

the problem with version one

The only thing that acted on the thumbs was the hourly Haiku re-triage.

Which meant: if I gave feedback and closed my laptop in the next 59 minutes, the signal was gone. Never flushed. Just gone.

And even if I left the server running — a full hour before my feedback did anything visible? That's not a feedback loop. That's a suggestion box with a very slow reader.

I stopped mid-build and made myself a note: "Ingest of learning should happen immediately, in case I close the server before 1 hour."

the tempting wrong fix

My first instinct was to just re-run the full LLM triage pipeline on every thumb click.

Instant. Done.

Except… no. Haiku post-June-15 is metered. Firing a model call on every button press is slow, spammy, and expensive. And on a local server that's supposed to be zero-cost to run — wrong move.

I needed immediacy without the model call.

the two-layer fix

The insight: immediate feedback and smart generalization are different jobs.

Split them.

Layer 1 — instant and deterministic. The moment you click 👎, the signal persists atomically to disk (tmp file + os.replace — no half-writes, no crash loss). At render time, plain Python code hides the row, demotes the sender, boosts the 👍'd one. Zero model calls. Zero cost. You see the effect immediately.

Layer 2 — slow and smart. The same persisted store gets read by Haiku at the next batch run. It pulls the category tags and example subjects and biases the triage prompt — "this user 👍'd emails like these, 👎'd emails like these, calibrate accordingly." The model generalizes to new senders it's never seen.

Two layers. One store. Both necessary.

the proof

A routine newsletter from a 👎'd sender got demoted — but its "payment failed, account suspends today" sibling broke through.

That's the smart-bias working. Not a dumb mute. An actual signal that understood the difference between newsletter noise and urgent account state from the same company.

A 👍'd client surfaced immediately on the next page load.

Exactly what I wanted.

why this matters to me

Every time I build a "learn from me" feature — thumbs, ratings, mute/pin, anything — I default to one layer now.

Either the instant-deterministic one (feels broken without it) or the slow-LLM one (feels dumb without it).

You need both. They're not alternatives. The fast one makes it feel like it works. The slow one makes it actually work on things you haven't seen yet.

Ship one, and your users will think it's broken or stupid.

P.S. Tell the user which effects are instant vs next-batch — out loud, in the UI or the docs. An empty surface after a thumb reads as "broken." "That takes effect next triage run" reads as "working as designed."