TLDR: Two required gates before any auto-fix lands: match on stable content identity (not LLM-regenerated labels), then check "is this actually a defect — or did I choose this on purpose?"

The Setup

I built a system I'm calling the fault miner — it scans my Claude Code transcripts (my AI agent is Apollo), finds recurring corrections and advisor catches, classifies them with Sonnet (an Anthropic model I use as a sub-classifier), and stages harness fixes for me to review weekly.

The whole point: instead of manually noticing I keep making the same mistake, the daemon does it and queues a fix.

The First Dry Run

First real pass: 8 candidates → 5 real faults. Not bad.

But the 3 misses weren't noise. They were both failures in the dedup layer — and they failed in completely different ways.

Problem A: Lexical Matching Doesn't See Meaning

existing_rule_for() matched on lexical similarity. A batch of mined advisor-outage faults came through… and the checker missed that they were semantic duplicates of a rule I already have: feedback_advisor_down_use_opus_agent.md.

Different words. Same lesson. Lexical dedup is blind to that.

I'd actually seen the shadow of this before. Back in April, Apollo's scanner (my inbox-to-task pipeline) was spawning duplicate Things tasks for the same Slack message — because it keyed on LLM-generated subjects and synthetic IDs that Sonnet regenerated fresh each scan. One message from a colleague produced 5 tasks.

The fix there: generate_content_dedup_id(source, sender, original_content) — hashing only immutable fields, no subject, no business tag, nothing the model can reinvent. Match on what it is, not what you called it this run.

The fault miner needed that same idea, generalized from byte-identity to meaning: RAG (retrieval-augmented search over existing rules) before staging anything.

Problem B: Mean-Reversion Is the Sneakier Bug

This one is harder. And I think more important.

One proposal from the dry run: "never render the inline 🔊 dashed-separator block."

That block is a DELIBERATE feature. It's the required read-when-muted visual companion, specified in feedback_speak_every_turn. Removing it would break something I chose on purpose.

A generic classifier doesn't know the difference between slop and a deliberate design decision. It sees "unusual pattern" → flags it. That's mean-reversion — the system drags idiosyncratic choices back toward whatever "normal" looks like, because it has no memory of your intentions.

Without a conflict-detection gate, that proposal stages. It sits in memory poisoning search results until /fault-review (my human-in-the-loop vetting skill) catches it.

The Fix (Phase B.5)

Dedup now runs as RAG + conflict-detection against existing rules BEFORE auto-promote. Two gates, in order:

  1. Does a rule already cover this? — catches semantic duplicates
  2. Does this contradict a deliberate rule? — catches mean-reversion before it lands

Anything that trips gate 2 routes OUT entirely. It never stages.

Why This Matters to Me

I'm building a system that improves itself. That's only useful if "better" means better for this system — not just more average.

If you're building anything that auto-learns from its own history, you need both checks. Content identity to avoid redundancy. Conflict-detection to protect the choices you made on purpose.

Without both, the system quietly corrects you into someone you didn't want to be.