TLDR: Pure semantic RAG sounds like the upgrade. It's not sufficient on its own. The gap isn't recall quality — it's freshness and coverage. Run grep alongside it, concurrently. Hybrid beats either arm alone, and the second arm is nearly free.

The Problem with Grep (and Why I Thought I Needed to Replace It)

Apollo (my AI assistant, running Claude Code with a custom memory layer) was searching my notes with plain adaptive grep.

And grep has a silent false-negative problem.

It finds what you already know to search for. "deploy" misses a memory file that says "ship to prod." You'd never know the miss happened — you just get a confident non-answer.

That's the worst kind of wrong.

So I Built Semantic RAG

Shipped a semantic RAG system on 2026-05-08. The architecture: Claude Sonnet chunks each file at semantic boundaries (not fixed-size windows), OpenAI text-embedding-3-large embeds the chunks, they land in sqlite-vec, BM25 builds a sparse keyword index alongside it, and at query time both arms run concurrently → Reciprocal Rank Fusion merges → Claude synthesizes.

It worked. The false-negative rate dropped. Meaning-based recall is genuinely better.

But I wasn't done.

What Pure RAG Missed

Two gaps showed up pretty quickly.

Gap one: freshness. Embeddings are only as fresh as the last reindex. Edit a memory file at 2pm, run a search at 2:03pm — that file is invisible to RAG until the hourly cron fires. Grep has always seen the live filesystem.

Gap two: instruction files. Some files aren't in the indexed corpus at all. Behavioral rules, skill files, things that live outside the vault. RAG structurally can't reach them. Grep can.

So I had two retrieval arms that covered each other's blind spots. The solution wasn't to pick one…

The Hybrid Memory Search — The Approach That Won

Built a hybrid memory search skill on 2026-06-11. It runs both arms — RAG semantic + adaptive multi-grep — in a single pass, merges the results, flags stale hits, and returns a synthesized answer with raw hits for audit.

The engineering detail I'm most proud of: grep runs in a background thread during RAG's embedding round-trip.

RAG takes ~3.94 seconds — almost entirely the OpenAI embeddings API call. Grep takes ~0.4 seconds total. Because they run concurrently, the second arm adds almost nothing to wall-clock time. And the grep variants cost ZERO tokens — they never touch the embedder.

Hybrid retrieval isn't a latency tradeoff. It's nearly free.

The Reranker I Almost Built (and Chose Not To)

Here's the honest part…

Before shipping, I evaluated a reranker on top of the RAG retrieval. Ran it against 18 real labeled queries. Baseline scored top-1 0.722 / top-3 0.944 / MRR 0.826.

A reranker only reorders the top-k. But synthesis reads the WHOLE returned set — so if the right file is in top-3 94% of the time, reordering rank-0 changes nothing about the synthesis input. It just adds latency and cost on the 3.94-second hot path.

So I rejected it. Saved the eval harness in the RAG system's eval directory to re-gate if recall ever degrades.

Measure before adding complexity. That's the lesson I'll take everywhere.

Why This Matters to Me

I'm building AI systems that have to be reliable about context — not just fast, not just "usually right." Hybrid retrieval is the pattern that got me there: semantic search for coverage, grep for freshness and structural reach, concurrency so neither arm slows the other down, and stale-flagging so old context can't masquerade as current truth.

Every memory system I build from here starts with this pattern.