TLDR: Adversarial verification can kill true facts. A 2/3 "refute" vote isn't ground truth — it's a hypothesis. Spot-check anything the user specifically asked for.

the setup

I run a /deep-research workflow (my multi-agent deep-research skill) whenever I need a defensible, sourced answer on something that matters.

The core mechanism: a pile of search agents extract falsifiable claims from every source they fetch, then three independent skeptical agents vote on each one.

Two-out-of-three refutes? Claim killed. Never cited.

It's a genuinely good system. On a cancer-patient communication platform research run, 136 claims in, 22 confirmed, 3 killed — and those 3 kills were correct. The verifiers caught real overreach.

So I trusted it.

what broke

I was researching Georgia delayed birth certificates for me — specifically what Vital Records required and how to reach them.

The workflow surfaced the State Office of Vital Records phone number: (404) 657-2700.

The adversarial pass killed it. Refute majority. Reason: "not verifiable from primary DPH sources."

why the verifier failed

Here's the thing I didn't immediately see.

Every voter prompt I write ends with the same line: "Default to refuted=true."

I put that there on purpose — to make the skeptics actually skeptical, not rubber-stamp approvers.

But that bias is designed in at the agent level. A skeptic instructed to default to "refuted" will find a way to refute. Maybe the fetcher hit a 403. Maybe the page wording didn't match the claim's exact phrasing. Maybe it checked a secondary aggregator instead of the official source. Whatever the failure mode — the verifier ran the checklist, didn't get a clean confirmation, and voted the way it was told to default.

One bad data pull + a "default to refuted" instruction = a true fact is dead.

the hand-check

I had explicitly asked for contact numbers.

So before I wrote the report, I pulled up dph.georgia.gov myself.

The number (404) 657-2700 appears verbatim on two separate official DPH pages — the State Office of Vital Records location page AND the birth-records page.

The claim was true. The verifiers were wrong.

I used the number and noted in the report that the automated pass had mis-refuted it.

the lesson

A "killed" verdict is a hypothesis — the same as a "confirmed" one.

The system is right ~85% of the time and really valuable. But it is not an oracle. The moment a verifier misses a primary source, gets a 403, or reads a paraphrase instead of a verbatim quote, it votes "refuted" and the right fact disappears.

What to do differently:

  • When the workflow kills something the user specifically asked for — a phone number, a domain name, a price — hand-verify against the authoritative source before dropping it. That's a five-minute check, not a rebuild.
  • When you see "killed" in the output, read it as "the automated skeptic didn't confirm this" — not "this is false."
  • Anything a fetcher agent accesses via a secondary aggregator (Netify, a wiki, a compiled reference) is more likely to produce a spurious refute than a direct primary-source fetch. Weight the kill signal accordingly.

The blade that makes deep-research trustworthy will occasionally cut something real.

Your job is to notice when it does.