TLDR: When Pydantic is validating LLM output, extra="forbid" is the wrong default. Switch to extra="ignore" — and wrap each item in its own try/except. One bad LLM response shouldn't kill your whole batch.

The Setup

Apollo, my personal AI assistant, runs a scanner pipeline that reads iMessage and email, passes each item through Claude Sonnet (collector) and Claude Opus (judge), and creates tasks in Things 3, my task manager, when something needs action.

The pipeline uses Pydantic to validate the JSON the model returns.

I'd set extra="forbid" on my models. Felt like the right call — strict is better than loose, right?

The Wall

The scanner started crashing mid-batch.

Pydantic ValidationError: unexpected field. The LLM was returning a shape that didn't exactly match my model — a stray extra field tacked onto the response, something it was never asked for.

One item failing… killed the entire batch. Everything processed before the crash? Gone.

What I Chased First

My instinct was to tighten the contract.

I built a unified JSON parser that could handle prose preambles, arrays, nested junk — basically trying to make the output parser bulletproof. That work was real and necessary (bug #6 was sitting right next door). But it wasn't fixing this.

I was solving the wrong problem.

I was trying to force the LLM to give me a cleaner shape, when I should have been making my validator more forgiving.

The Fix That Worked

Two changes, shipped together:

  1. extra="forbid" → extra="ignore" on the Pydantic models
  2. Per-item try/except — wrap each item's validation so one failure degrades gracefully instead of crashing the whole batch
# Before: one unexpected field crashes everything
class ScanResult(BaseModel):
    model_config = ConfigDict(extra="forbid")

# After: accept the known fields, silently discard the rest
class ScanResult(BaseModel):
    model_config = ConfigDict(extra="ignore")

That's it. Scanner's been stable since.

Why This Matters

Here's what I had backwards: extra="forbid" is exactly right when you control the data producer. It catches schema drift, typos, bad refactors. Hard contract enforcement is the point.

But an LLM is a probabilistic producer. It will occasionally add fields you never asked for. It just will. Demanding a perfect contract from a stochastic process is a losing bet.

The tradeoff is real though — extra="ignore" is drift-blind. If the model starts quietly omitting required fields or renaming things, you won't get a loud crash. You'll get silent degradation.

So the rule I carry now: match validation strictness to the producer.

  • Data you control → extra="forbid". Enforce that contract hard.
  • LLM output → extra="ignore" + per-item try/except. Capture the signal, survive the noise.

The per-item isolation is the other half of this fix. Even with extra="ignore", a model can return something totally unparseable. If your whole batch fails atomically, you lose everything. Wrap each item. Log the failure. Move on to the next one.

These two together are what made my scanner actually resilient instead of just probably fine.

P.S. Also pinned Pydantic to >=2.0,<3.0 while I was in there. If you're building on v2, do yourself a favor and pin it — the v2→v3 migration is a day you don't want to discover by surprise in production.