TLDR: Don't put secrets in the prompt. Stripping sensitive fields from your system prompt JSON is the real lock — "do not share this" instructions are defense #2, not #1.

The Leak

I've been building a chat-first app for my cancer education business that helps cancer patients find integrative oncology clinics — and when the first real user transcripts came in, I found this:

user: I'd like to book a consultation. assistant: Hi! That's wonderful you're reaching out. An integrative oncology clinic is known for their caring approach… You can easily book a consultation by visiting their website at [clinic website] or calling [clinic phone number].

User got the phone number. User left. I captured nothing — no name, no email, no follow-up channel.

Lead lost.

Why It Happened (The Real Reason)

My first instinct was to slap a system prompt instruction on it: "do not share contact information until a lead is captured."

That helps. But it's not the lock.

The actual problem was simpler: I was JSON-stringifying the full facility record into the prompt for deep-link mode (a URL param that routes a user directly to a single clinic). Phone, website, hours, doctor name — all of it, right there in the context. The model shared what it saw because of course it did.

Multi-facility mode was already safe because it had only ever sent id, name, treatments, location, verificationStatus to the model. I'd accidentally data-minimized the right way there without thinking about it.

The fix that mattered: conditionally build a slimmer JSON when !hasLeadInfo in src/app/api/chat/route.ts. If no lead exists, the prompt never receives a phone number. You can't leak what the model doesn't know.

Then I Over-Corrected

Here's where I embarrassed myself a little.

I built a strict gate. The AI would lead with clinic vibes, then: "Once you share your details below, I can help you book a consultation."

Felt airtight. Felt smart. Felt like a doorman standing at a velvet rope.

I ran scripts/analytics-funnel.ts against Supabase a week later. The numbers were brutal:

  • 291 conversations. 9 leads. 3 consultations — all from the same person.
  • 207 of 291 sessions (71%) started with "I'd like to book a consultation" and produced ZERO leads.

The AI wasn't helping anyone find cancer care. It was bouncing elderly patients because it wouldn't answer their question.

The Relaxed Gate (What Actually Works)

On c8cccf4 (2026-04-24), I relaxed the prompt:

  • AI MAY share: clinic names, treatments, doctor names, philosophy of care, city/state
  • AI STILL BLOCKS: phone numbers, websites, emails, full addresses, booking links
  • AI MUST lead with value first — 2-3 useful facts in the first response, then a soft offer
  • AI must NOT reference the form — let it stand on its own above the thread

The lead form still renders. Returning visitors with a cached lead skip the gate entirely — the API seeds their info and the form pre-fills for one-click confirm.

Why This Matters

If you're building an AI with a conversion requirement, here's what I'd do differently:

  1. Data minimization first. Strip sensitive fields from your prompt payload when access isn't earned yet. Prompt instructions are defense #2.
  2. Watch what your gate says. "Share your details below" is a bounce trigger. A soft, value-first offer is not.
  3. Pull your session data early. Don't wait for "enough" users — 50 conversations will show you the choke point faster than you think.

The gate was the right instinct. The doorman framing was the mistake.

P.S. The pre-overhaul baseline: 5.8% engagement past 1 message, 3.1% lead capture, 1.0% consultation rate. I'll post the deltas once the data settles — too many changes shipped at once to cleanly attribute lift right now.