Hallucination Is a Broken Off-Switch, Not a Knowledge Gap

Most advice about hallucination assumes the model didn't know the answer. Anthropic's interpretability team looked inside Claude and found the opposite framing: refusal is the default. There is a circuit that is on out of the box and makes the model say it has insufficient information to answer anything. When you ask about something it knows well — their example is Michael Jordan — a competing "known entities" feature fires and inhibits that refusal circuit, which is what lets an answer through.

Hallucination is that inhibition happening when it shouldn't. The researchers proved it by hand: activating the "known answer" features (or suppressing the "can't answer" ones) made the model claim, quite consistently, that a made-up person named Michael Batkin plays chess. In the wild, the same misfire happens when Claude recognises a name but knows nothing else about it.

That reframes the fix completely. You are not asking the model to try harder or think more. You are doing two mechanical things: giving the refusal path a licence to win, and putting a real document in front of it so "known" means "in this text" instead of "vaguely familiar".

1. Ground it — quotes first, analysis second

The order matters. Extracting verbatim quotes before the reasoning forces the answer to be built on the text; asking for sources afterwards invites the model to decorate a conclusion it already wrote. Anthropic's own guidance recommends the quote-first pattern specifically for documents over 20k tokens.

Answer using ONLY the document below. Do not use general knowledge.

Step 1 — Extract the exact quotes relevant to my question, word for word,
and number them. If there are none, write "No relevant quotes found."
Step 2 — Answer my question using only those quotes, citing them by number.
Step 3 — Delete any sentence you cannot tie to a numbered quote and leave
an empty [] where it was.

<document>{{PASTE}}</document>
Question: {{QUESTION}}

Catches: invented statistics, claims smuggled in from training data, sources that don't say what the answer claims.

Misses: anything wrong inside your document, and misread quotes. And it only works when you supply the text — if the model is browsing, you still have to open the links.

If you're building this into a product, stop prompt-engineering it: the Claude API's Citations feature returns the exact sentences a response used. Anthropic reports it lifts recall accuracy by up to 15%, and the customer Endex took source hallucinations from 10% to 0% while getting 20% more references per response.

2. Licence the refusal — make "I don't know" a valid answer

This is the cheapest one and the closest to the mechanism. It's a rule you set for the session, not a follow-up question.

Rule for this whole conversation: "I don't know" is a complete, acceptable
answer. I would rather have a gap than a confident wrong answer.

Structure every factual reply as:
- Confident: claims you'd defend, with what they rest on.
- Uncertain: claims you believe but can't ground — label them.
- Unknown: what you'd need (a document, a search, a version number) to answer.

Never fill a gap with fluent prose.

Catches: the half-recognition guess — the Batkin case, where the model has a name and nothing else.

Misses: everything the model is confidently wrong about. If the "known entities" feature is fully lit, the model isn't uncertain, so it has nothing to report. This prompt surfaces doubt; it does not create it.

3. Attack it from a clean context

Open a new chat — ideally a different model — paste the answer as a claim to be disproven, and ask for a verdict per claim. Same-session review is compromised: the assistant already committed to that text and reads it as its own.

Below is an answer from another model. Assume it contains at least one error.
Your job is to find errors, not to agree.

For EACH factual claim, return: the claim | verdict (supported /
unsupported / false) | the evidence or the exact search you'd run.
Then list every claim that cannot be verified at all.

<answer>{{PASTE}}</answer>

Catches: fabricated citations, internal contradictions, numbers that don't survive a second look.

Misses: shared blind spots — one model family tends to be wrong in the same places, which is why a different provider is worth the extra tab. And don't treat the reviewer's explanation as proof. In Anthropic's faithfulness study, Claude 3.7 Sonnet mentioned the hint it had actually used only 25% of the time (DeepSeek R1: 39%) — and the unfaithful chains of thought were longer than the faithful ones. A confident five-paragraph justification is not evidence of anything.

Why this matters more if you're interviewing abroad

Applying to remote roles in the US or Europe, your written artefacts are the interview: the take-home, the design doc, the PR description, the follow-up email. One fabricated benchmark number in an RFC costs more credibility than a slow reply ever will — and you rarely get the chance to explain that a model wrote it. Verification isn't caution here, it's the cheapest reputation insurance available.

Today: take the last AI answer you actually used for something and run prompt 3 on it in a fresh chat with a different provider. Count the unsupported claims. That number is your personal calibration — and it's usually higher than you'd like.

Sources

• Anthropic Research — Tracing the thoughts of a large language model
• Claude Docs — Reduce hallucinations
• Anthropic — Introducing Citations on the Claude API
• Anthropic Research — Reasoning models don't always say what they think