Why AI Hallucinates
Recognize unsupported claims and invented APIs, then use evidence, evaluation and abstention to reduce their impact.
Before you start: Generative AI. Reading time is an estimate; allow extra time for practice.
What you’ll learn
- Distinguish observed facts, unsupported claims and contradictions.
- Check generated API claims against the installed version and behavior.
- Audit whether citations support the specific claims attached to them.
- Design evaluations that measure both useful answers and appropriate uncertainty.
On this page
Fluent output can still be unsupported
In this context, a hallucination is generated material that is false, invented or unsupported by the relevant evidence. A coding model might invent a method name that fits a library’s naming style. A summarizer might supply a date that was never in the source. The output can be grammatically excellent while failing the task.
A language model generates plausible continuations; that process alone does not check a claim against reality. Missing context, stale information, ambiguous instructions and a demand to answer every question can all contribute to errors. Avoid explaining every failure as a mysterious property of the model when a missing dependency version may be the immediate cause.
Investigate an invented API
Suppose an assistant suggests cache.refreshAll() for a package you use. Do not assume the method exists because the surrounding code is convincing. Inspect the installed package’s type declarations or official documentation for that version. If it does not exist, ask what behaviour is actually required and find a supported operation.
Next, compile or type-check the smallest example and run an independent behavioural test. A successful compile checks only part of the contract: a valid method can still invalidate the wrong cache, expose data or fail under concurrency. Verification should follow the consequences of the proposed change.
Claim: cache.refreshAll() reloads all entries.
1. Is refreshAll exported by the installed version?
2. What do the official docs say it does?
3. What happens with an empty cache and a failed reload?
4. Does a test observe the required result?
5. Are authorization and concurrency preserved?This is a fictional API claim for practice. Checking the name is the beginning of verification, not the end.
Ground answers and allow uncertainty
Provide authoritative, relevant evidence and ask for claims tied to that evidence. When using retrieval, check that citations resolve and support the associated statement. A source link to a real page can still accompany a fabricated interpretation. Source selection, freshness and coverage need their own evaluation.
Give the product a useful “not enough information” path. For example, a documentation helper can request the library version or show the closest source without inventing an answer. A self-reported confidence score is not a calibrated probability unless you have independently established that relationship for the task.
Measure errors that users would notice
Create a small evaluation set with answerable and unanswerable questions, deliberately misleading premises and conflicting sources. Record unsupported claims, wrong citations and failures to ask a necessary question. Keep examples separate from prompt tuning so improvements are not simply memorized fixes.
Reducing sampling variability, adding a second model review or asking the model to “be accurate” cannot guarantee truth. Use executable checks for executable claims and independent sources for factual claims. For high-impact outputs, assign a qualified reviewer and make the unresolved evidence visible.
Separate absence of evidence from a contradiction
Suppose an assistant summarizes a fictional manual that says “Version 3 supports CSV export; scheduled exports are not supported.” It replies, “Version 3 supports CSV and nightly scheduled exports.” The first claim is supported. The second contradicts the manual. If it also says “export completes in one second,” that claim is unsupported by this source; it might be true elsewhere, but the supplied evidence does not establish it.
This distinction guides the next action. A contradiction needs correction or an explanation of conflicting sources. An unsupported claim needs additional evidence or removal. A true statement can still fail a source-only task if its required evidence is missing. Do not silently convert “not established” into “proven false.”
For each factual clause, record the source, relevant passage, version and verdict. Open a citation and read the supporting text rather than checking only that the URL loads. For a software claim, compare documentation with the installed version and an execution result when possible. If these disagree, expose the discrepancy and investigate instead of selecting whichever evidence makes the draft sound right.
| Generated claim | Verdict | Next action |
|---|---|---|
| Version 3 supports CSV | Supported | Keep with the relevant source |
| Nightly scheduled exports are supported | Contradicted | Correct the claim; the source explicitly says otherwise |
| Export completes in one second | Not established | Remove or obtain a relevant measurement |
Try it yourself
Take a generated explanation of a function and mark each statement as directly observed, supported by documentation or still unverified.
Show hint
Separate what the code does from why its author wrote it. The latter may be speculation even when the behaviour is easy to test.
Check your understanding
5 questions to revisit the ideas. No score is saved, and you can retry. This is separate from Skill Quiz; it does not award XP or unlock guides.
Sources & further reading
Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.