Hallucination
A hallucination is when a model states something false or fabricated with confidence — inventing facts, sources, or details that were never in its input or true in that form.
Language models generate the most likely continuation, not verified truth, so they can produce fluent, confident text that is simply wrong: a made-up citation, a non-existent API, a plausible but false fact. For agents this is dangerous because a hallucinated tool argument or claim can propagate through an entire run. The main defences are grounding answers in retrieved sources (RAG), asking the model to cite, verifying tool outputs, and designing tasks so the model can say "I don’t know".
Key points
- The model produces fluent, confident text that is factually wrong or fabricated.
- It happens because models predict likely text, not verified truth.
- In an agent, a hallucinated value can propagate through the whole run.
- Mitigate with grounding/RAG, citations, output verification, and room to say "I don’t know".
Learn it by building
Way of the Agent teaches AI agent engineering level by level — concepts like this one, then hands-on practice, XP, and the AI Chronicles.
Start learning free →