Almost everyone’s first encounter with this is a citation. You ask for background on something and get back a tidy paragraph with a reference — a paper, a plausible author, a journal, a year. You go to look it up and it isn’t there. Not moved, not paywalled: it never existed. And nothing about that answer looked any different from the ten correct things you were told an hour earlier.
That gap — between how an answer looks and whether it’s true — is what the four terms below are circling, and it’s why they belong on one page rather than four. Hallucination is the failure mode itself, and the guide underneath explains the mechanism properly. The one-line version is that it isn’t lying and it isn’t a bug someone forgot to fix: a language model composes the next plausible piece of text, and “plausible” overlaps with “true” most of the time but not all of it. The machinery has no way to tell those two cases apart, which is exactly why the fluency of an answer carries no information about its accuracy.
Alignment is a different thing that gets filed under the same worry, and separating them helps. When Claude declines a request or pushes back on your framing, that is not the system malfunctioning — it’s the deliberate work of making a very capable tool behave the way people actually intend. Being wrong and being unwilling are unrelated problems. Treating a refusal as a bug to route around is a category error, and usually a sign that the request needs rethinking rather than rephrasing.
Evals and benchmarks are how anyone knows any of this instead of guessing. A benchmark is a standardized test that runs several models over the same questions so “which is better?” becomes a number you can line up — useful for reading the “state-of-the-art” claims in a launch announcement with a clear head , and never the whole story, because your work is not the benchmark. Evals are the same idea pointed at your job: a set of real examples with known-good answers, scored repeatably, so a team can prove that a change helped rather than tuning prompts on vibes. If you are rolling this out to a group, that distinction is the difference between an opinion and evidence.
The practical upshot is narrower than the anxiety suggests. Work where the model transforms material you supplied — summarizing a document you attached, restructuring your own notes, drafting from your template — carries very little of this risk. The risk climbs the moment an answer depends on knowledge that wasn’t in the room: a specific figure, a date, a price, a citation, what some tool’s setting is called. So verify at the seam where it’s cheapest, the same instinct that makes seeing and undoing what Claude changed worth thirty seconds. And where you need answers grounded in real documents rather than recall, that’s what retrieval is for.
The words
- Hallucination
- When an AI states something false with complete confidence — citing a function that doesn't exist, inventing a file, or misremembering how a tool works. It isn't lying; it's pattern-matching to a plausible-sounding answer when it doesn't actually know.
- Alignment AI safety · Guardrails
- The work of making an AI's behavior match what people actually intend and value — being helpful and honest, and declining requests to do real harm. Guardrails are the practical limits that enforce this in day-to-day use: the boundaries that keep a capable system from being misused or going off the rails.
- Evals Evaluations · Evaluation
- Evals are repeatable tests that measure how well an AI does a specific job — like unit tests, but for model behaviour. You assemble a set of example inputs with known-good answers, run the model, and score how close it gets. They're how a team answers 'did that change actually make it better?' instead of guessing from a handful of hand-tries.
- Benchmark
- A standardized test used to compare models on a particular skill — coding, math, reading, reasoning — by running them all on the same set of questions and scoring the results. It turns 'which model is better?' into a number you can line up side by side, which is handy but never the whole story.