AI & models
Evals
Evaluations · Evaluation
Evals are repeatable tests that measure how well an AI does a specific job — like unit tests, but for model behaviour. You assemble a set of example inputs with known-good answers, run the model, and score how close it gets. They're how a team answers 'did that change actually make it better?' instead of guessing from a handful of hand-tries.
Why it matters
Part of When AI is wrong → Without evals you're tuning prompts and models on vibes; with them you can prove a change helped — or didn't — before you ship it.
see also