Evaluating AI Models
Build task-specific evaluations, interpret precision and recall, and examine failure cases before and after release.
Before you start: Training and inference · Basic JavaScript for the example. Reading time is an estimate; allow extra time for practice.
What you’ll learn
- Define acceptance criteria before comparing models.
- Calculate precision and recall and interpret their tradeoff.
- Build independent evaluation sets with meaningful slices.
- Evaluate generative applications beyond a single accuracy score.
On this page
Measure the task, not the demonstration
Imagine an issue classifier that flags reports needing urgent review. A false positive sends an ordinary report to the urgent queue; a false negative misses an urgent report. The consequences differ. Begin by naming the desired behavior and acceptable failure modes, then choose measurements that expose them.
Keep a baseline such as the current rule or previous model. Record the dataset version, prompt, model identifier, settings and preprocessing. A compelling example can reveal a capability, but it does not establish reliability across the application’s inputs.
Read a confusion matrix
For a binary classifier, a true positive is an urgent report correctly flagged; a false positive is a normal report incorrectly flagged. A false negative is an urgent report missed, and a true negative is a normal report correctly left unflagged. Precision is TP / (TP + FP); recall is TP / (TP + FN).
Precision asks how many flags were correct. Recall asks how many actual urgent reports were caught. Overall accuracy can hide poor performance on a rare urgent class. A system that never flags anything can look accurate when almost every report is ordinary.
const cases = [
{ urgent: true, score: 0.9 }, { urgent: true, score: 0.6 },
{ urgent: false, score: 0.7 }, { urgent: false, score: 0.2 }
];
function metrics(threshold) {
let tp = 0, fp = 0, fn = 0;
for (const item of cases) {
const predicted = item.score >= threshold;
if (predicted && item.urgent) tp++;
if (predicted && !item.urgent) fp++;
if (!predicted && item.urgent) fn++;
}
return { precision: tp + fp ? tp / (tp + fp) : null,
recall: tp + fn ? tp / (tp + fn) : null };
}
const low = metrics(0.5), high = metrics(0.8);
if (low.precision !== 2/3 || low.recall !== 1) throw new Error('Low threshold');
if (high.precision !== 1 || high.recall !== 0.5) throw new Error('High threshold');
if (metrics(1).precision !== null) throw new Error('Undefined precision');
console.log(low, high);At 0.5 the toy classifier catches both urgent reports but flags one ordinary report. At 0.8 it produces no false positive in these four cases, but misses one urgent report. These scores are fictional, not calibrated probabilities.
Select thresholds on development data
Changing a threshold changes which cases are positive. Lowering it cannot reduce the number of predicted positives for a fixed score set, but its effect on precision depends on the labels of the additional cases. Choose a threshold using validation data and the costs of errors, then measure the chosen rule on separate test data.
A model output of 0.9 is not automatically a 90% chance of correctness. Calibration asks whether stated probabilities match observed frequencies across comparable cases. A confidence-looking number produced in a language-model response is not a substitute for a calibrated estimator or task evidence.
Build an evaluation set that can disappoint you
Collect ordinary, difficult, incomplete and out-of-scope examples. Keep similar users, documents or time periods grouped when leakage would otherwise exaggerate results. Document where labels came from and resolve disagreements; a model cannot be evaluated reliably against an undefined answer policy.
Report sample sizes and uncertainty, especially for rare cases. Break results down by relevant slices such as language, input length or device type. An average can improve while a smaller group gets worse. Do not invent demographic labels or collect sensitive attributes without a justified process merely to fill a fairness dashboard.
Use several checks for generated answers
Open-ended answers do not always have one exact matching string. Define a rubric: factual support, completeness, useful abstention, instruction-following and harmful or unauthorized behavior. Use deterministic checks for schema and code tests where possible, and human review for qualities those checks cannot establish.
A model-based judge can speed review but can also be sensitive to wording, answer order and its own preferences. Compare it against human judgments and include disagreements in the review process. Do not use a model’s approval of its own answer as independent verification.
Evaluate components and the end-to-end application. For a retrieval assistant, distinguish missing sources from unsupported generation after the right source was found. Include request failures, timeouts and incorrect permissions alongside answer quality. A correct-looking answer that exposes another user’s document is a system failure.
Turn results into a release decision
State the acceptance criteria before looking at the final results. Compare candidate and baseline on the same cases and inspect both improvements and regressions. Repeat stochastic generation when variability matters, recording the sampling setup rather than selecting only the best run.
After release, monitor latency, failure rates and reviewed samples of real behavior with suitable privacy controls. Changing traffic, prompts or providers can invalidate an old result. Keep rollback practical and label reports by the exact configuration evaluated; “tested” is incomplete without saying what was tested.
Try it yourself
Add one urgent report with score 0.1 to the executable fixture. Predict precision and recall at both thresholds, update the expected values in the assertions, then run it. Explain what this reveals that the original four cases missed.
Show hint
The predicted positives stay the same, so precision stays unchanged. The denominator for recall increases by one at each threshold. Keep the original fixture as a separate regression case. Then propose a real-world slice that could contain similarly missed cases; the toy numbers alone do not justify a production threshold.
Check your understanding
4 questions to revisit the ideas. No score is saved, and you can retry. This is separate from Skill Quiz; it does not award XP or unlock guides.
Sources & further reading
Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.