LLM Evaluation: Cases, Graders, Failures, Evidence

Start with the operating problem

A single average can hide missing coverage, severe boundary failures, inconsistent grading, or examples that no longer resemble production work. Evaluation needs case-level evidence and failure categories, not only a leaderboard.

The suite should distinguish fixed rules from qualities requiring judgment. Structural validity and preserved state belong in mechanical assertions. Clarity may need a rubric calibrated against human-reviewed examples.

Stack Overflow documents developer use alongside concern about accuracy, while Deloitte describes scaling and governance barriers. These findings motivate repeatable evaluation without defining one passing score. The approved evidence is Stack Overflow and Deloitte; it is directional context rather than proof of a Sinc LLM capability or a guaranteed outcome.

A decision framework for LLM evaluation

Evaluate component contracts, trajectory, end state, and adversarial behavior separately. A correct answer reached through a forbidden action is failure; a harmless alternate path may still pass.

1. Representative cases. Cover normal, alternative, ambiguous, and failed flows drawn from real task shapes. Replace sensitive details while preserving decision structure.
2. Hard assertions first. Use code for schemas, permissions, state changes, calculations, stop reasons, and required evidence rather than a model opinion.
3. Calibrated judgment. Define rubric dimensions and examples, compare grades with human review, and retain disagreement instead of forcing consensus.
4. Versioned evidence. Keep case, model, instructions, settings, result, grade, and failure class so a regression can be reproduced.

The normal path

Build evaluation from the task contract outward. Expected behavior should include both the requested artifact and the permitted path. Inspect severe failures before summarizing distributions.

1. Define the contract. Write the end state, allowed effects, unchanged resources, evidence, review points, and stop conditions.
2. Collect fixtures. Choose representative requests and label why each matters, including exclusions and known coverage gaps.
3. Write assertions. Express hard invariants in code and reserve rubrics for relevance, completeness, and calibrated uncertainty.
4. Run by version. Record every changed component and repeat variable cases enough to expose unstable failure categories.
5. Review before averaging. Inspect hard violations, grader disagreements, and segment failures before promoting a change.

The failure path and its guards

A suite can pass while testing the wrong thing. Challenge it with valid alternate trajectories, corrupted fixtures, biased graders, and rare boundary violations. The evaluation system needs regression evidence too.

• Narrow phrasing. Add paraphrases and alternate inputs while keeping the same invariant. Do not reward memorization of one request shape.
• Shared blind spot. Compare the grader with independent human examples and mechanical evidence. Escalate unresolved disagreement.
• Hidden forbidden behavior. Make permission and state violations hard release gates reported separately from quality averages.
• Missing regression run. Block promotion when the prior suite, changed versions, and observed differences are unavailable.

A practical next action

Create five cases from real task shapes with sensitive details replaced by synthetic fixtures. For each, write required end state, forbidden behavior, mechanical assertions, judgment rubric, and retained evidence. Then add a case inventory that records the task segment, input conditions, expected trajectory, reviewer rationale, and failure severity for every fixture. Run each case across the baseline and proposed version, preserving raw outputs and hard assertion results. When a case changes class, determine whether the requirement changed, the fixture drifted, the grader disagreed, or the system regressed. Do not repair a weak average by deleting difficult cases. Promote incidents into named regression fixtures, and leave uncovered requirements visible until an owner accepts the gap or supplies a test.

Run a baseline and inspect every failure before tuning. When changing instructions, retrieval, model, or tools, rerun the same bundle and promote only when hard gates pass and the owner accepts documented tradeoffs.

Create a coverage map that links each requirement to a normal case, an alternative case, and a failure probe. Mark requirements with no fixture as gaps rather than assigning them an implicit pass. Add incident-derived cases without deleting the earlier baseline.

Review grader disagreements by category. If humans disagree because the rubric is vague, revise the dimension and rescore the preserved outputs. If the disagreement reflects legitimate judgment, report the distribution instead of forcing a single label.

Limitations

A small suite cannot cover every future request, dependency failure, update, or adversarial strategy. Production observations must supply new cases.

Rubric graders remain fallible even when calibrated. Keep consequential decisions with the responsible owner and preserve disagreements.