Test Normal, Failure, and Adversarial LLM Cases

Start with the operating problem

A polished happy-path demonstration can hide the conditions that dominate real operations: missing context, incompatible formats, stale sources, timeouts, conflicting instructions, and requests outside the approved boundary. Model output also varies, so exact text comparison rarely captures every requirement. Teams need assertions about claims, structure, actions, and end state without accepting any fluent variation as success.

Workflow testing should separate deterministic contracts from qualities that require judgment. A schema, identifier, denial, and destination readback can be checked mechanically. Relevance or completeness may need a rubric and independent review. Combining those layers into one vague score makes regressions difficult to diagnose because a pass never identifies which obligation held and a failure never identifies the repair.

Stack Overflow's survey evidence highlights the gap between AI use and trust, while Deloitte reports enterprise concerns around governance, risk, and scaling. They motivate stronger evaluation practice but do not prescribe one universal test suite or guarantee removal of model error. The approved evidence is Stack Overflow and Deloitte; it is directional context rather than proof of a Sinc LLM capability or a guaranteed outcome.

A decision framework for workflow testing

A useful case names its input, allowed context, expected actions, forbidden actions, end-state assertions, and evidence. Coverage should follow failure cost and reachable states, not the number of prompt phrasings a team can generate.

1. Test the whole path. Include retrieval, generation, validation, action, approval, update, and readback. A correct answer is not a passing workflow if the wrong source was used or the intended destination never reached the required state.
2. Separate invariant from rubric. Use exact checks for schema, boundaries, links, totals, and state transitions. Use a documented scale for semantic qualities, with examples of acceptable and unacceptable evidence, so reviewer judgment remains inspectable.
3. Choose risk-shaped fixtures. Prioritize cases whose failure would mislead a customer, expose private material, exceed authority, or leave partial state. Add realistic ambiguity and dependency faults rather than relying on random wording changes.
4. Demand reproducibility. Capture approved inputs, source references, prompt and model versions, action responses, validator output, and disposition. Substitute synthetic details before a failing case enters the reusable suite.

The normal path

Start with a compact suite that represents distinct operating states. Each run should reveal which layer failed and whether the workflow stopped before consequential action.

1. Write acceptance assertions. Translate each requirement into an observable condition: required fields, supported claims, allowed actions, approval points, destination state, and resources that must remain unchanged.
2. Seed representative cases. Create an ordinary input, a valid variation, an ambiguous request, an unavailable dependency, and a denied action using synthetic or sanitized material.
3. Run deterministic checks first. Reject malformed output, unsupported identifiers, forbidden actions, and incorrect end states before spending reviewer attention on tone or semantic quality.
4. Apply the judgment rubric. Have a qualified reviewer score only qualities that rules cannot decide. Preserve rationale and source references beside the score rather than storing a bare label.
5. Promote diagnosed failures. When a failure is understood, add its smallest reproducing fixture and expected safe behavior to regression coverage. Track changes to assertions as carefully as prompt changes.

The failure path and its guards

Adversarial testing is most useful when it targets authority and evidence, not theatrical attempts to make the model say something strange. A safe failure may be a refusal, partial result, or precise request for missing input.

• Untrusted instruction wins. Place conflicting commands inside retrieved text and verify that they remain data. The workflow should follow its binding policy, flag the conflict, and avoid expanding its action boundary.
• Dependency fails after progress. Interrupt a synthetic dependency after an intermediate step. Confirm that the run reports partial state, avoids duplicate updates, and follows a defined recovery route.
• Citation looks relevant but is not. Supply a source on the same topic that does not support the material claim. The evaluator should reject entailment rather than reward the presence of a plausible link.
• Flaky judgment hides regression. If reviewers disagree, inspect the rubric and evidence examples before averaging scores. Narrow the criterion or route the case for owner judgment instead of treating instability as a pass.

A practical next action

Take one planned workflow and list its externally observable obligations. Turn each obligation into a fixture with expected state and a named validator. Include a case where correct behavior is to stop without producing the requested result, plus a case that preserves a valid partial result.

Run the suite against a fixed candidate and save the trace. For every failure, identify whether the missing artifact is a guard, fixture, assertion, rubric example, boundary rule, or recovery path. Change one layer at a time, rerun the smallest case, and then run the affected regression group.

Limitations

A finite suite cannot represent every future input or model variation. Reopen coverage when sources, integrations, policies, models, or user behavior change.

Passing tests provide mechanical evidence for the cases exercised; they are not final certification and do not eliminate the need for monitoring or authorized review.