LLM Output Validation and Repair: A Fail-Closed Design

Validate in increasing order of consequence

Run cheap, deterministic checks first and stop as soon as the output cannot safely continue. The pipeline should return typed failure reasons so the product can distinguish a retryable transport problem from an invalid business instruction.

Separate structural validity from semantic correctness

JSON Schema provides a vocabulary for describing and validating JSON documents. Use it to make the interface explicit: required fields, primitive types, enumerated states, numerical or length bounds, nested objects, and whether unknown fields are accepted. Keep the schema version beside the prompt and consumer version so a release can explain which contract was active.

OpenAI's Structured Outputs documentation states that supported strict schemas constrain responses to the supplied shape and separately exposes refusals. That removes many parsing failures, but a schema cannot prove that an identifier belongs to the caller, a citation supports a claim, a date is operationally available, or an action is wise. Those are application checks.

Design the schema around a discriminated result such as success, abstention, refusal, or error. Do not force every path into a seemingly successful object. The companion structured outputs versus function calling guide shows when the contract describes data and when it proposes an external action.

Repair only what can be changed without interpretation

A repair policy needs a defect allowlist, attempt budget, validator, and terminal state. Safe mechanical examples include removing a known wrapper around an otherwise parseable value or normalizing an encoding artifact. Riskier changes—choosing an enum, inventing a missing identifier, filling a price, or reversing contradictory dates—alter meaning and should not be automated as formatting repair.

If the model gets another attempt, send the validation errors and the original task contract, not an invitation to produce “better JSON.” Validate the entire replacement from the beginning. Record attempt count, failure paths, model and prompt version, and whether a fallback was used. A retry that changes the action target must go through authorization again.

Repair can hide regressions when teams count only the final success. Track first-pass validity and repaired validity separately. A rising repair rate may signal prompt drift, a model change, an unsupported schema feature, or a consumer contract that is too ambiguous.

Output-validation failure cases

• Parsing with a permissive extractor: the system pulls the first object from surrounding prose and ignores an injected instruction. Require the declared response format.
• Schema equals truth: a perfectly shaped citation or account ID is assumed to exist. Resolve references against authoritative state.
• Repair until success: repeated model calls eventually produce a passing object while cost, latency, and semantic drift grow. Bound attempts and terminate explicitly.
• Validation after side effects: a tool receives arguments before the policy check. Keep execution behind the final gate in the guardrail architecture.
• Unknown fields silently survive: a consumer begins honoring an unreviewed property. Decide whether additional properties are forbidden or deliberately ignored.
• Diagnostic exposure: a failed response is echoed into a broadly visible channel. Redact by field and keep protected samples under explicit access and retention rules.

A practical implementation sequence

1. Map downstream consequences. Record each product destination and assign its path a risk tier.
2. Write a versioned result contract. Model success, abstention, refusal, and error explicitly, with narrow fields and stable identifiers.
3. Add deterministic validators. Parse strictly, apply the schema, then check domain invariants, references, authorization, and state preconditions.
4. Bound repair. Permit only transformations that preserve meaning, cap attempts, and retain the initial failure record.
5. Test malformed and incomplete results. Include truncation, extra prose, unknown fields, invalid enums, contradictory values, wrong-scope references, refusals, and timeouts.
6. Gate releases on both rates and examples. Segment first-pass failures and repaired cases by schema, model, task, and prompt version; add confirmed incidents to prompt regression tests.

Decision summary

Use constrained generation to reduce structural variability, JSON Schema to specify the data interface, application code to enforce meaning and authority, and human review when the remaining uncertainty is consequential. If a team cannot say which failures are repairable without guessing, the safe policy is rejection. See the structured prompting guide for making the requested output contract legible before validation begins.