LLM Security Red-Team Failure Modes: What Breaks and How to Contain It

The direct answer

Trace the failure case “testing without written authorization” through the workflow, then require a recovery check that can re-establish support for “scope and prohibited actions are signed off”.

For a scoped adversarial campaign against an LLM application, the relevant audience is teams that need attack evidence, not only a design checklist. The decision should cover authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning. The supplied boundary starts with authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and ends with a threat model, per-attack evidence record, and prioritized control list, presented in reviewable form.

A scoped campaign cannot certify the system, prove the absence of unknown vulnerabilities, or replace broader application and infrastructure security testing.

Map each failure to a signal and containment action

Only the security observer may record pass, hold, fail, repair, or stop against the registered acceptance statements.

Inspect the interfaces in the workflow

The operating path includes authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning.

Use “testing without written authorization” as an entry-point fixture and “attack cases copied from a checklist without system context” as a downstream fixture.

Treat retry as a separate consequential action

For a path affected by “successful prompts recorded without downstream impact evidence”, preserve an idempotency key, remote readback, or human decision before another attempt.

Preserve evidence before repair

• Freeze the triggering input and provenance for “unsafe data or tools left in scope”.
• Capture the last valid and first divergent state in authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning.
• Record the dependency, configuration, model, prompt, and policy versions that matter to a scoped adversarial campaign against an LLM application.
• Assign hypothesis testing for “unsafe data or tools left in scope” to the data owner without granting new authority.
• Require the security observer to accept, reject, or escalate the recovery result.

Repair should not erase the evidence needed to explain “unsafe data or tools left in scope”.

Verify recovery against acceptance statements

Recovery is incomplete until the team reruns the original failure and checks whether “scope and prohibited actions are signed off” holds. Add a regression case that also tests “each result has reproducible evidence” under the repaired condition.

If the failure case “controls recommended without a retest condition” remains possible, keep the affected path at hold.

An error message is not containment for “testing without written authorization”; recovery must also re-establish support for “scope and prohibited actions are signed off”.

Know when the failure model has expired

Revisit the failure model for a scoped adversarial campaign against an LLM application after any of three changes: the input boundary no longer matches authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts; the operating path no longer matches authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning; or the expected output no longer matches a threat model, per-attack evidence record, and prioritized control list.

Also reopen the model when permissions, dependencies, or operators introduce a path for a scoped adversarial campaign against an LLM application that the original fixtures never exercised.

How the sources bound the failure modes decision

For a scoped adversarial campaign against an LLM application, the live catalog limits the offer to two elements. The supplied boundary is authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts. The catalog names the deliverable as a threat model, per-attack evidence record, and prioritized control list. It cannot establish whether “scope and prohibited actions are signed off” holds in the buyer's environment.

Connect those narrow roles to a local fixture for “attack cases copied from a checklist without system context” rather than treating citation status as a pass.

For a scoped adversarial campaign against an LLM application, limit the conclusion to the documented workflow and let the red-team lead retain the current source-to-claim map. Keep the source decision provisional while the failure case “unsafe data or tools left in scope” remains unresolved.

Product-specific failure modes review drills

These drills connect a scoped adversarial campaign against an LLM application to concrete inputs, failures, acceptance statements, and owners. For a scoped adversarial campaign against an LLM application, the drills connect detection, containment, recovery, and regression.

The system owner models failures for a scoped adversarial campaign against an LLM application with synthetic, non-secret stand-ins for authorized application access, scope documentation, prohibited actions, and test data. Incident contacts are assigned separately from material custody. State-changing actions and every external effect remain inside the isolated fixture throughout and after each drill.

Trigger capture

Make “unsafe data or tools left in scope” the negative case for the trigger capture review. The system owner follows the case through authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning until the first unsupported transition.

Reproduce the condition within the boundary covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts, then have the red-team lead document whether the retained observation supports or contradicts the requirement that “scope and prohibited actions are signed off” holds.

The security observer records pass only for “scope and prohibited actions are signed off”. Any wider claim about a threat model, per-attack evidence record, and prioritized control list stays outside the drill. The trigger capture review records pass after support, fail after contradiction, and hold while evidence remains unresolved.

The system owner repeats the drill after a material change to the fixture, workflow, or evidence used to judge whether “scope and prohibited actions are signed off” holds.

First divergence

The first divergence review examines a case involving “controls recommended without a retest condition”. The red-team lead separates the trigger, current state, and next decision within authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning.

Connect a scope record covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts to one test of “each result has reproducible evidence”. Record both the observation and the review boundary.

The security observer compares the result with “each result has reproducible evidence” and records one bounded outcome. Unresolved scope cannot be converted into a pass. The first divergence review records pass after support, fail after contradiction, and hold while evidence remains unresolved.

Do not reuse the disposition when the failure case “controls recommended without a retest condition” occurs under conditions outside the recorded input and authority boundary.

Containment state

Ask how the containment state review handles the failure case “testing without written authorization”. The data owner freezes the local portion of authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning before drawing a conclusion.

The data owner checks a versioned boundary record covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts for “control fixes have retest cases”. A result from different conditions cannot close this drill.

The security observer closes the containment state review only after reconstructing why the criterion “control fixes have retest cases” passed or failed. A fluent explanation is not enough. The containment state review records pass after support, fail after contradiction, and hold while evidence remains unresolved.

Retest this decision when the team changes authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning or can no longer reproduce the record for “control fixes have retest cases”.

Retry decision

Use the occurrence of “attack cases copied from a checklist without system context” to begin the retry decision review. The data owner retains the workflow evidence available before containment.

For this drill, bind the fixture to the recorded boundary covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and the condition “attacks map to named threat hypotheses”. The remediation owner compares the artifact with a direct readback.

The security observer closes the retry decision review only when the record resolves “attacks map to named threat hypotheses”; otherwise the listed deliverable remains provisional. The retry decision review records pass after support, fail after contradiction, and hold while evidence remains unresolved.

Do not carry this verdict into a changed workflow, input class, or response to “attack cases copied from a checklist without system context”; create a new bounded record.

Recovery proof

Exercise the recovery proof review against the known risk “successful prompts recorded without downstream impact evidence”. Ask the remediation owner to mark the earliest point where the expected handoff diverges.

Let the system owner inspect a scope record covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and the evidence for “findings separate exploitability from impact”. For a scoped adversarial campaign against an LLM application, the recovery proof review cannot rely on a demonstration selected after execution.

For the recovery proof review, the security observer selects go, repair, or stop based on “findings separate exploitability from impact”. The selected outcome is retained with its evidence. The recovery proof review records pass after support, fail after contradiction, and hold while evidence remains unresolved.

Repeat the judgment when the workflow boundary for authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning adds a new handoff or removes the rollback state used in the test.

Regression fixture

Frame the regression fixture review around “unsafe data or tools left in scope”. Before testing a response, the system owner captures the input, decision boundary, and residual state.

Pair a scope record covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts with a direct observation of whether “scope and prohibited actions are signed off” holds. The red-team lead retains the source and result together.

The security observer records pass, repair, or stop after judging whether “scope and prohibited actions are signed off” holds. No disposition may imply that all of a threat model, per-attack evidence record, and prioritized control list was proven. The regression fixture review records pass after support, fail after contradiction, and hold while evidence remains unresolved.

An altered input source, acceptance owner, or response to “unsafe data or tools left in scope” invalidates only this drill and its dependent decisions.

Frequently asked question

What are the main failure modes for LLM Security Red-Team?

Begin with the failure cases “testing without written authorization” and “attack cases copied from a checklist without system context”. Give each condition a detection signal, containment owner, recovery check, and a regression test that checks whether scope and prohibited actions are signed off.

A product bridge, with a boundary

The LLM Security Red-Team is the relevant sincLLM offer for this narrow problem. The frozen live catalog describes its required boundary as authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and its deliverable as a threat model, per-attack evidence record, and prioritized control list. Treat the catalog language as a description of delivery; local evidence must still decide fit, safety, compliance, technical adequacy, and business value.