A Go-or-No-Go Pilot Plan for a Scoped Adversarial Campaign Against an LLM Application

The direct answer

Use a bounded slice to test whether “scope and prohibited actions are signed off” holds, make “testing without written authorization” a stop case, and leave expansion to the security observer.

For a scoped adversarial campaign against an LLM application, the relevant audience is teams that need attack evidence, not only a design checklist. The decision should cover authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning. The supplied boundary starts with authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and ends with a threat model, per-attack evidence record, and prioritized control list, presented in reviewable form.

A scoped campaign cannot certify the system, prove the absence of unknown vulnerabilities, or replace broader application and infrastructure security testing.

Write a pilot charter that can return no

Choose the riskiest assumptions

Start with the assumptions behind “scope and prohibited actions are signed off” and “attacks map to named threat hypotheses”.

Include “testing without written authorization” and “attack cases copied from a checklist without system context” as bounded negative fixtures.

Freeze a comparison baseline

The comparison asks whether “each result has reproducible evidence” holds without weakening the authority or evidence rules.

Run the canary as a sequence of gates

1. Confirm that the system owner still authorizes the charter.
2. Verify the supplied boundary matches authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts.
3. Exercise the normal path and inspect whether “scope and prohibited actions are signed off” holds.
4. Run the failure case “successful prompts recorded without downstream impact evidence” without widening authority.
5. Compare the candidate and baseline evidence for “findings separate exploitability from impact”.
6. Ask the security observer to record go, revise, or stop.

Use explicit decision outcomes

Prove rollback before expansion

If the failure case “testing without written authorization” occurs, stop writes, capture the live state, and compare it with the manifest before rollback.

Close the pilot with a bounded claim

A pilot is only a demonstration when it cannot stop for “testing without written authorization” or withhold expansion after the criterion “scope and prohibited actions are signed off” fails.

A passing result supports only the tested slice of a scoped adversarial campaign against an LLM application.

How the sources bound the pilot plan decision

For a scoped adversarial campaign against an LLM application, the live catalog limits the offer to two elements. The supplied boundary is authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts. The catalog names the deliverable as a threat model, per-attack evidence record, and prioritized control list. It cannot establish whether “scope and prohibited actions are signed off” holds in the buyer's environment.

Connect those narrow roles to a local fixture for “attack cases copied from a checklist without system context” rather than treating citation status as a pass.

For a scoped adversarial campaign against an LLM application, limit the conclusion to the documented workflow and let the red-team lead retain the current source-to-claim map. Keep the source decision provisional while the failure case “unsafe data or tools left in scope” remains unresolved.

Product-specific pilot plan review drills

These drills connect a scoped adversarial campaign against an LLM application to concrete inputs, failures, acceptance statements, and owners. For a scoped adversarial campaign against an LLM application, the drills bound the canary, stop rule, and expansion decision.

The pilot boundary for a scoped adversarial campaign against an LLM application records authorized application access, scope documentation, prohibited actions, and test data but exercises only synthetic, non-secret markers. Incident contacts are assigned separately from material custody. The system owner confirms that no enqueue, send, write, or external call may exit the canary fixture throughout or after the pilot.

Charter boundary

Use the charter boundary review to examine what follows from the failure case “testing without written authorization”. Before intervention, the system owner retains the observable handoff.

For the charter boundary review, the red-team lead reviews a scope record covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts against the requirement that “scope and prohibited actions are signed off” holds. Unrelated artifacts are excluded.

The security observer limits acceptance to “scope and prohibited actions are signed off” and nothing beyond it, leaving a named hold for any unsupported part of a threat model, per-attack evidence record, and prioritized control list. The charter boundary review advances with pass for support, fail for contradiction, and hold for unresolved evidence.

Reopen this result after a change to the input, the authority of the system owner, or the workflow condition represented by “testing without written authorization”.

Risk hypothesis

Ask how the risk hypothesis review handles the failure case “attack cases copied from a checklist without system context”. The red-team lead freezes the local portion of authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning before drawing a conclusion.

Use “each result has reproducible evidence” as the explicit criterion for a case drawn from the boundary covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts. The resulting receipt belongs to the data owner.

The security observer records pass, repair, or stop after judging whether “each result has reproducible evidence” holds. No disposition may imply that all of a threat model, per-attack evidence record, and prioritized control list was proven. The risk hypothesis review advances with pass for support, fail for contradiction, and hold for unresolved evidence.

A new dependency, owner, or instance of “attack cases copied from a checklist without system context” expires the evidence for the risk hypothesis review and requires a focused rerun.

Baseline comparison

Reproduce a safe case involving “successful prompts recorded without downstream impact evidence” as the entry condition for the baseline comparison review. The data owner preserves the last state that the workflow can prove.

Reproduce the condition within the boundary covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts, then have the data owner document whether the retained observation supports or contradicts the requirement that “control fixes have retest cases” holds.

The security observer records a decision for the baseline comparison review that cites the evidence for “control fixes have retest cases”. Unsupported parts of a threat model, per-attack evidence record, and prioritized control list remain open. The baseline comparison review advances with pass for support, fail for contradiction, and hold for unresolved evidence.

An altered input source, acceptance owner, or response to “successful prompts recorded without downstream impact evidence” invalidates only this drill and its dependent decisions.

Canary case

Create the canary case review scenario from a safe case involving “unsafe data or tools left in scope”. The data owner records the affected portion of authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning before intervention.

Source the test from a documented scope covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and state the criterion “attacks map to named threat hypotheses” before execution. The remediation owner retains the resulting observation.

Let the security observer decide whether the criterion “attacks map to named threat hypotheses” passed under the recorded conditions. That verdict controls only this review slice. The canary case review advances with pass for support, fail for contradiction, and hold for unresolved evidence.

Repeat the canary case review when the failure case “unsafe data or tools left in scope” appears with new data, permission, or consequences that the data owner did not review.

Stop decision

During the stop decision review, reproduce a safe case involving “controls recommended without a retest condition”. The remediation owner records what remains observable before the next role acts.

Use an authorized test case within the boundary covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts to establish whether “findings separate exploitability from impact” holds. Record configuration and reviewer identity beside the result.

The security observer closes the stop decision review with a bounded ruling on “findings separate exploitability from impact”. The ruling does not certify untested behavior in a threat model, per-attack evidence record, and prioritized control list. The stop decision review advances with pass for support, fail for contradiction, and hold for unresolved evidence.

The next review is triggered when evidence for “findings separate exploitability from impact” becomes stale or the remediation owner loses authority over the case.

Expansion record

Stage a safe instance of “testing without written authorization” inside an authorized fixture for the expansion record review. The system owner notes the last trusted state in authorization, threat modeling, attack-case selection, safe execution, per-attack evidence, control mapping, prioritization, and retest planning.

Retain a boundary record covering authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts, the observed output, and the test for “scope and prohibited actions are signed off”. This makes the decision reproducible.

The disposition belongs to the security observer: accept the evidence for “scope and prohibited actions are signed off”, request a repair, or preserve the current state. The expansion record review advances with pass for support, fail for contradiction, and hold for unresolved evidence.

Create a fresh record when the failure case “testing without written authorization” appears beyond the tested boundary or when the prior evidence becomes stale.

Frequently asked question

How should I pilot LLM Security Red-Team?

Pilot a narrow slice using authorized application access, scope documentation, prohibited actions, and test data. Name incident contacts in a separate role record. Require evidence that scope and prohibited actions are signed off, and stop on the failure case “testing without written authorization”. The security observer records go, revise, hold, or rollback.

A product bridge, with a boundary

The LLM Security Red-Team is the relevant sincLLM offer for this narrow problem. The frozen live catalog describes its required boundary as authorized application access, scope documentation, prohibited actions, and test data, together with a separate assignment of incident contacts and its deliverable as a threat model, per-attack evidence record, and prioritized control list. That catalog statement defines the offer and does not establish buyer-specific fit, technical sufficiency, legal compliance, safety, or business results.