Skip to content
sinc-LLMBook a 15-minute call

How to Test a Prompt: Criteria, a Worked Example and a Pass/Fail Sheet

Test a prompt like code: write success criteria, build a case set with edge cases, grade every output and confirm the fix on cases the prompt has never seen.

On this page

By Mario Alexandre · November 19, 2026 · 7 min read

The short answer: to test a prompt, decide what a correct output looks like before you run anything, then run the prompt on 30 to 50 realistic inputs that include awkward cases, grade every output with a check you could repeat, and fix the prompt only against failures you can name. Keep 10 to 20 cases aside that you never edit against, and run them last. A prompt passes when it meets your criteria on those held-out cases, not when a few outputs look good.

What does it mean to test a prompt?

Testing a prompt means measuring its outputs against criteria you wrote in advance. Anthropic's guidance for building LLM applications puts it as a cycle: define success criteria, develop test cases (including edge cases), test the prompt, refine it, and then "Test against held-out evals" before shipping. The same guidance asks for criteria that are specific, measurable, achievable and relevant to the application.

A prompt test therefore has four parts:

  1. Criteria. What must be true of every output, written so two people would grade it the same way.
  2. Cases. Inputs that look like real traffic, plus the cases that tend to break prompts: mixed requests, empty or very long inputs, other languages, missing information.
  3. A grader. Code where the answer can be checked mechanically, a rubric-driven model grader where judgment is needed, and people only where neither works.
  4. A record. Which prompt version ran on which cases, with which model and settings, and what each case scored.

Without the record you cannot tell whether a change helped or whether you simply looked at different examples.

How to evaluate a prompt: a worked example

The example below is fictional. A software company wants a prompt that reads an incoming support ticket and returns JSON with a category (billing, bug, account or other) and an urgent flag that is true only when the customer reports they cannot use the product at all.

Step 1: write the criteria. The team agrees on three, each with a number:

  • The output parses as the agreed JSON on 100% of cases.
  • The category matches the support lead's label on at least 90% of cases.
  • The urgent flag is never true for a ticket the lead marked as not urgent. A false alarm pages an engineer at night, so the tolerance is zero.

Step 2: build the case set. The lead pulls 40 real tickets from last month, removes customer names and labels them. Thirty are ordinary. Ten are chosen because they are hard: three mention both a charge and a bug, two are in Portuguese, two are a single line, one is empty, one quotes an earlier ticket and one is a reply that only says "thanks". Twenty more tickets go into a separate held-out file that nobody reads while editing the prompt.

Step 3: grade version 1. A short script parses each output and compares it with the label. Version 1 parses on all 40 cases, gets the category right on 34 (85%), misses one urgent ticket and raises one false urgent flag.

Case Ticket (shortened) Expected Version 1 output Grade
7 "Charged twice and now the export button crashes" billing, urgent false bug, urgent false Fail: category
12 "Não consigo entrar na conta desde ontem" account, urgent true other, urgent false Fail: category and flag missed
19 "" (empty) other, urgent false bug, urgent false Fail: category
23 "The whole app is down for our team" bug, urgent true bug, urgent true Pass
31 "Thanks, that fixed it" other, urgent false other, urgent false Pass
38 "Invoice shows wrong VAT, need it before Friday" billing, urgent false billing, urgent true Fail: false urgent flag

Step 4: fix against named failures. Version 1 made eight mistakes: six wrong categories, one missed urgent flag and one false urgent flag. They fall into four groups. The three mixed tickets need a tie-break rule, so the team adds "if a ticket reports a charge problem, the category is billing, even if it also reports a bug". The two Portuguese tickets need one sentence saying the ticket may be in any language; that also fixes the missed urgent flag on case 12, because a customer who cannot log in cannot use the product. The empty ticket needs a rule that a ticket with no content is "other". The false urgent flag came from the word "need" with a deadline, so the definition of urgent gains an explicit exclusion: deadlines and frustration are not urgency; only being unable to use the product is.

Step 5: re-run everything, then the held-out file. Version 2 parses on all 40 cases, gets 38 of 40 categories right (95%), flags case 12 as urgent and raises no false urgent flags. On the 20 held-out tickets it scores 18 of 20 (90%), still with no false alarms. That meets all three criteria, so version 2 ships. If the held-out score had dropped well below the score on the edited cases, the fixes would have been tuned to those 40 tickets rather than to the task, and the team would have gone back to step 4.

How to grade outputs without grading everything by hand

Anthropic's guidance ranks grading methods by speed and reliability. Code-based grading, such as exact match or checking that a key phrase is present, is "Fastest and most reliable, extremely scalable" but cannot judge nuance. Human grading is "Most flexible and high quality, but slow and expensive". Model-based grading sits between the two and should be tested for reliability before you scale it.

In practice:

  • Use code for anything with a fixed answer. JSON validity, a category, a number, whether a required field or disclaimer is present.
  • Use a model grader with a rubric for judgment. Give it one criterion at a time and ask for a fixed output such as "correct" or "incorrect", or a score from 1 to 5. Grade a sample of its verdicts yourself before you rely on it.
  • Use people to label the reference answers. The support lead's 60 labels in the example took about an hour and were the most valuable hour of the project.

The same guidance also says to "Prioritize volume over quality": more cases with slightly noisier automated grading usually beat a handful of carefully hand-graded ones.

How to check prompt quality in 15 minutes

When you do not yet have a labelled set, a short check still catches the most obvious problems:

  1. Write one sentence that states what a correct output is. If you cannot, the prompt is not ready to test.
  2. Collect ten real inputs, including the two most awkward ones you can find.
  3. Run each input three times. If the answers differ on a case with one correct answer, the prompt leaves too much to chance on that case.
  4. Mark each output pass or fail against your sentence, and write down why each failure happened.
  5. Change one thing in the prompt, then re-run all ten, not only the failures.

The pass/fail sheet

Copy this sheet for each prompt version. A version ships only when every row passes on the held-out cases.

Check How it is measured Pass when
Format Output parsed by code 100% of cases parse
Task accuracy Output compared with reference labels At or above your agreed threshold
Costly errors Count of the one error you cannot tolerate Zero, or your stated limit
Edge cases Pass rate on the tagged hard cases No group fails entirely
Stability Same case run three times Same answer where one answer is correct
Generalization Score on held-out cases Within a few points of the edited set
Record Prompt version, model, settings, date Written down next to the scores

Get a second pair of eyes on one prompt

If one prompt is underperforming and you want an outside review before building a test suite, Prompt Audit reviews it and returns a rewritten prompt in its sinc JSON format, with a before and after signal to noise measurement and the analysis behind each change, within 48 hours, for $49. Send the current prompt and a description of the ideal output.

If your team needs an evaluation suite and regression harness built around its agreed examples, so quality is checked before a prompt or model change is released, LLM Eval Harness is the scoped project for that.

Book a 15-minute call about testing your prompts · Request an estimate by email

Frequently asked questions about testing prompts

How many test cases do I need?

Enough to cover each kind of input the prompt will see, with several examples of each hard case. Thirty to fifty labelled cases plus a held-out set of ten to twenty is a practical start for one prompt. Add every real failure you find in production to the set, so the same mistake cannot return unnoticed.

Can I use an LLM to grade my prompt's outputs?

Yes, for criteria that need judgment, as long as the grader gets a clear rubric and a fixed output format, and you check a sample of its verdicts against your own before trusting it. For anything with a single correct answer, a code check is faster and more reliable.

Should I re-test when the model version changes?

Yes. Run the full case set and the held-out set on the new model before switching. A prompt that passed on one model version can fail on the next, and the case set is what lets you see it before your users do.

Sources

Anthropic's documentation describes the test cycle, the success criteria and the trade-offs between code, human and model grading used in this guide. The ticket prompt, its thresholds and its scores are a fictional example. Anthropic: Define success criteria and build evaluations