Hallucination Radar: Applying Radar Detection Theory to LLM Outputs

Most AI apps catch hallucinations by hoping users notice. They take what the model says and pass it on. When the model says something wrong with confidence, the system has no way to catch it. Users find the error by getting hurt.

Another field already solved this. Radar engineers have separated real signals from noise since 1940. The field is called detection theory. The same math that decides if a blip on a screen is a plane or a bird can decide if an AI output is true or made up.

The Setup: Three Models, One Truth

The Hallucination Radar is a free tool you can try right now. It sends any AI claim to three independent free models at the same time:

• Nvidia Nemotron 3 Super 120B: a large reasoning model with 262K context
• Google Gemma 4 31B: a dense multimodal model, fast at structured output
• MiniMax M2.5: a different architecture trained on independent data

Each model gets the same fact-checker prompt. Each one must return strict JSON: a verdict (LIKELY_TRUE, UNCERTAIN, or LIKELY_FALSE), a confidence score, the concerns it found, and facts it can back up.

Three independent checks of the same claim. That is the heart of detection theory.

Why Three Models, Not One

One model can be confidently wrong. Two models trained on similar data can both be wrong in the same way. Three models with different architectures, training data, and fine-tuning are much harder to fool in the same direction. When all three agree, that agreement is a real signal. When they disagree, that disagreement is also useful. It tells you the claim is not settled.

In radar terms, this is a three-receiver array. One receiver picks up real signals and noise. Three receivers working together cut out the noise. The chance of a false alarm (Pfa) is much lower for the three together than for any one alone. The chance of catching a real signal (Pd) stays high.

The ROC Tradeoff Made Visible

Every detector makes a tradeoff. Tighten the threshold and you miss real hallucinations. Loosen it and you flag everything. The curve that plots false-alarm rate against detection probability is called the Receiver Operating Characteristic (ROC) curve. Choosing where to sit on that curve is the main engineering decision in any detection system.

The Hallucination Radar makes this visible. The final verdict represents one specific point on the curve. It only flags a claim when two or three of the models agree. The agreement flag in the telemetry tells you which case it was. The confidence score tells you the average certainty of the models that agreed.

From a wiki synthesis I built mapping radar concepts to AI: "Emergence detection IS radar detection theory. Pfa = emergence alert false positives. Pd = catching real threats. Detection threshold = stuck predicate firing conditions. ROC curve = tradeoff between alert sensitivity and alert fatigue."

What Visible Telemetry Looks Like

When you run the tool, the display shows:

• Audit ID: every run is logged so you can trace it later
• Models returned: how many of the three actually replied (free-tier OpenRouter rate limits are real)
• Per-model latency: the slowest model controls how long the whole verdict takes
• Agreement: a simple yes or no showing whether the three models agreed
• SNR / Confidence: averaged across the models that responded, lower when they disagree
• Fallback events: a visible record of when a model was rate-limited or failed

This is not a chatbot. This is what a real hallucination detector looks like. The telemetry is visible. Failure modes can be inspected. Every decision is traceable.

How to Use It in Your Workflow

Three ways to use it:

1. Check before publishing: paste an AI paragraph before you send it to a customer. If the three models disagree, review it by hand.
2. Spot-check past outputs: paste a response a customer questioned. The radar will tell you if it was actually wrong.
3. Learn the pattern for your own pipeline: the Hallucination Radar shows you how multi-model verification works. Build the same structure into your production stack.

The third use case is where this becomes real engineering. The design shown, including three independent calls, structured-output prompts, JSON parsing, agreement aggregation, and ROC-tuned thresholds, is something you can copy. The live tool is a working reference. The public audit feed shows real runs.

The Pattern Behind It

This tool exists because I found a direct connection: radar detection theory applies to AI hallucination detection. The mapping is clean. Pfa maps to false-alarm rate. Pd maps to detection probability. A matched filter maps to known hallucination signatures. A three-receiver array maps to parallel model calls. Each concept has decades of engineering research behind it. I pull that research into AI deployment.

For production deployments where the stakes are high, you may need ROC-tuned thresholds, custom matched-filter signatures, Kalman state estimation on streaming outputs, and live monitoring dashboards. See the paid service below.