AI Safety: Applying IEC 61508 / ISO 26262 to LLM Deployments

An LLM in a chatbot is annoying when wrong. An LLM recommending medication dosages is dangerous when wrong. An LLM controlling an autonomous vehicle is lethal when wrong. The difference between these three is not the model — it is the consequence of failure. And there is a discipline, fifty years old, that exists specifically to engineer systems where the consequence of failure is severe. It is called functional safety, and it is governed by standards: IEC 61508 (general), ISO 26262 (automotive), DO-178C (avionics), IEC 62304 (medical software).

The AI industry has, with very few exceptions, ignored these standards. The result is a wave of LLM deployments in regulated domains where the engineering substrate that would catch foreseeable failures is simply absent. Medical chatbots. Legal document AI. Drone autonomy. Industrial control. The systems work in demos. They fail in ways that the standards would have flagged, if the standards had been applied.

The Three Pillars of Functional Safety Analysis

Three artifacts dominate functional-safety practice, and they are exactly what the free AI Safety Hazard Analyzer produces:

1. Hazard Analysis (HA)

Every hazard the system can pose to users, operators, bystanders, or the environment, characterized by three parameters from ISO 26262:

• Severity (S1-S4): how badly does the harm injure?
• Exposure (E1-E4): how often is the system in a situation where this harm can occur?
• Controllability (C1-C3): how easily can the operator avoid the harm if the system behaves unsafely?

These three combine into the ASIL rating — Automotive Safety Integrity Level — from ASIL-A (lowest) to ASIL-D (highest). For non-automotive systems, the analogous IEC 61508 rating is SIL-1 to SIL-4. A higher rating mandates more rigorous engineering: redundancy, formal verification, third-party certification, documented test coverage.

2. Failure Mode and Effects Analysis (FMEA)

For each component, what are its failure modes? What is the effect of each failure on the system? What causes the failure? How can it be detected? What mitigates it? The FMEA table is the operational checklist that engineers use when designing redundancy and monitoring. For an LLM system, the failure modes include:

• Hallucinated output (cause: training-distribution gap; mitigation: multi-model verification)
• Refusal on valid input (cause: overzealous safety filters; mitigation: fallback to less-restricted model)
• Token-limit truncation mid-output (cause: poor prompt budgeting; mitigation: explicit max_tokens enforcement)
• Provider downtime (cause: external API failure; mitigation: cached fallback responses)
• Latency spike (cause: provider-side congestion; mitigation: deadline-driven model selection)

3. Fault Tree Analysis (FTA)

Working backward from a top-level system failure ("medication dose recommendation harms patient"), enumerate all the combinations of basic events that could cause it. Each branch is connected by an AND gate (all events must occur) or an OR gate (any one event suffices). The FTA reveals single points of failure, identifies cut sets that an attacker or freak event could exploit, and quantifies the reliability you need from each component.

The SIL/ASIL Recommendation

The Analyzer takes your use-case description and returns a recommended SIL or ASIL level. This matters because it dictates the engineering rigor required:

• SIL-1 / ASIL-A: Low risk. Standard development practice plus documentation. Fine for internal-only LLM tooling.
• SIL-2 / ASIL-B: Medium risk. Requires structured testing, defined safety functions, basic redundancy. Fine for external-facing LLMs with human review.
• SIL-3 / ASIL-C: High risk. Requires formal methods or extensive integration testing, redundant safety functions, independent verification. Required for medical decision-support, financial trading, industrial control.
• SIL-4 / ASIL-D: Highest risk. Requires formal proof of correctness, multiple independent redundant channels, third-party certification. Required for life-critical autonomous systems.

If you describe a chatbot that recommends medication doses to nurses on bedside tablets, the Analyzer should return SIL-3 or higher and recommend specific safeguards: human-in-loop on irreversible recommendations, independent dose verification, audit logging, output bounded by formulary database.

The Deploy Verdict

The Analyzer also returns a verdict in three categories:

• DEPLOY_OK: low-stakes use case, current AI tooling is adequate
• DEPLOY_WITH_GUARDS: medium-stakes; specific safeguards must be in place before deployment
• DO_NOT_DEPLOY: stakes too high for current AI reliability; recommend non-AI fallback or human-only operation

The verdict is conservative by default. If you are designing a system where lives or licenses are at stake, you want the verdict to err toward DO_NOT_DEPLOY rather than DEPLOY_OK. That is the engineering posture the safety standards encode. Hope is not a strategy. Verifiable safeguards are.

From the wiki synthesis on direct AI application: "Reinforcement learning, cyber-physical systems architecture, AI in real-time systems, systems integration methodology — these transfer directly without analogy. CPS thinking applied to LLM deployment IS the safety engineering layer."

Why This Is Underbuilt Right Now

Functional safety expertise is concentrated in industries where the standards have been mandatory for decades — automotive, aerospace, medical devices, industrial control. AI expertise is concentrated in software-first companies that did not need to think about safety standards until they started deploying AI into the regulated domains. The intersection of "fluent in AI" and "fluent in IEC 61508" is statistically tiny.

This tool exists because that intersection is where the next wave of AI deployment will get blocked, lawsuited, or recalled. Run your AI use case through the Analyzer before you ship it. The cost of catching a SIL-3 hazard in design is hours. The cost of catching it in production is everything.

From Audit to Production

The free Analyzer gives you the diagnostic. Building an AI system that actually meets the safeguards it recommends — formal verification, independent redundancy, certified inference, deterministic latency — is engineering work. For deployments in medical, automotive, aerospace, or industrial domains where the standards are not negotiable, see the paid service. The standards exist for a reason. AI does not get a pass.