ISO/IEC TS 22440: The Emerging Direction for Functional Safety and AI Systems

ISO/IEC TS 22440, Artificial intelligence — Functional safety and AI systems, is an emerging standard intended to address the use of artificial intelligence (AI) in safety-related systems. It is currently in Committee Draft and is not expected to be released until early to mid-2027. The final content may change, but the current draft provides a useful indication of how functional safety principles are being adapted for AI-enabled systems.

Unlike ISO/PAS 8800, which is specific to the automotive industry, ISO/IEC TS 22440 is agnostic to industry. It addresses AI within a safety-related control system, AI used in support of safety-related control systems, and AI-based software tools used during safety-related development.

Why Can’t We Just Apply Functional Safety to Develop Physical AI Systems?

The short answer is: we do. However, it is very important to understand the specific differences in how AI, or rather machine learning, algorithms are developed compared to traditional software development. Standards like ISO/IEC 5338 (AI system life cycle processes) use software lifecycle processes, such as ISO/IEC/IEEE 12207, as the framework within which AI characteristics are built. Similarly, in the Physical AI safety space, we need to integrate these unique properties within the framework of functional safety. Addressing this is exactly what drives the need for a standard like ISO/IEC TS 22440.

We could talk about the differences between using explicit rules versus learning from statistical patterns in data. While that is all true, I want to point out two of many key differences that we really need to pay attention to.

First, having a proper data lifecycle is absolutely essential when promoting the safety of AI-enabled systems. In the functional safety world that many of us grew up in, this simply wasn’t the case.

Second, we must have a well-defined continuous operation monitoring strategy. Yes, in functional safety we care about the product after the start of production (SOP); however, this need is amplified by orders of magnitude in the Physical AI systems world.

A robust data lifecycle must consider data quality, how it is synthesized and augmented, and how it is maintained over the product’s lifetime, among other important steps. When the datasets are wrong, we simply can’t trust the inference. If AI is part of a safety-related control system, we cannot assure safe behavior.

Since we cannot fully validate an AI system against an infinite number of scenarios, it will naturally be prone to system misbehaviors. So, even after applying all functional and AI safety measures during development, it is essential to continuously monitor its performance in the field. This requires a strategic deployment plan, such as small fleets with heavy human monitoring and real-time dashboards, to build justifiable assurance arguments. Without a deployment strategy that centers on safety, the likelihood of commercial success drops significantly.

Breaking Down the ISO/IEC TS 22440 Framework

ISO/IEC TS 22440 is planned as a three-part standard:

• Part 1: Requirements (Normative)
• Part 2: Guidance (Informative)
• Part 3: Examples of application (Informative)

The current draft of Part 1 addresses terminology, AI classification, lifecycle considerations for AI safety-related control systems, AI fault analysis, mitigations, testing, and AI-based development tools. Part 2 mirrors the content in Part 1 clause by clause, but it provides informative guidance on how to practically apply those normative requirements. Part 3 provides six hypothetical application examples and walks through how to use the requirements from Part 1.

It is important to note that this standard is not intended to replace existing functional safety standards. IEC 61508, along with other domain-specific functional safety standards, remains the core foundation that ISO/IEC TS 22440 builds upon.

So key aspects of ISO/IEC TS 22440 include:

• Classifying how the AI technology is being used: This determines the path followed within the standard and establishes the required rigor that must be applied.
• Creating an AI safety lifecycle: This includes a dedicated data lifecycle.
• Performing AI fault analyses: This addresses the unique characteristics and properties introduced by AI.
• Establishing AI fault mitigations: This includes both proactive mitigation, avoiding the AI fault, and reactive mitigation, controlling the fault when it occurs.
• Testing the AI appropriately: This ensures we can build justifiable assurance arguments.
• Addressing AI-based software tools: This covers the considerations needed when using these tools during development.

Relationship to Existing Standards

There is already a released technical report on functional safety and AI systems: ISO/IEC TR 5469:2024. It has much of the same look and feel as ISO/IEC TS 22440. However, it can only serve as a guideline. Because it contains no normative requirements and is purely informative by nature, it isn’t auditable. Some of the terminology and approaches differ, but the core ideas are very similar between the two documents.

The reason we stress IEC 61508 over other functional safety standards, such as ISO 13849, is because in many Physical AI systems like humanoids, these alternative standards fall short when trying to support the functional safety of the entire system. ISO 13849 still plays an important role for specific functions and actuators, but the robot as a whole needs something more comprehensive.

The one methodology that is really missing outside of the automotive industry is the handling of system misbehaviors caused by functional insufficiencies and triggering conditions. Although there isn’t a direct industrial equivalent today, it is important to introduce key concepts from ISO 21448, Safety of the Intended Functionality, or SOTIF.

While the Hazard and Risk Assessment plays an important role in defining functional safety risk-reduction measures, it falls short of deriving a complete set of measures when factoring in system misbehaviors. It is therefore critical to use the concepts from ISO 21448 to analyze all functional insufficiencies of the entire AI system, including the perception system, and to understand their triggering conditions. This helps expand the required risk-reduction measures.

Once that foundation is established, we have to use techniques beyond ISO/IEC TS 22440 to discover unknown, unsafe behaviors. This can be achieved through various methods such as safety analyses, real-world data collection, closed-environment testing, simulation, edge-case discovery, and scenario expansion. Once those unknown, unsafe behaviors are minimized, it is important to incorporate all risk-reduction measures through V&V testing. This confirms an acceptable residual risk, or acceptance criteria, for the known unsafe behaviors the system can exhibit and supports the development of the assurance case.

Uncertainty Quantification and ISO/IEC AWI TS 25223

One standard referenced by ISO/IEC TS 22440 that is also currently in development is ISO/IEC AWI TS 25223, which provides guidance and requirements for uncertainty quantification in AI systems. We believe this will be an important direction when considering the safety of Physical AI systems. This standard enables the quantification of uncertainties arising from many of the concerns related to our previous discussion on ISO 21448. It addresses multiple categories of uncertainty, but two in particular are of interest.

First is aleatoric uncertainty, which arises from the world being, in simple terms, noisy. This represents the inherent and unavoidable randomness in our physical environment. We can generally break this down into three main areas: sensor noise and limitations, environmental variability, and unpredictable human behavior.

The second main category considered in ISO/IEC AWI TS 25223 is epistemic uncertainty, which arises from a lack of knowledge or insufficient training data. When looking at epistemic uncertainty, we are typically dealing with out-of-distribution (OOD) scenarios, commonly known as “edge cases.” These are scenarios we never accounted for during training, and therefore, the Physical AI system simply lacks the knowledge to handle the situation.

Quantifying these effects during runtime will be a critical ingredient in designing a safe system that can resort to a safe state, or execute a minimal risk maneuver (MRM), when exposed to such scenarios.

One final thought on tying ISO/IEC TS 22440 activities into actual development is the importance of building a safety case. In fields where we run into standards fatigue, applying a Goal Structuring Notation (GSN) safety case can be highly beneficial. You can start at a very high level and make the claim, “The robot is acceptably safe to collaboratively operate with humans.”

This claim is then systematically broken down into various paths or pillars, considering all aspects of safety, from functional safety to security concerns, operations, and beyond. At the bottom of the GSN, we then establish the acceptable evidence needed to back up those higher-level claims.

Considerations Before Publication

Organizations do not need to wait for final publication to begin addressing the underlying issues.

A practical starting point is to identify where AI performs, supports, or influences a safety-related function. Teams can then define the intended operating conditions, identify relevant data and model assumptions, analyze AI-specific failure modes, and determine what monitoring or architectural controls are needed around the AI function.

The key point is not that conventional functional safety must be replaced. Safety-relevant AI-enabled systems may require additional evidence and controls where behavior is learned from data, cannot be fully specified in advance, or may change in operation.

To explore these concepts in more detail:
Join SRES’ ISO/IEC TS 22440 Functional Safety and AI Systems Training