Safety and Artificial Intelligence – A look into the ISO 8800 Standard
This article offers an in-depth look at topics related to Functional Safety and Responsible AI.
For expert-level training—including certification-based programs—on these topics and more, explore our Automotive trainings and Responsible AI trainings. To learn how we support product development, compliance, and organizational safety goals with consulting support, visit our Functional Safety & Cybersecurity and Responsible Artificial Intelligence pages—or contact us directly.
As of October 17, 2024, the ISO/PAS 8800 standard went under publication. Generally, it takes 7 weeks from this time for the standard to be released. It is expected that the Road Vehicles – Safety and artificial intelligence standard, ISO 8800, will become available sometime in December of 2024.
The ISO 8800 standard is an extension of concepts from the ISO 26262 (functional safety) and ISO 21448 (safety of the intended functionality – SOTIF) standards. ISO 26262 addresses safety risks when E/E systems (including software) break, or malfunction, as shown by the development process in Figure 1. Whereas the ISO 21448 standard addresses when E/E systems are not performing good enough for the operational environment, even in the absence of malfunctions. ISO 21448 focuses on the functional insufficiencies, which include insufficiency of specifications or performance insufficiencies. Neither standard addresses the specific malfunctions or insufficiencies of artificial intelligence (AI) with respect to safety. That is where the ISO 8800 standard comes in and is needed, including autonomous vehicles (AV). While the standard itself doesn’t address a particular AI or machine learning (ML) technology (e.g., Deep Neural Networks (DNNs)), it aims to cover the underlying insufficiencies of AI/ML data and models used in safety-critical automotive applications.
Figure 1: Automotive functional safety development process.
It is common in AVs to have an incomplete picture, or specification, of the complete operational domain. In those cases, AI/ML is often utilized. ISO 8800 evaluates the impact on AI due to performance insufficiencies such as errors due to bias and lack of robustness, insufficiencies in training data, and many other aspects. It introduces an AI safety lifecycle specific to automotive, while aligning with the safety lifecycle prescribed by the ISO 26262 standard. The ISO 8800 will take in requirements derived from ISO 26262 and ISO 21448 but extend the requirements to properties specific to AI such as bias, prediction, robustness and generalization. There are methods prescribed allowing one to reduce the ML performance requirements by compensating using other methods, such as non-AI monitors or supervisors. We can also expect to have safety requirements and safety analysis associated with data sets, since the quality of data plays a significant role in ML.
In addition to combining aspects of all three of these standards, it will be required to have a robust quality management system as a basis for risk management. The ISO/IEC 42001 standard establishes an AI management system which can also be aligned to responsible AI and can support efforts toward EU AI Act compliance. Lastly, the ISO/IEC 5469 technical report provides practical design guidance for addressing functional safety concerns with AI systems. Figure 2 shows the updated development process in automotive when considering safety applications of AI systems.
Figure 2: Automotive safety AI development process. Note: *ISO/IEC 5469 is a Technical Report and is expected to be replaced by the ISO/IEC AWI TS 22440 standard.
There is a complex landscape evolving around standards and regulations for the use and development of AI. This article only addresses a small subset specific to the automotive industry. At SRES we provide consulting and training related to responsibly safe and secure development of products.
