ISO 26262 is like bringing a knife to a gun fight with AI
In the rapidly evolving automotive industry, artificial intelligence (AI) and machine learning (ML) algorithms have emerged as a game-changer, promising safer, more efficient, and self-driving vehicles. However, relying solely on functional safety and ISO 26262 to address the challenges of AI-powered systems is like bringing a knife to a gunfight – inadequately prepared for the complexities and unpredictabilities AI and ML introduce. Automotive functional safety, outlined in the ISO 26262:2018 standard, has long been the gold standard for ensuring automotive electronic systems’ safety – Although it will still be a crucial aspect of AI safety, it will no longer be sufficient to ensure the overall safety of AI-powered systems. Here are a few reasons why functional safety must be supplemented with additional measures to adequately prepare AI-powered automotive systems and development tools for the challenges they face:
Data Quality and Training
Functional safety standards do not directly address the quality and diversity of training data, which significantly influence AI systems’ performance. Inadequate or biased data can lead to incorrect predictions or actions, potentially causing accidents and endangering lives.
Robustness
AI-enabled systems like self-driving cars must navigate unpredictable real-world scenarios gracefully. Functional safety, while excellent at dealing with known failure modes, does not adequately address the ever-changing and unanticipated situations robotaxis encounter on the road.
Continuous Learning
AI systems evolve continuously during development and after release. They adapt and learn over time, which necessitates ongoing monitoring and updates. Functional safety, built for systems that are more static in nature after their release, is insufficient for the dynamic nature of AI-enabled systems. To adequately evaluate AI-powered automotive systems for the battle on the road, the automotive industry is developing comprehensive safety frameworks that go beyond traditional functional safety.
Data Validation and Quality Assurance
High-quality training data, validated sources, and data quality assurance processes are vital to sharpen AI systems. Addressing bias and ensuring data integrity is critical. ISO/CD PAS 8800 – “Road Vehicles Safety and Artificial Intelligence” includes these considerations to build a more overall safety argument framework for AI systems safety in the automotive industry.
Robustness Testing
Rigorous testing procedures serve as the versatile arsenal, exposing AI systems to a broad spectrum of real-world scenarios to ensure they can adapt effectively to any situation. The ISO 21448:2021 “Safety of the Intended Functionality” standard, ISO/TR 4804 – “Road vehicles Safety for automated driving systems” standard, supported by the ISO 3450x series of standards together provide a strong framework for appropriately verifying and validating AI-driven autonomous vehicle systems operating across their stated operational design domain (ODD).
Continuous Monitoring and Updating
Ongoing monitoring and updates provide the real-time intelligence and adaptive strategies needed for AI systems to evolve and respond to changing conditions. In addition to the development of minimal risk maneuvers defined using ISO 21448:2022, there are standards like ISO/AWI TS 17691 “Road vehicles Principles for human remote support of automated systems” that attempt to define best practices for intervention for low-speed autonomous vehicles.
To adequately prepare for the challenges and complexities introduced by artificial intelligence, a comprehensive approach that encompasses data quality, robustness, and continuous monitoring is necessary.
Some organizations have claimed to assess and certify the safety of ML algorithms for autonomous systems, like a radar sensor system, using only ISO 26262 and IEC 61508. But this is far from complete.
If the only tool you have is a hammer, you tend to see every problem as a nail.
Over-reliance on ISO 26262 can result in approaching AI problems in ways that are not always helpful or even destructive. The additional standards and frameworks that are highlighted above are by no means perfect or exhaustive but they are a step in the right direction when functional safety thinking is simply only scratching the surface when it comes to AI/ML risks. Let’s get to work!
