SRES Fireside Chat: Validation and Data in AI-Driven Systems
How do organizations build confidence in AI-driven, safety-critical systems when their behavior is learned from data rather than explicitly programmed?
In this Fireside Chat, SRES partners Jody Nelson and Gokul Krithivasan are joined by Drago Špoljarić, Principal Engineer, and Ivan Peris, Senior R&D Engineer, from Visage Technologies to explore verification, validation, and data assurance for AI-driven systems.
They examine the limits of conventional testing, the distinct roles of open-loop and closed-loop evaluation, the balance between real-world testing and simulation, and the challenge of addressing unknown unsafe scenarios. The discussion also considers data coverage, quality, synthetic data, and building assurance for probabilistic, data-dependent behavior.
Drawing on perspectives from AI assurance and real-world AI product development, this discussion is relevant to engineers and technical leaders working on automotive, autonomous, robotic, and other safety-critical systems.
Watch the full discussion below:
Selected Audience Q&A
The following questions were raised by attendees during the live session.
How can organizations approach requirements-based testing for AI-based physical systems when requirements are probabilistic rather than atomic?
For AI-based physical systems, requirements must first be framed with meaningful acceptance criteria and quantitative metrics. In automotive applications, vehicle-level safety strategies and driving policies established through ISO 21448 can provide the higher-level context. ISO/PAS 8800 then expects AI safety requirements to address expected model behavior, relevant safety properties such as robustness and accuracy, and dataset requirements.
These requirements often begin qualitatively, then mature into quantitative, statistically evaluated acceptance criteria. The key challenge is defining criteria that can be justified as sufficiently safe for the intended application.
How practical is explainable AI (XAI) for autonomous trucking and autonomous driving?
XAI can be valuable when investigating unexpected system behavior. Teams can isolate a specific situation and use techniques such as heat maps to better understand why an AI model made a particular decision.
However, applying XAI to every decision across a complex autonomous system is not generally practical. Its strongest role is in targeted analysis and spot checks rather than as a complete strategy for explaining all AI behavior in operation.
What can teams do when real-world data are extremely limited?
Limited real-world data can be used to evaluate synthetic data or the tools that generate it. By comparing critical attributes of real and generated data, teams can assess whether synthetic data is sufficiently representative to augment the available dataset.
Synthetic data can help address scenarios that are difficult, costly, or unsafe to capture in the real world, including challenges related to sensors and sensor positioning. It does not fully replace real-world data, but it can expand coverage when used with a clear qualification and validation strategy.
Further Reading: Early Detection of Dataset Insufficiencies in ADAS
In this Visage Technologies white paper discussed during the Fireside Chat, learn how dataset insufficiencies can contribute to AI errors and vehicle-level hazards—and how teams can identify and mitigate those gaps earlier in the dataset lifecycle.
The paper explores ODD-based dataset requirements, metadata and traceability, labeling and annotation risks, data review, and dataset safety analysis in the context of ISO 8800 data-related safety properties for safety-relevant ADAS perception systems.
Meet the Speakers
Jody Nelson
Co-Founder & Managing Partner, SRES
Jody is a veteran safety leader with more than 24 years of experience advancing safety-critical automotive and autonomous systems for OEMs, suppliers, semiconductor companies, and autonomy developers. A former global assessment and certification lead at UL Solutions and co-founder of kVA and SRES, he combines hands-on development experience with consulting and independent assessment to deliver rigorous, defensible safety outcomes.
Gokul Krithivasan
Co-Founder & Managing Partner, SRES
Gokul is a senior safety expert with more than 14 years of experience in automated driving systems and safety assurance for complex interconnected products. He has supported technology companies, OEMs, and suppliers; led expert teams across safety, cybersecurity, and AI/ML; and brings experience spanning autonomous robotaxis, autonomous logistics, and robotics. He is a certified expert in ISO 26262, ISO 21448, and ISO 8800.
Drago Špoljarić
Principal Engineer, Visage Technologies
Drago is a Principal Engineer at Visage Technologies with nearly a decade of experience developing computer vision and AI systems. His work includes ADAS functions, real-time computer vision on embedded automotive hardware, and the application of perception and safety expertise to off-highway equipment, industrial vehicles, and mobile robotics. He holds a Ph.D. in Mathematics and Statistics from the University of Zagreb.
Ivan Peris
Senior R&D Engineer, Visage Technologies
Ivan is a Senior R&D Engineer at Visage Technologies, where his work focuses on the functional safety of AI-driven systems. He brings experience from AI and computer vision development as well as prior work as a development engineer at a major automotive Tier 1 supplier, supporting the development of safety-critical systems.
About the Series
The idea for a series of Fireside Chats traces back to the very early days of Jody and Bill Taylor’s work together. In one of their first training sessions, a tricky FMEDA discussion left participants frustrated. That evening, the two of them stayed up late sketching a new way to explain it — an approach that ultimately found its way into the second edition of ISO 26262.
That moment sparked a tradition of technical discussions that has continued for years, now with the SRES team. These sessions have become a cornerstone of how we work — exploring difficult questions, challenging assumptions, and pushing toward better answers.
With these Fireside Chats, we’re opening that tradition to the broader safety community. Each session will bring the same candid, technical conversations that shape our work to a public forum.
Related Services
If you’re actively working on the topics discussed in this Fireside Chat, the following SRES services may be helpful.
SRES Consulting
SRES supports organizations developing automotive, autonomous, robotic, and other Physical AI products. Our experts help teams apply functional safety, AI safety, validation, safety assurance, and responsible AI practices to complex safety-critical systems.
SRES Training
SRES offers practical training across functional safety, AI safety, cybersecurity, SOTIF, and related safety and security standards for automotive and Physical AI applications.
