Thursday, November 12, 2026 - 1:00pm to 2:00pm ET
Abstract
As machine learning (ML) systems are increasingly integrated into all aspects of society, including high-consequence systems, ensuring that these models are worthy of our trust has become a central challenge. The concept of trustworthiness in ML spans many different dimensions (such as reliability, security, explainability) which can be in tension with one another. Moreover, to the extent that we can measure trustworthiness, there can be significant gaps between what different stakeholder groups understand about the models and what evidence they need to make informed trust decisions — we refer to these as information asymmetries; even the developers of ML models themselves may have limited insight into the behaviors of their own products. These gaps can lead to both harmful over-trust and under-trust in ML-enabled systems, and the ongoing work of our team at Sandia is to research methods and tools for closing those gaps.
Speakers
Reed M. Milewicz Ph.D., Computer Scientist and Software Engineering Researcher, Department of Software Engineering and Research, Sandia National Laboratories
Michael Darling, Data Scientist, Department of Secure Algorithms, Sandia National Laboratories
J. Jake Nichol, Senior Member of Technical Staff, Department of AI Credibility, Sandia National Laboratories
About the Speakers
Reed M. Milewicz Ph.D., is a computer scientist and software engineering researcher in the Department of Software Engineering and Research at Sandia National Laboratories. His research focuses on developing better practices, processes, and tools to improve software development in the domains of science and engineering. Reed's current work involves serving the PI for DevSecOps Research for the Advanced Simulation and Computing (ASC) program at Sandia, and as the PI of the DARPA PROVERS evaluation program which seeks to improve the usability and efficacy of formal methods tools in high-consequence software development contexts.
Michael Darling is a Data Scientist in Sandia National Laboratoy’s Secure Algorithms Department. His work focuses on trustworthy machine learning, including adversarial robustness, uncertainty quantification, and secure and reliable ML development. Michael joined Sandia in 2015 as an intern, first in the Risk and Reliability Department and then in the Machine Intelligence Department, before transitioning to Secure Algorithms in Livermore as a staff member. He holds a Ph.D. in Computer Engineering from the University of New Mexico (2019, Advised by Don Hush); his doctoral work centered on methods for quantifying and interpreting uncertainty in machine learning predictions.
J. Jake Nichol is a Senior Member of Technical Staff in the AI Credibility department at Sandia National Laboratories. His primary focus is model credibility: how well the evidence shows that a model approximates the real-world process behind its data. In high-consequence settings, where models need to be reliable, that evidence has to be strong enough to act on. Jake leads the credibility team on Sandia's trustworthy machine learning project, co-leads a team that builds cause-and-effect models to detect threats to the power grid, and studies whether targeted changes can make AI weather models more or less realistic. His background is in causal discovery, the study of how to infer cause and effect from data. He earned his PhD in Computer Science at the University of New Mexico in 2025. Advised by Melanie Moses and Matthew Fricke, he developed CaStLe, a method that maps cause and effect across space and time in Earth science data, and used it to trace how the 1991 eruption of Mount Pinatubo affected the atmosphere.
About the Moderator
Emily M. Taylor, LANL
About the NISS-CANSSI Collaborative Data Science Web Series:
The NISS-CANSSI Collaborative Data Science initiative that the National Institute of Statistical Sciences (NISS) in collaboration with the Canadian Statistical Sciences Institute (CANSSI) brings together experts from various fields to tackle complex data challenges through interdisciplinary teamwork and innovative methodologies.
Goals of the Initiative
The goal is to foster progress in:
- Developing new ideas for experimental and observational data-driven learning and discovery that address key questions at the cutting edge of science and scientific deduction;
- Quantifying and summarizing uncertainty in data-driven theories, as well as complex Data Science models, algorithms, and workflows; and
- Establishing new practices for scientific reproducibility and replicability through Data Science.
Learn more: NISS-CANSSI Collaborative Data Science
Event Type
- NISS Hosted
Website
Location
United States
Policy
