AI, Statistics and Data Science in Practice Webinar: Tim Hesterberg - Covariate Adjustment, Intro to Resampling, and Surprises

Friday, October 3, 2025 - 12:00pm to 1:30pm ET

Abstract

Covariate Adjustment, Intro to Resampling, and Surprises

Covariate adjustment is a relatively simple way to improve A/B test accuracy and velocity, and to reduce bias. Huh? Aren't randomized experiments unbiased? Well, yes and no - yes by the usual statistical definition, but no by a more common-sense definition. Furthermore, in practice there are some complications, particularly related to measuring how accurate the results are. Enter resampling (bootstrap, jackknife) - generally useful techniques for measuring accuracy. These bring their own surprises. Remember the old n >= 30 rule for using normal approximations? We'll see just how bad that is.
 

Tim Hesterberg, Staff Data Scientist at Instacart

Moderator

Coming soon!
 

About the Speaker

Dr. Tim Hesterberg is a Data Scientist at Instacart, with previous experience as a DS at Google. His career has spanned multiple domains, including positions in academia, energy, and software development. He earned his Ph.D. in Statistics from Stanford University and his B.A. in Mathematics from St. Olaf College, and also spent two years studying in Germany. Dr. Hesterberg is co-author, with Laura Chihara of Carleton College, of Mathematical Statistics with Resampling and R (Wiley), a widely used text in statistics education. He has also contributed to instructional materials on teaching introductory statistics using resampling methods, in collaboration with colleagues including David Moore. His research and applied work cover a wide range of topics, including statistical methodology, electric demand forecasting, web traffic analysis, clinical trials, display advertising, computer vision, and natural phenomena such as streams and earthquakes. He has served in leadership roles with the Canadian Statistical Sciences Institute and the National Institute of Statistical Sciences and was a contributing author to the American Statistical Association’s Guidelines for Undergraduate Statistics Programs. Fun version: https://www.timhesterberg.net/home/tim-hesterberg-bio-sketch
 

About the Moderator

Moderator coming soon!

About AI, StAtIstics and Data Science in Practice

The NISS AI, Statistics and Data Science in Practice is a monthly event series will bring together leading experts from industry and academia to discuss the latest advances and practical applications in AI, data science, and statistics. Each session will feature a keynote presentation on cutting-edge topics, where attendees can engage with speakers on the challenges and opportunities in applying these technologies in real-world scenarios. This series is intended for professionals, researchers, and students interested in the intersection of AI, data science, and statistics, offering insights into how these fields are shaping various industries. The series is designed to provide participants with exposure to and understanding of how modern data analytic methods are being applied in real-world scenarios across various industries, offering both theoretical insights, practical examples, and discussion of issues.

The AI, Statistics & Data Science in Practice Series during Fall 2025 will focus on the critical role of experimentation in the development and refinement of artificial intelligence (AI) systems: "Incorporating principles of design of experiments and randomization ensures that AI models are trained on reliable, unbiased data, leading to more generalizable and interpretable results. By planning data collection with experimental design and randomization, researchers can minimize bias from uncontrolled variables and improve the statistical validity of their conclusions, whether the models are inferential or predictive. However, in many real-world scenarios, fully controlled experiments may not be feasible. When working with observational data, researchers can employ quasi-experimental techniques to approximate the benefits of randomized trials. These methods help isolate the effects of key variables and adjust for potential confounders, improving the robustness of AI-driven insights. By integrating structured experimentation and causal inference methodologies, AI developers can enhance the reliability and applicability of their models in practice.

Featured Talks:

 

Event Type

  • NISS Hosted

Host

National Institute of Statistical Sciences

Cost

Free Webinar

Website

Register on Zoom here!

Location

Online Zoom Webinar

Policy