Ingram Olkin Forum Series: Unplanned Clinical Trial Disruptions - Estimands and Missing Data

September 1, 2020 10 - 1 pm ET

[Please Note: This session has already occurred.  Go to the News Story for this event to read about what happened. And click here for the entire IOF series.]

Session Three:  Estimands and Missing Data

In recent years there has been increasing scrutiny of precisely what treatment effect is being estimated (the estimand) by a clinical trial, in the context of complications such as treatment switching and discontinuation and competing risks. Last year the ICH E9 addendum on estimands laid out a framework for defining a variety of different estimands. Disruptions to the conduct of clinical trials due to COVID-19 raise a range of questions in regards to trial estimands. Should estimands be changed because of COVID-19 disruptions? Should statistical analysis methods for handling missing data be modified? This session aims to explore the estimand and missing data issues raised by COVID-19 disruptions, with a view to stimulating new research efforts to address these and other questions. 

The first two sessons were free and open sessions and served the purpose of exposing the statistical issues surrounding unplanned disruptions of clinical trials and laying the foundations for the issues to be addressed in sessions 3-6, and possibly more.


Please read the information about registration carefully.  

1) If you attended either Day One and/or Day Two of this forum AND have filled out the Registration of Interest Form, then all you need to do is to select a registration/payment option above the 'Register for this Event' button ($25/session, $75 for forum sessions Day 3-6.). NISS affiliates, (, please send an email to

2) If you attended either the Day One and/or Day Two sessions of this forum but did not fill out the Registration of Interest Form.  Please do so BEFORE you select a registration/payment option above the 'Register for this Event' button to pay for Day Three.

3) If you did not attend either the Day One or Day Two sessions of this forum, please watch the videos from one or more of the prior days to "get up to speed" on this working series, (recordings of the sessions as well as the slides used by the speakersBEFORE you fill out the Registration of Interest FormAfter completing the Registration of Interest form, then pay as explained above.

Forum Organizing Committee

Jonathan Bartlett, University of Bath
Adam Lane, University of Cincinnati
Nancy Flournoy, University of Missouri
Chris Jennison, University of Bath
Assaf Oron, Institute for Disease Modeling
Sergey Tarima, Medical College of Wisconsin


10:00-10:15 Introduction 

James Rosenberger (NISS director, Pennsylvania State University)
Nancy Flournoy (chair of the organizing committee, University of Missouri) 
Jonathan Bartlett (the session organizer, University of Bath)


David Murray, (Office of Disease Prevention, NIH)

Unplanned Disruptions in Pragmatic Clinical Trials: Examples from the Health Care Systems Collaboratory

Abstract: The designs most commonly used in pragmatic clinical trials are traditional randomized controlled trials, group- or cluster-randomized trials, stepped-wedge group- or cluster-randomized trials, and individually randomized group-treatment trials. This presentation will provide an overview of the design and analytic methods for these designs, focusing on the designs that involve randomization of groups or clusters or delivery of interventions in a group-format. These designs will be illustrated using examples from the Health Care Systems Collaboratory Project, supported by the NIH Common Fund beginning in 2012, and now involving 19 trials. The focus will be on unplanned disruptions of these trials due to the COVID pandemic and on how the trials adapted in response. The Collaboratory Biostatistics and Design Working Group is working with the current trials to help them develop adaptations to adjust to the pandemic. This work is guided by the PICOT framework and by the CONSORT reporting guidelines for these designs.


Mouna Akacha, (Novartis)

"COVID-19 – Estimands to the rescue?"

Abstract: The COVID-19 pandemic results in various complications for subjects and sites participating in ongoing clinical studies. Some of these complications affect the interpretation of the measurements associated with the clinical question of interest while others prevent relevant data being collected and result in a missing data problem. In this presentation, we will discuss the value of the estimand framework (ICH 2019) in
•    assessing the impact of the pandemic on the scientific question of interest that is targeted by a given trial;  
•    identifying the relevant data and information related to the pandemic that needs to be collected;
•    supporting proper analysis and interpretation of the trial data
ICH (2019), “Topic E9(R1) on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials,” available at

11:15-11:25  -  Break


Suzie M. Cro, (Imperial College of London)

"Handling unplanned disruptions in randomized trials using missing data methods: a four-step strategy"

Abstract: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including unplanned treatment disruptions, participant infections and an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. This presentation explores a four-step strategy for handling unplanned disruptions in the analysis of randomised trials that are ongoing during a pandemic using missing data methods. The framework is consistent with the statistical principles outlined in the ICH-E9(R1) addendum on estimands and sensitivity analysis in clinical trials. Following an outline of the main issues raised by a pandemic we describe each point of the guidance in turn, which we illustrate using the ASCOT trial, an ophthalmic trial ongoing during Covid-19. Scenarios where treatment effects for a ‘pandemic free world’ and ‘world including a pandemic’ are of interest are considered. Open questions surrounding how to choose the most appropriate treatment estimand with respect to the pandemic, identifying affected data and the application of non-missing data methods to target these estimands will be discussed.

11:55-12:45  -  Break-out rooms discussions

Breakout room moderators: 

Toshimitsu Hamasaki (George Washington University), 

Frank Bretz  (Novartis)

12:45-13:00  -  Remarks and Next Steps

About the Speakers

David Murray is Associate Director for Disease Prevention and Director of the Office of Disease Prevention (ODP).  Over the past 40 years, Dr. Murray has worked on more than 50 health promotion and disease prevention research projects funded by the NIH and other agencies. He served on more than 40 grant review panels for the NIH and as the first Chair of the Community Level Health Promotion study section. He has published more than 250 articles in the peer-reviewed literature.  Dr. Murray has a passion for prevention research done well and believes that we can best advance the nation’s health by ensuring that prevention programs are based on good science, that they are carefully designed and evaluated, that effective interventions are disseminated, and that ineffective interventions are identified and discarded. This view is entirely consistent with the mission of the ODP, which is to work with the NIH Institutes and Centers and other partners to provide leadership and direction for the development, refinement, implementation, and coordination of a trans-NIH plan to increase the scope, quality, dissemination, and impact of NIH disease prevention and health promotion research.

Mouna Akacha is the Group Head of the Statistical Methodology group of Novartis Pharma AG, based in Basel, Switzerland. She and her team provide internal advice for clinical projects in all development phases and therapeutic areas. She is engaged in developing and implementing innovative statistical methods for clinical projects covering estimand discussions and approaches for missing data, longitudinal data, and recurrent event data. Before joining Novartis, Akacha studied mathematics at the University of Oldenburg in Germany. She holds a PhD in statistics from the University of Warwick in the United Kingdom.

Suzie Cro is a Research Fellow at Imperial Clinical Trials Unit (ICTU), School of Public Health. Within ICTU she is a member of the Trials Methodology team (StatsCI) and Clinical Trial Statistics group. She has a broad range of experience in the design and analysis of clinical trials and other interventional studies across clinical areas including ophthalmology, dermatology, musculoskeletal and in opiate addiction. Her statistical research focusses on relevant accessible methods for handling missing data and post-randomization events, such as rescue medication and non-compliance in randomised controlled trials. Suzie is a core group member of the MRC-NIHR TMRP Statistical Analysis. 

Event Type


National Institute of Statistical Sciences


National Institute of Statistical Sciences


Online Webinars and Workshop
Ingram Olkin Forum Series: Unplanned Clinical Trial Disruptions