International Total Survey Error Workshop (ITSEW) 2021

September 24, October 1, and October 8, 2021

Total Survey Error in the Age of COVID-19

After a hiatus in 2020, ITSEW returns this year as a series of three virtual weekly seminars on the theme Total Survey Error in the Age of COVID-19.  As with earlier workshops, the goal is to promote discussion of questions of research, methodology and practice relating to Total Survey Error (TSE) and Total Survey Quality. Each session will be comprised of relatively short presentations and catalyzing discussion by a designated discussant, followed by floor discussion among all participants.

The dates, topics and speakers are:

Friday, September 24, 2021 (9 am – 12:00 Noon ET)

TSE for Data Collections about COVID


Jason Fields, (US Census Bureau);
Carina Cornesse, (University of Mannheim);
Andrew Phelps, (UK Office for National Statistics);
Jamie C. Moore, (University of Essex - UK)

Discussant: Paul J. Lavrakas, (Private consultant)

Friday, October 1, 2021 (9 am – 12:00 Noon ET)

Effects of COVID on Other Data Collection


Martina Helme and Salah Merad, (UK Office for National Statistics);
Jennifer Truman, (US Bureau of Justice Statistics);
Nicky Rogers and Duncan Elliott, (UK Office for National Statistics);
Jan van den Brakel, (Statistics Netherlands)

Discussant: John Eltinge, (US Census Bureau)

Friday, October 8, 2021 (9 am – 12:00 Noon ET)

Total Quality for Censuses


J. David Brown, (US Census Bureau);
Owen Abbott, (UK Office for National Statistics);
Stefano Falorsi, (Italian National Institute of Statistics);
Christiane Laperrière, (Statistics Canada)

Discussant: Nancy Potok (NAPx Consulting)

The session on October 8 will conclude with Remembrances of Lars Lyberg, led by Paul Biemer, (RTI International).

Titles, abstracts and, ultimately, the talks themselves, will be available on the ITSEW 2021 web site in due course.


Registration fees are $25 US for students and postdoctorals and $50 US for all others.

For students and postdoctorals who participate, the registration fee will be refunded following the third session.

The registration deadline is September 10, 2021Registration coming soon!

Registrants will receive links for the events by September 17, 2021.

Organizing Committee

Ruben Bach (University of Manheim);
Marcus Berzofsky, co-chair (RTI International);
Annamaria Bianchi (University of Bergamo);
Paul Biemer (RTI International);
Silvia Biffignandi (University of Bergamo);
Alan F. Karr, co-chair (AFK Analytics, LLC);
Olga Maslovskaya (University of Southampton);
James Rosenberger (NISS);
Natalie Shlomo (University of Manchester) 

Financial Support

The Organizing Committee thanks RTI International and Westat for their generous, ongoing financial support of ITSEW.


Friday, September 24, 2021 (9 am – 12:00 Noon ET)

TSE for Data Collections about COVID

Friday, October 1, 2021 (9 am – 12:00 Noon ET)

Effects of COVID on Other Data Collection

Friday, October 8, 2021 (9 am – 12:00 Noon ET)

Total Quality for Censuses


Population census data are an essential component of national statistics because they are used for forming policies, distributing funds, calibrating surveys and a many other important uses. Errors in the census, if uncontrolled, can defeat these purposes and may even destroy public trust in official statistics. Regardless of the methodology to ensure the highest quality, the census is never perfect and there will always be errors that affect quality. In 2020, there were additional challenges from COVID19 that may have further increased the error risks. In addition, there is  increasing use of administrative data to mitigate errors  or even replace  census data and this leads to other types of quality concerns. The focus of this session is the evaluation of census errors to understand how errors may affect census fitness for use.

Introductory Comments: Session Chair:    TBA

Speaker 1  -  J. David Brown, U.S. Census Bureau

Title: Using Administrative Records to Evaluate the Quality of the 2020 U.S. Census   
Author:  J. David Brown, U.S. Census Bureau

Abstract: We examine the quality of the 2020 U.S. Census through comparison to administrative data. We document the extent and characteristics of persons apparently counted multiple times, not alive, and not residents on the reference date. Persons in the administrative data, but not the 2020 Census, are studied to understand the magnitude and characteristics of potential omissions. We show how discrepancies between the sources vary by survey response mode.   

Speaker 2 - Owen Abbott, Office for National Statistics, UK     

Title:  Busting the Census Count Myth
Authors:  Owen Abbott, Jon Wroth-Smith and Cal Ghee, Office for National Statistics, UK.
Abstract:  The 2021 Census in England and Wales, like its predecessors in 2011 and 2001, does not produce a count of the population. All of the outputs are estimates. The estimates are derived from a series of statistical methods designed to mitigate against any errors that result from the way in which the population respond to the census collection process. This includes item imputation to deal with non-response to individual questions, identification and removal of duplicates, measurement of under-coverage using a Census Coverage Survey and imputation of households (and persons) to correct for missed households (and persons). The principle is to reduce the bias due to errors, turning the collected data into approximately unbiased statistical estimates which are subject to confidence intervals. This presentation will outline the series of methods used to produce the 2021 Census estimates and the additional challenges faced due to the COVID-19 pandemic. Administrative data is playing a larger role in 2021 than in previous censuses, and we will outline where this is being used. Lastly, to educate census users in the use of the estimates, we will describe efforts to measure the variability introduced by the statistical processes and how these will be disseminated.

Speaker 3 - Stefano Falorsi, Italian National Institute of Statistics Speaker  

Title:    The Italian Experience for the Population Census in the Year of COVID  
Authors: Stefano Falorsi, Danila Filipponi, Silvia Loriga and Marco Di Zio.

Abstract:  The paper will talk about the problem of the census estimation for the2020 Census round, in which the census master sample has not been carried out due to Covid19 outbreak. In particular, the contribution will describe how the census estimation will be done using administrative register information  and data coming from 2018 and 2019 surveys. A reduced set of hypercubes will be produced for 2020 referring to variables arising mainly from administrative information: Demographic figures + level of education. Furthermore a particular focus will be devoted to the estimation of coverage rates of the Population Register exploiting administrative ‘signals of life’ more extensively than previous years. 

Speaker 4 - Christiane Laperrière - Statistics Canada

Title:  Quality-driven collection operations during the 2021 Canadian Census of Population
Author:  Christiane Laperrière – Chief Methodologist of Census Operations, Statistics Canada

Abstract: The Canadian Census of Population involves a large-scale collection operation which covers a vast territory with different regional characteristics that calls for various collection methodologies. Many efforts were undertaken to ensure the success of the Census operations amid the COVID 19 pandemic – self-response was strongly encouraged (in particular via the electronic questionnaire) and adjustments were made to the follow-up operations to minimise in-person contacts to protect the health and safety of Canadians. Furthermore, the use of administrative data was explored in the context of a statistical contingency. This presentation will highlight the strategies that were implemented to reduce errors and to ensure high quality data for the 2021 Census. Some of these operations include telephone follow-ups to resolve ambiguities related to coverage, when respondents were uncertain about who to include on their questionnaire, and field visits to verify the occupancy status of a sample of dwellings. Field enumeration also took place, when self-response could not be obtained. During these non-response follow-up activities, quality indicators were monitored to measure progress and identify errors (such as dwelling occupancy classification errors) while there were still field enumerators available to make verifications and corrections. This monitoring also ensured that high response rates were achieved uniformly across the country. In this presentation, we will introduce these quality-driven operations and provide preliminary results based on the recent Canadian Census experience.

Discussant: Nancy Potok

Floor Discussion: 30 minutes

Concluding Session: Remembrances of Lars Lyberg, led by Paul Biemer

Event Type


RTI International


$25 US for students and postdoctorals and $50 US for all others


Virtual Conference
The session on October 8 will conclude with Remembrances of Lars Lyberg (1944-2021), led by Paul Biemer, (RTI International).