ITSEW 2019 - International Total Survey Error Workshop

ITSEW 2019

The International Total Survey Error Workshop (ITSEW) was held on June 10-12  at the University of Bergamo, Italy brought together survey researchers, practitioners, and methodologists (especially in national statistics offices) to foster an exchange of ideas and preliminary research findings toward a better understanding of total survey error. This workshop has been held annually since 2005 and typically gathers 50-60 attendees.

The goals of the workshop include:

  • Reviewing progress on important TSE problems
  • Defining current data quality problems in detail, and articulating a research agenda to address them
  • Forming research collaborations to carry out needed research
  • Identifying emerging research needs at an early stage
  • Investigating the quality of alternative data sources.

The topic of the 2019 edition of ITSEW was "Integration of surveys and alternative data sources". Before the start of the workshop on Monday, June 10, there was a short primer on Total Survey Error. This primer reviewed the literature on Total Survey Error and a discussion of recommended best practices.

The 2019 Workshop Program (with links to slides)

MONDAY, 10 June 2019

Primer on TSE: Total Survey Error; Roots and Evolution -  Lars Lyberg, Stockholm University

SESSION 1: Innovations at National Statistical Institutes

Chair: Martin Beaulieu, Statistics Canada

Effects of Istat CDC (Centralised Data Collection) approach on the reduction of the Total Survey Error - Loredana De Gaetano and Pasquale Papa*, Italian National Statistical Institute.

Placing administrative data at the heart of the UK population statistics system; how the total survey error framework is helping to inform the use and design of integrated data solutions - Louisa Blackwell and Nicola Rogers*, Office for National Statistics.

Adaptive survey design at Statistics Netherlands  - Kees van Berkel*, Statistics Netherlands.

Total Error and variability measures with integrated disclosure limitation for the quarterly workforce indicators - Kevin L. McKinney(1)*, Andrew S. Green(2), Lars Vilhuber(3), and John M. Abowd(1) (1) U.S. Census Bureau, (2) OECD, (3) Cornell University.

SESSION 2: Integrating probability and non probability samples

Chair: Anders Wallgren, Formerly of the Department of Research and Development at Statistics Sweden

The AmeriSpeak® experience: methods of using a national probability sample panel for studies that combine probability and nonprobability samples - Vicki Pineau*, Ed Mulrow, Nada Ganesh, Michael Yang, and J. Michael Dennis NORC at the University of Chicago.

Data integration of national surveys and nonprobability samples: balancing enhanced analytic capacity within representational constraints - Steven B. Cohen*, RTI International.

Integrating different data sources for total estimation with unknown population size - Zhaoce Liu* and Lynne Stokes, Southern Methodist University.

TUESDAY, 11 June 2019

Keynote Address: Total Survey Error for longitudinal surveys - Peter Lynn, University of Essex

Chair: Paul Biemer, RTI International

SESSION 3: TSE in multi-sources and mixed-mode approaches

Chair: Silvia Biffignandi, University of Bergamo

Evaluating mode effects for an in-person panel survey that transitioned to a mixed web mode - Paul Biemer*, RTI International.

A new Total Error Framework for multi-source processes - Fabiana Rocci, Roberta Varriale*, and Orietta Luzi, Italian National Statistical Institute.

Building a Quality Indicators Framework in a multi-source environment - Martin Beaulieu*, Ryan Chepita, and Susie Fortier, Statistics Canada.

The Total Survey Error – A time series and system approach - Anders Wallgren* and Britt Wallgren, Formerly of the Department of Research and Development at Statistics Sweden

SESSION 4: Data Quality

Chair: Alan F. Karr, RTI International

Assessing interviewer effects in the BRFSS - Ting Yan(1), Antonia Warren(1), Carol Pierannunzi(2), Doug Williams(1), Jennifer Crafts(1), and Sonya Gamble(2),  (1) Westat, (2) Centers for Disease Control.

Keep the baby, throw out the bath water: The promises of real-time data quality evaluations Second generation at school - Steven Snell and Carol Haney, Qualtrics

Second generation at school - Daniela Cocchi and Francesco Giovinazzi, University of Bologna

SESSION 5: TSE evaluation

Chair: Oliver Lipps, FORS Lausanne

Considering digital privacy attitude measures in the context of TSE: Results from an international workshop - David L. Vannette(1)*, Frauke Kreuter(2), and Julia Lane(3); (1) Facebook, (2) University of Maryland & IAB, (3) New York University.

Social media data for social indicators: assessing the quality through case studies - Silvia Biffignandi(1), Annamaria Bianchi(1)*, and Camilla Salvatore(2); (1) University of Bergamo, (2) University of Milano Bicocca

Applications of R Shiny to explore, evaluate and improve Total Survey Quality - Xiaodan Lyu*, Heike Hofmann, Emily Berg, and Jie Li, Iowa State

WEDNESDAY, 12 June 2019

SESSION 6: Nonresponse modeling

Chair: Annamaria Bianchi, University of Bergamo

Predictive modeling as an alternative to (re-)weighting - Alan F. Karr*, RTI International

In an era of declining response rates, a note on the importance of estimation procedures - Martin Klein, Joanna Fane Lineback*, and Joseph L. Schafer, U.S. Census Bureau.

The effect of measurement error on clustering algorithms: a sensitivity analysis - Paulina Pankowska(1)*, Daniel Oberski(2), and Dimitris Pavlopoulos(1); (1) Vrije Universiteit Amsterdam, (2) Utrecht University.

SESSION 7: Undercoverage and nonresponse

Chair: David L. Vannette, Facebook

Undercoverage and nonresponse as sources of representativeness bias in nonprobability online panels. The Italian case - Chiara Respi* and Emanuela Sala, University of Milano Bicocca

“Can’t live with ‘em, can’t live without ‘em” Assumptions – the necessary evil - Abby Morgan*, Rebecca Green, and Patrick Graham, Statistics New Zealand

Utilising interviewer observations on housing unit characteristics in the Hungarian LFS nonresponse analysis: a research plan - Linda Mohay and Ferenc Mújdricza*, Hungarian Central Statistical Office

SESSION 8: Administrative data and errors

Chair: Roberta Varriale, Italian National Statistical Institute

Integrating administrative and survey agricultural data with Statistical Matching - Riccardo D’Alberto* and Meri Raggi, University of Bologna.

Minimizing sampling error: Preliminary findings of sampling design simulations using survey and administrative data - Gró Einarsdóttir* and Anton Örn Karlsson, Statistics Iceland.

Are screener questions measures of individual-level general attentiveness to surveys? Evidence from an Italian electoral panel study - Riccardo Ladini* and Cristiano Vezzoni, University of Milan.

SESSION 9: Nonresponse bias assessment

Chair: Vicki Pineau, NORC at the University of Chicago

Does adding a survey language pay off? A method for analysing the potential for reducing representation bias - Micheal Ochsner(1,2) and Oliver Lipps(1)*; (1) FORS Lausanne, (2) ETH Zurich.

Assessing nonresponse bias by permitting individuals to opt out of a survey - Taylor Lewis*, RTI International.

Strategies to minimize non-response bias - Nils Galberg Enoksen*, Bo Bilde, and Monika Klingsbjerg-Besrechel, Danmarks Statistik.

SESSION 10: Non-survey data sources and errors

Chair: Kevin L. McKinney, U.S. Census Bureau

Measuring statistical uncertainty in admin-based population estimates - Louisa Blackwell(1)*, Katy Stokes(1), and Peter W. Smith(2); (1) Office for National Statistics, (2) University of Southampton.

Estimates of accuracy of statistical registers based on Anticipated Variance - Giorgio Alleva(1), Piero Demetrio Falorsi(2), Francesca Petrarca(2)*, and Paolo Righi(2); (1) Sapienza University of Rome, (2) Italian National Statistical Institute.

Correcting survey measurement error using road sensor data - Jonas Klingwort(1,2)*, Bart Buelens(3), Joep Burger(1), and Rainer Schnell(2); (1) Statistics Netherlands, (2) University of Duisburg-Essen, (3) Flemish Institute for Technological Research.

Monday, June 24, 2019 by Glenn Johnson