ASA Releases Task Force Report on 2020 Census Analysis and Replicability Report

The 2020 Census Quality Indicators Task Force, composed of census experts working under the auspices of the American Statistical Association (ASA), released a report today on the Census Bureau’s 2020 population totals for the 50 states and District of Columbia.

The task force evaluated a set of 10 state-level ‘process statistics’ relevant for evaluating the quality of census numbers for apportionment and concluded that, despite concerns about political interference, it found “no evidence of anything other than an independent and professional enumeration process” and “no major anomalies that would indicate census numbers are not fit for use for purposes of apportionment.” The task force also concluded that the Census Bureau’s decision to delay release of data products to ensure careful review and processing of the data according to Census Bureau quality standards was appropriate.

Additionally, the task force reported its ability to evaluate the quality of the 2020 Census was hampered by limits on available information and research. It concluded that the “indicators that have been released to date by the bureau do not permit a thorough assessment of the 2020 Census data quality.”

Nancy Potok, chair of the task force and former US chief statistician, praised the Census Bureau for providing external researchers broad access to 2020 Census data. “The bureau’s receptiveness to outside, early review and their subsequent cooperation and support during an extraordinarily busy time demonstrate their commitment to transparency and quality. The task force thanks the staff who supported its outside review and commends the bureau staff for their devoted work to make the 2020 Census as fair and accurate as possible during a very difficult period. The task force is also deeply grateful to our three researchers who gave of their time so generously over an extended period to gather and review the data provided by the Census Bureau,” Potok said.

The task force was formed in September 2020 in response to concerns about the quality of the 2020 Census data in light of the data collection challenges throughout the year. It was initially co-chaired by Potok and ASA 2021 President Robert Santos. (Santos stepped down from the committee in April after President Joe Biden announced his intention to nominate Santos as Census Bureau director.)

The task force released a preliminary report in October 2020, which presented several indicators of census quality, such as those from demographic analysis and the Post-Enumeration Survey, and indicators of census operations that could provide an early look at the potential for error in the 2020 Census. That report recommended quality indicators be specified and analyzed by a group of non-Census Bureau statisticians.

The use of quality indicators was suggested in August 2020 in a public statement to Congress by four former Census Bureau directors. The authors noted the Census Bureau would face an “enormous burden” of determining whether its 2020 Census results are adequate. They requested that Congress ask an independent institution to establish metrics to assess whether the 2020 Census data match other population estimates produced by the Census Bureau. In September, two former US chief statisticians wrote an op-ed for The Washington Post that supported this idea. Since Congress was unlikely to be able to take quick action, the ASA worked with the op-ed authors to form the task force soon thereafter.

In addition to its five conclusions, the task force made two recommendations in its report: the Census Bureau and National Academy of Sciences Committee on National Statistics should examine additional process statistics and patterns at more detailed levels of geography and planning for the 2030 Census should include explicit attention to evaluating and reporting on data quality in a timely manner. The task force also noted it did not have access to data at levels lower than states or data with characteristics of household occupants, both important for a thorough assessment of overall quality, especially with regard to data used in redistricting or for distribution of federal funds.

The members of the task force:

  • Nancy A. Potok (co-chair), Chief Statistician of the United States (2017–2020); Former Deputy Director, Census Bureau
  • Robert Santos (co-chair, resigned in April after being nominated as the next census director), Vice President and Chief Methodologist, Urban Institute; 2021 President, American Statistical Association
  • Constance F. Citro, Senior Scholar, National Research Council, Committee on National Statistics
  • Robert E. Fay, Senior Scientist, Westat; Former Senior Mathematical Statistician, Census Bureau
  • Robert Groves, Provost, Georgetown University; Former Director, Census Bureau
  • Howard Hogan, Adjunct Statistics Professor, The George Washington University; Former Chief Demographer and Chief of the Decennial Statistical Studies Division, Census Bureau
  • D. Sunshine Hillygus, Political Science Professor, Duke University; Former member, Census Scientific Advisory Committee
  • Thomas A. Louis, Biostatistics Professor Emeritus, Johns Hopkins Bloomberg School of Public Health; Former Chief Scientist, Census Bureau
  • Kenneth Prewitt (resigned in April upon becoming senior adviser in the office of the director of the US Census Bureau), Carnegie Professor of Public Affairs, Columbia University; Former Director, Census Bureau
  • Denice Ross, Fellow, Beeck Center for Social Impact + Innovation, Georgetown University
  • Matthew Snipp, Burnett C. and Mildred Finley Wohlford Professor of Humanities and Sciences, Department of Sociology, Stanford University; Director, Institute for Research in the Social Sciences
  • John H. Thompson, Distinguished Institute Fellow, Biocomplexity Institute and Initiative, University of Virginia; Former Director, Census Bureau
  • Katherine K. Wallman, Chief Statistician of the United States (1992–2017); 1992 President, American Statistical Association

The three researchers selected by the task force who carried out analyses:

  • Paul Biemer, Distinguished Fellow, Statistics, RTI International
  • Joseph Salvo, Senior Adviser, National Conference on Citizenship
  • Jonathan Auerbach, Assistant Professor of Statistics, George Mason University; former ASA Science Policy Fellow (added to the team in June)

Relevant files:
2020 Census State Population Totals: A Report from the American Statistical Association Task Force on 2020 Census Quality Indicators 

The Quality of the 2020 Census: An Independent Assessment of Census Bureau Activities Critical to Data Quality
by Paul Biemer, Joseph Salvo, and Jonathan Auerbach 

Review of Research Supporting the Use of Administrative Record Enumeration in the 2020 Census
by Robert E. Fay 

A Note on Demographic Benchmarks
by Howard Hogan 

2020 Census Quality Indicators: A Report from the American Statistical Association
prepared by the 2020 Census Quality Indicators Task Force 

The task force will hold a public webinar on Tuesday, September 14, 3:00 pm EDT. View the recording.