Principles and Practices for a Federal Statistical Agency: Fourth Edition (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Principles and Practices for a Federal Statistical Agency F o u r t h e d i t i o n Committee on National Statistics Constance F. Citro, Margaret E. Martin, and Miron L. Straf, Editors Division of Behavioral and Social Sciences and Education 

THE NATIONAL ACADEMIES PRESS  500 Fifth Street, N.W.   Washington, D.C. 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Grant No. SBR-0453930 between the National Academy of Sciences and the National Science Foundation, which provides funding from a con- sortium of federal agencies. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project. Principles and practices for a federal statistical agency / Constance F. Citro, Margaret E. Martin, and Miron L. Straf, editors. ; Committee on National Statistics, Division of Behavioral and Social Sciences and Education, National Research Council. — 4th ed. p. cm. Includes bibliographical references. ISBN 978-0-309-12175-0 (pbk.) -- ISBN 978-0-309-12176-7 (pdf ) 1. United States—Statistical services. I. Citro, Constance F. (Constance Forbes), 1942- II. Martin, Margaret E. III. Straf, Miron L. IV. National Research Council (U.S.). Committee on National Statistics. V. National Research Council (U.S.). Division of Behavioral and Social Sciences and Education. HA37.U55P75 2009 352.7’50973—dc22 2009003377 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, D.C. 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. Copyright 2009 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Research Council. (2009). Principles and Practices for a Federal Statistical Agency, Fourth Edition. Committee on National Statistics. Constance F. Citro, Margaret E. Martin, and Miron L. Straf, Editors. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a man- date that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the Na- tional Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org 



COMMITTEE ON NATIONAL STATISTICS 2008-2009 WILLIAM F. EDDY (Chair), Department of Statistics, Carnegie Mellon University KATHARINE ABRAHAM, Department of Economics and Joint Program in Survey Methodology, University of Maryland ALICIA CARRIQUIRY, Department of Statistics, Iowa State University WILLIAM DuMOUCHEL, Phase Forward, Inc., Waltham, MA JOHN HALTIWANGER, Department of Economics, University of Maryland V. JOSEPH HOTZ, Department of Economics, Duke University KAREN KAFADAR, Department of Statistics, Indiana University DOUGLAS MASSEY, Department of Sociology, Princeton University SALLY MORTON, Statistics and Epidemiology, Research Triangle Institute, Research Triangle Park, NC JOSEPH NEWHOUSE, Division of Health Policy Research and Education, Harvard University SAMUEL H. PRESTON, Population Studies Center, University of Pennsylvania HAL STERN, Department of Statistics, University of California, Irvine ROGER TOURANGEAU, Joint Program in Survey Methodology, University of Maryland ALAN ZASLAVSKY, Department of Health Care Policy, Harvard Medical School CONSTANCE F. CITRO, Director, Committee on National Statistics  



Acknowledgments The Committee on National Statistics thanks the many people who contributed their time and expertise to the preparation of this report. We are most appreciative of their cooperation and assistance. In expressing our gratitude to the staff, a special measure of recognition is due to Margaret Martin and Miron Straf, directors of the Committee on National Statistics in 1972-1978 and 1987-1999, respectively, who were coeditors of the first edition of this report. In preparing subsequent editions, they were joined as editors by Constance Citro. This edition benefited from the editing of Eugenia Grohman of the Division of Behavioral and Social Sciences and Education and from editing and graphic assistance by Daniel Cork of the committee staff. We also are indebted to many others who of- fered valuable comments and suggestions, too numerous to mention. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with proce- dures approved by the Report Review Committee of the National Research Council (NRC). The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confiden- tial to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Alfred Blumstein, H. John Heinz III College, Carnegie Mellon University; Daniel Kasprzyk, vii 

viii ACKNOWLEDGMENTS Surveys and Information Services Division, Mathematica Policy Research, Inc., Washington, DC; Janet Norwood, consultant, Chevy Chase, MD; Fritz Scheuren, Vice President’s Office, National Opinion Research Center, The University of Chicago; and John H. Thompson, President’s Office, National Opinion Research Center, The University of Chicago. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by John C. Bailar, III, Department of Health Studies (emeritus), The University of Chicago. Ap- pointed by the NRC’s Report Review Committee, he was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review com- ments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. Finally, we recognize the many federal agencies that support the Com- mittee on National Statistics directly and through a grant from the National Science Foundation. Without their support and their commitment to im- proving the national statistical system, the committee work that is the basis of this report would not have been possible. William F. Eddy, Chair Committee on National Statistics, 2009 

Contents Preface xi Part I Principles and Practices for a Federal Statistical Agency 1 Part II Commentary 14 References 55 Appendixes A Organization of the Federal Statistical System 65 B Legislation and Regulations That Govern Federal Statistics 91 C Fundamental Principles of Official Statistics of the Statistical Commission of the United Nations 107 D Selected Federal Statistical Websites, January 2009 110 E Prefaces to the First, Second, and Third Editions 118 ix 



Preface The Committee on National Statistics (CNSTAT) has, since 1992, produced a report on principles and practices for a federal statistical agency, which draws from CNSTAT’s many reports on specific agencies, programs, and topics. This report has been widely cited and used by Con- gress and federal agencies; it has helped shape legislation and executive actions to establish and evaluate statistical agencies; the U.S. Office of Management and Budget has cited it in regulatory language; and the U.S. Government Accountability Office has used it as a benchmark in reports to Congress. Statistical agencies have used it to inform new appointees, advisory committees, and others about what constitutes an effective and credible statistical organization. Beginning with the second edition in 2001, CNSTAT committed to updating the document every 4 years on a schedule to provide a current edition to newly appointed cabinet sec- retaries and other federal personnel at the beginning of each presidential administration (or second term). This fourth edition presents and comments on four basic principles that statistical agencies must embody in order to carry out their mission fully: (1) They must produce data that are relevant to policy issues, (2) they must achieve and maintain credibility among data users, (3) they must achieve and maintain trust among data providers, and (4) they must achieve and maintain a strong position of independence from the appearance and reality of political control. The paper also discusses 11 important practices that are means for statistical agencies to live up to the four principles. These xi 

xii pREFACE practices include a commitment to quality and professional practice and an active program of methodological and substantive research. The first three principles and 10 of the 11 practices have appeared in each prior edition; this fourth edition adds the principle that statistical agencies must operate from a strong position of independence and the practice that agencies must have ongoing internal and external evaluations of their programs. The fourth edition retains the basic structure of previous editions in that Part I presents the principles and practices in summary form, and Part II, Commentary, further explains, defines, and illustrates the topics in Part I. The fourth edition includes new appendix material to orient the reader: Appendix A provides an overview of the organization of the U.S. federal statistical system, which is the most decentralized in the developed world, and compares the size of the system to the size of the federal government as a whole. Appendix B summarizes key legislation and regulations that affect federal statistical agencies, such as the Confidential Information Protection and Statistical Efficiency Act of 2002. Appendix C reproduces the Fun- damental Principles of Official Statistics of the United Nations Statistical Commission. Appendix D provides addresses of Internet sites for major federal agencies that provide statistical data, and Appendix E reproduces the prefaces to the first, second, and third editions of the CNSTAT report. We are sometimes asked what distinguishes a “principle” from a “prac- tice.” Although the distinction is not hard and fast, we deem “principles” to be fundamental and intrinsic to the concept of a federal statistical agency. Without policy relevance, credibility with data users, trust of data provid- ers, and a strong position of independence, an agency cannot provide the benefits to policy makers and the public in a democratic society that are the rationale for establishing a statistical agency. We deem “practices” to be ways and means of making the basic principles operational and facilitating an agency’s adherence to the basic principles. Although focused on federal statistical agencies, many of the principles and practices articulated here likely also apply to statistical activities else- where, such as in federal policy, evaluation, research, and program agencies, in state and local government agencies, and in other countries. Finally, the principles and practices in this report remain guidelines, not prescriptions. We intend them to assist statistical agencies and to inform policy makers, data users, and others about the characteristics of statistical agencies that enable them to serve the public good. William F. Eddy, Chair Committee on National Statistics, 2009