National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

Health Performance Measurement in the Public Sector

Principles and Policies for Implementing an Information Network

Edward B. Perrin, Jane S. Durch, and Susan M. Skillman, Editors

Panel on Performance Measures and Data for Public Health Performance Partnership Grants

Committee on National Statistics

Commission on Behavioral and Social Sciences and Education

National Research Council

NATIONAL ACADEMY PRESS
Washington, D.C.
1999

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

NATIONAL ACADEMY PRESS
2101 Constitution Avenue, N.W. Washington, D.C. 20418

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by Contract No. 282-95-0034 between the National Academy of Sciences and the U.S. Department of Health and Human Services. Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (No. SBR-9709489). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.

Library of Congress Cataloging-in-Publication Data

Health performance measurement in the public sector : principles and policies for implementing an information network / Edward B. Perrin, Jane S. Durch, and Susan M. Skillman, editors ; Panel on Performance Measures and Data for Public Health Performance Partnership Grants, Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council. 

p. cm.

Includes bibliographical references (p. ). 

ISBN 0-309-06436-8 (pbk.) 

1. Public health—United States—Information services. 2. Public health—United States—Evaluation—Data processing. I. Perrin, Edward. II. Durch, Jane. III. Skillman, Susan M. IV. National Research Council (U.S.). Panel on Performance Measures and Data for Public Health Performance Partnership Grants. 

RA423.2 .H43 1999

362.1'0973—dc21 99-6159

Additional copies of this report are available from
National Academy Press,
2101 Constitution Avenue, N.W., Box 285, Washington, D.C. 20055. Call (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area). This report is also available on line at http://www.nap.edu

Printed in the United States of America

Copyright 1999 by the National Academy of Sciences. All rights reserved.

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

PANEL ON PERFORMANCE MEASURES AND DATA FOR PUBLIC HEALTH PERFORMANCE PARTNERSHIP GRANTS

EDWARD B. PERRIN (Chair),

Department of Health Services, School of Public Health and Community Medicine, University of Washington

JOHN C. BAILAR III,

Department of Health Studies, University of Chicago

THOMAS J. BARRETT,

Mental Health Services, Colorado Department of Human Services

DAVID W. FLEMING,

Center for Disease Prevention and Epidemiology, Oregon Health Division

V. JOSEPH HOTZ,

Departments of Economics and Policy Studies, University of California, Los Angeles

IRA R. KAUFMAN,

University of Medicine and Dentistry of New Jersey, Department of Environmental and Community Medicine, Robert Wood Johnson Medical School

JOHN R. LUMPKIN,

Illinois Department of Public Health

WILLIAM A. MORRILL,

Mathtech, Inc., Princeton, New Jersey

R. HEATHER PALMER,

Center for Quality of Care Research and Education, Harvard School of Public Health

J. SANFORD SCHWARTZ,

School of Medicine and the Wharton School, University of Pennsylvania

MARY E. STUART,

Maryland Collaboratory for Business and Health, Department of Sociology and Anthropology, University of Maryland, Baltimore County

CYNTHIA P. TURNURE,

Minnesota Department of Health

PAUL J. WIESNER,

DeKalb County Board of Health, Decatur, Georgia

GAIL R. WILENSKY,

Project Hope, Bethesda, Maryland

JANE S. DURCH, Study Director

JEFFREY J. KOSHEL, Study Director (through December 1997)

SUSAN M. SKILLMAN, Senior Research Associate

TELISSIA M. THOMPSON, Senior Project Assistant

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

COMMITTEE ON NATIONAL STATISTICS 1998–1999

JOHN E. ROLPH (Chair),

Department of Information and Operations Management, University of Southern California

JOSEPH G. ALTONJI,

Institute for Research on Poverty and Department of Economics, Northwestern University

JULIE DaVANZO, RAND,

Santa Monica, California

WILLIAM F. EDDY,

Department of Statistics, Carnegie Mellon University

WILLIAM KALSBEEK,

Department of Biostatistics, University of North Carolina

RODERICK J.A. LITTLE,

School of Public Health, University of Michigan

THOMAS A. LOUIS,

School of Public Health, University of Minnesota

CHARLES F. MANSKI,

Department of Economics, Northwestern University

WILLIAM NORDHAUS,

Department of Economics, Yale University

JANET L. NORWOOD,

Urban Institute, Washington, D.C.

EDWARD B. PERRIN,

Department of Health Services, School of Public Health and Community Medicine, University of Washington

PAUL R. ROSENBAUM,

Department of Statistics, Wharton School, University of Pennsylvania

FRANCISCO J. SAMANIEGO,

Division of Statistics, University of California, Davis

RICHARD L. SCHMALENSEE,

Sloan School of Management, Massachusetts Institute of Technology

MIRON L. STRAF, Director

ANDREW A. WHITE, Deputy Director

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

2

 

Putting Performance Measurement in Context

 

30

   

Use of Performance Measurement in Accountability Systems

 

30

   

Performance-Based Accountability Systems

 

31

   

Operation of Performance-Based Systems

 

33

   

Applications of Performance Measurement

 

37

   

Examples of Performance Monitoring and Accountability Systems

 

40

   

Government Performance and Results Act

 

41

   

Use of Performance Measures in the Maternal and Child Health Services Block Grant Program

 

42

   

Reporting Requirements for the Temporary Assistance for Needy Families Program

 

44

   

State Developments in Performance-Based Budgeting

 

45

   

Health Care Performance Measurement in the Private Sector

 

48

   

Conclusions

 

52

   

References

 

55

3

 

Performance Measurement Considerations for Publicly Funded Health Programs

 

59

   

Broad Array of Health-Related Services and Service Relationships

 

60

   

Measurement Considerations for Population-Based Health Services

 

61

   

Population Health Services

 

61

   

Monitoring Population Health Services

 

62

   

Monitoring the Infrastructure for Publicly Funded Health Programs

 

64

   

Some Performance Measurement Considerations Related to Program-Specific Matters

 

66

   

Environmental Health Programs

 

66

   

Mental Health Programs

 

68

   

Substance Abuse Programs

 

71

   

Process Guidelines as a Basis for Performance Measurement

 

74

   

Guidelines for Personal Health Services

 

75

   

Guidelines for Population-Based Health Services

 

76

   

Research Needs for Practice Guidelines and Performance Measurement

 

76

   

Conclusions

 

78

   

References

 

79

4

 

Data and Information Systems: Issues for Performance Measurement

 

83

   

Health Data Resources

 

84

   

Registries

 

85

   

Surveys

 

86

   

Patient Records and Related Clinical Encounter Data

 

87

   

Administrative Data

 

89

   

Claims Data

 

90

   

Linkage of Data Sets

 

91

   

Steps Toward Integration of Data Sets

 

93

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
   

Assuring the Quality of Data and Data Analysis

 

95

   

Random Variation and Bias

 

96

   

Data Management

 

97

   

Challenges in Data Analysis

 

98

   

Developing and Implementing Standards for Data and Data Systems

 

103

   

Standardization Activities

 

103

   

Balancing Standardization and Change

 

108

   

Enhancing Performance Measurement through Advances in Information Technology

 

109

   

Data Collection and Transmission Technologies

 

109

   

Data Management and Analysis

 

110

   

Computer-Based Patient Records

 

111

   

The Internet and the World Wide Web

 

111

   

Limits of Technology

 

114

   

Privacy, Confidentiality, and Security of Health Data

 

114

   

Investing in Health Data and Data Systems

 

116

   

Data and Information Systems

 

117

   

Technology

 

119

   

Training and Technical Assistance

 

120

   

Taking a Collaborative Approach to the Development of Health Data and Information Systems

 

122

   

Meeting the Needs of Many Data Users

 

123

   

Collaboration in the Design and Implementation of Data Systems

 

125

   

Conclusions

 

126

   

References

 

127

5

 

Strategies for Supporting Performance Measurement Through a National Information Network

 

132

   

Conclusions Regarding Performance Partnership Agreements

 

133

   

A National Information Network for Health-Related Data

 

134

   

Essential Features of a Health Information Network to Serve Different Levels of Decision Making

 

135

   

Major Policy Actions Needed

 

138

   

National Collaboration

 

138

   

Integration of Data Systems

 

143

   

Technology Policy

 

144

   

Operational Principles

 

145

   

Involve a Broad Range of Public Agencies

 

145

   

Build on Existing Information Systems

 

146

   

Implementation Standards

 

147

   

Required Investment for an Information Network

 

152

   

Data Systems

 

152

   

Training and Technical Assistance

 

154

   

A National Research Agenda

 

156

   

Final Observations

 

157

   

References

 

158

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

Preface

This is the second of two reports from the Panel on Performance Measures and Data for Public Health Performance Partnership Grants. In summer 1995, as part of its response to the need to ensure that public funding of health programs is related to documented program performance, the Office of the Assistant Secretary for Health of the U.S. Department of Health and Human Services (DHHS) requested that the National Research Council (NRC) convene an expert panel to examine and report on the technical issues involved in establishing performance measures in ten substantive program areas. Such measures were to be required as part of the proposed Performance Partnership Grants (PPG) Program, under which each state was to negotiate with DHHS an action plan with performance objectives that would be specific in terms of outcomes, processes, and capacity to be achieved within a 3- to 5-year period.

The panel divided its work into two phases. The objective of the first phase was to develop a report to the Secretary of DHHS on performance measures in specified areas that would be useful to the PPG Program in its first 3–5 years. The report Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health (National Academy Press, Washington, D.C., 1997) presents the findings and recommendations resulting from the first phase of the panel's work. The focus of the second phase was on recommended improvements to existing local, state, and federal data systems that would facilitate the collection of data for use with existing and proposed performance measures. This two-phase approach was adopted by the panel because of our conclusion that developmental work is needed in both the public and private sectors to adapt, refine, or add to existing data systems to make them more useful in performance measure-

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

ment, and to delineate more clearly the complex causal links between program processes and outcomes.

Although formal legislation mandating PPGs has not been adopted, various DHHS programs and a growing number of states and communities are moving to monitor and analyze health outcomes on their own. Moreover, there appears to be a growing consensus within the public health, substance abuse, and mental health communities about the value of performance measurement. Indeed, many people believe that the case for increasing, or even maintaining, public funding will depend on documented program performance. The development of performance measures is a continuation of earlier efforts to assess progress toward important public health goals. The broad acceptance of immunization rates and other such measures developed for Healthy People 2000, for example, has been instrumental in the creation of databases and the mobilization of resources in many jurisdictions to assess progress toward those objectives.

The work of this panel was furthered significantly by four regional meetings of state officials and consumers convened by DHHS early in the process—in Portland, Oregon; San Francisco, California; Chicago, Illinois; and Philadelphia, Pennsylvania—and by input from several national associations of state agency administrators (the Association of State and Territorial Health Officials, the National Association of County and City Health Officials, the National Association of State Alcohol and Drug Abuse Directors, and the National Association of State Mental Health Program Directors). Our work during the second phase of the project was aided by the contributions of participants in a workshop held by the panel in July 1997. The workshop agenda and participants are listed in Appendix B. We also wish to thank Winnie Mitchell of the Substance Abuse and Mental Health Services Administration and David Moriarty of the Centers for Disease Control and Prevention, who provided special assistance by arranging several meetings with agency staff members.

The panel was fortunate to have for its deliberations two specially commissioned working papers. ''Improving Federal-State Data Collection to Monitor Program Performance" was prepared by the Science and Epidemiology Committee of the Association of State and Territorial Chronic Disease Program Directors and the Council of State and Territorial Epidemiologists. Christopher Maylahn, New York State Department of Health, chaired the group that prepared this paper. "Quality Education Data: Unprecedented Opportunity for a Decade to Build" was prepared by Emerson Elliott, National Council for Accreditation of Teacher Education, and John Ralph, National Center for Education Statistics. Copies of these papers are available from the Committee on National Statistics (2101 Constitution Avenue, N.W., Washington, DC 20418).

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC's Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institu-

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

tion in making the published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their participation in the review of this report: Thomas F. Babor, School of Medicine, University of Connecticut Health Center; Maureen M. Henderson, School of Medicine, University of Washington (emeritus); Ed Lichtenstein, Oregon Research Institute, Eugene; Thomas Louis, School of Public Health, University of Minnesota; J. Michael McGinnis, scholar in residence, National Research Council; Barbara J. McNeil, Department of Health Care Policy, Harvard Medical School; Donald M. Steinwachs, School of Hygiene and Public Health, Johns Hopkins University; Bernard Turnock, Community Health Sciences, University of Illinois, Chicago; and Albert W. Wu, Health Services Research Center, Johns Hopkins University.

While the individuals listed above have provided constructive comments and suggestions, it must be emphasized that responsibility for the final content of this report rests entirely with the authoring committee and the institution.

The panel appreciates the assistance of the staff of the Committee on National Statistics (CNSTAT) in preparing this report. We wish to thank in particular Miron Straf, staff director of CNSTAT, for developing the original project design; Telissia Thompson for organizing panel meetings and carrying out innumerable administrative tasks; and Kathleen Saslaw for providing financial management for the study. Several former CNSTAT staff members provided valuable assistance at various stages in the production of the panel's two reports: Anu Das, Theresa Raphael, Michelle Ruddick, and Ashley Bowers. We appreciate the editorial work of Rona Briere and the guidance of Eugenia Grohman, associate director for reports of the Commission on Behavioral and Social Sciences and Education. Thanks also go to Sue Skillman, University of Washington, for her valued assistance to the committee chair. Finally, the panel extends its special thanks to Jeff Koshel, the panel's study director for the first part of the study, and to Jane Durch, for her very capable management of and contribution to the process leading to the production of this second report.

EDWARD B. PERRIN, CHAIR

PANEL ON PERFORMANCE MEASURES AND DATA FOR PUBLIC HEALTH PERFORMANCE PARTNERSHIP GRANTS

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. William A. Wulf is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council.

Page xiii Cite
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×

Health Performance Measurement in the Public Sector

Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
This page in the original is blank.
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R13
Suggested Citation:"Front Matter." National Research Council. 1999. Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network. Washington, DC: The National Academies Press. doi: 10.17226/6487.
×
Page R14
Next: Executive Summary »
Health Performance Measurement in the Public Sector: Principles and Policies for Implementing an Information Network Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

There is growing interest in using performance measurement as a means of holding federal, state, and local health agencies accountable for their use of public funds. Health Performance Measurement in the Public Sector is the second of two books for the U.S. Department of Health and Human Services on using and improving performance measurement in publicly funded health programs and the implications for data needs and systems.

This book focuses on data and information system issues at the federal, state, and local levels. Recommendations address:

  • Policy framework for selecting performance measures and using performance measurement.
  • Operational principles related to data and data systems that support performance measurement.
  • Essential investments in data systems and in training and technical assistance.
  • Research needed to improve performance measures and performance measurement.
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!