Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Evaluating AIDS
Prevention Programs
Expanded Edition
Susan L. Coyle, Robert F. Boruch, and
Charles F. Turner, Editors
PANEL ON THE EVALUATION OF AIDS INTERVENTIONS
COMMITTEE ON AIDS RESEARCH
AND THE
BEHAVIORAL, SOCIAL, AND STATISTICAL SCIENCES
COMMISSION ON THE BEHAVIORAL AND
SOCIAL SCIENCES AND EDUCATION
NATIONAL RESEARCH COUNCIL
NATIONAL ACADEMY PRESS
WASHINGTON, D.C. 1991
OCR for page R2
NATIONAL ACADEMY PRESS · 2101 Constitution Avenue, N.W. · Washington, D.C. · 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board
of the National Research Council, whose members are drawn from the councils of the National
Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The
members of the committee responsible for the report were chosen for their special competences
and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures
approved by a Report Review Committee consisting of members of the National Academy of
Sciences, the National Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of
distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance
of science and technology and to their use for the general welfare. Upon the authority of the
charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to
advise the federal government on scientific and technical matters. Dr. Frank Press is president
of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of
the National Academy of Sciences, as a parallel organization of outstanding engineers. It is
autonomous in its administration and in the selection of its members, sharing with the National
Academy of Sciences the responsibility for advising the federal government. The National
Academy of Engineering also sponsors engineering programs aimed at meeting national needs,
encourages education and research, and recognizes the superior achievements of engineers. Dr.
Robert M. White is president of the National Academy of Engineering.
The Lnstitute of Medicine was established in 1970 by the National Academy of Sciences
to secure the services of eminent members of appropriate professions in the examination of
policy matters pertaining to the health of the public. The Institute acts under the responsibility
given to the National Academy of Sciences by its congressional charter to be an adviser to the
federal government and upon its own initiative, to identify issues of medical care, research, and
education. Dr. Samuel O. Thier is president of the Institute of Medicine.
The National Research Council was established by the National Academy of Sciences in
1916 to associate the broad community of science and technology with the Academy's purposes
of furthering knowledge and of advising the federal government. Functioning in accordance with
general policies determined by He Academy, the Council has become the principal operating
agency of bow the National Academy of Sciences and the National Academy of Engineenng in
providing services to the govemment, the public, and the scientific and engineering communities.
The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank
Press and Dr. Robert M. White are chairman and vice-chauman, respectively, of the National
Research Council.
The work that provided the basis for this volume was supported by a conuact from He
U.S. Public Heals Service
Library of Congress Cataloging-in-Publication Data
Evaluating AIDS prevention programs I Susan L. Coyle, Robert F. Boruch, and Charles F. Turner, editors;
Panel on the Evaluation of AIDS Interventions, Committee on AIDS Research and the Behavioral, Social
and Statistical Sciences, National Research Council. - Expanded em
p. cm.
Includes bibliographical references and index.
ISBN ~309 (}4281-X (paper). - ISBN ~309 04288-7 (clod)
1. AIDS (Disease - Prevention-Evaluation-Methodology. I. Coyle, Susan L. II. Boruch, Robert
F. m. burner, Charles F. IV. mile.
[DNLM: 1. Acquired Immunodeficiency Syndrome-prevention & control. 2. Program Evaluation.
WD 308 N27725e] RA644.A25N27 1991
614.5'993~c20
DNLMJDLC
for Library of Congress
Copyright (~)1991 by the National Academy of Sciences
Printed in the United States of America
First Printing, January 1991
Second Printing, December 1993
9~13649
CIP
OCR for page R3
Panel on the Evaluation of AIDS Interventions
ROBERT F. BORUCH (Chairs, Graduate School of Education and Department
of Statistics, University of Pennsylvania
THOMAS J. COATES (Liaison to Committee), Division of General Internal
Medicine and Center for AIDS Prevention Studies, University of
California at San Francisco
JEFFREY P. DAVIS, Bureau of Community Health and Prevention, Wisconsin
Division of Health
BRIAN R. FLAY, Prevention Research Center, University of Illinois at
Chicago
ROBERT L. HUBBARD, Center for Social Research and Policy Analysis,
Research Triangle Institute
RONALD C. KESSLER,* Institute for Social Research, University of Michigan
NAN M. LAIRD, Department of Biostatistics, Haward University School of
Public Health
JOHN L. MARTIN, School of Public Health, Columbia University
LINCOLN E. MOSES, Department of Statistics, Stanford University
JESSICA M. UTTS, Division of Statistics, University of California at Davis
JOHN B. WAILER, JR., Department of Community Medicine, Wayne State
University School of Medicine
National Research Council Staff
TRACY L. BRANDT, Research Assistant, Committee on AIDS Research and
the Behavioral, Social, and Statistical Sciences
SUSAN L. COYLE, SIUdY Director, Panel on the Evaluation of AIDS
Interventions
KIRSTEN J. JOHNSON, Sr. Project Assistant, Panel on the Evaluation of
AIDS Interventions
CHARLES F. TURNER, Director, Committee on AIDS Research and the
Behavioral, Social, and Statistical Sciences
*
Served through Mary 31, 1990.
. . .
OCR for page R4
Committee on AIDS Research and the
Behavioral, Social, and Statistical Sciences
MARSHALL H. BECKER, School of Public Health, University of Michigan
ROBERT F. BORUCH, Graduate School of Education and Depamnent of
Statistics, University of Pennsylvania
THOMAS J. COATES, Division of General Internal Medicine, and Center for
AIDS Prevention Studies, University of California at San Francisco
RAMON C. CORTINES,* Superintendent of Schools, San Francisco Unified
School District
ROBYN M. DAWES, Department of Social and Decision Sciences, Carnegie
Mellon University
DON C. DES JARLAIS (Vice ChairJ, Chemical Dependency Institute, Beth
Israel Medical Center, New York, and Mount Sinai School of Medicine
JOHN H. GAGNON, Department of Sociology, State University of New York
at Stony Brook
ALBERT R. JONSEN, Department of Medical History and Ethics, University
of Washington at Seattle
SHIRLEY LINDENBAUM, Department of Anthropology, Graduate Center, City
University of New York
JANE MENKEN (Chair), Department of Sociology, and Population Studies
Center, University of Pennsylvanua
LINCOLN E. MOSES, Department of Statistics, Stanford University
CLADD E. STEVENS, The New York Blood Center
BAILUS WALKER, School of Public Health, University of Oklahoma
National Research Council Staff
TRACY L. BRANDT, Research Assistant, Committee on AIDS Research and
the Behavioral, Social, and Statistical Sciences
SUSAN L. COYLE, Study Director, Panel on He Evaluation of AIDS
Interventions
KIRSTEN J. JOHNSON, Sr. Project Assistant, Panel on the Evaluation of
AIDS Interventions
HEATHER G. MILLER,! Study Director, Panel on AIDS Interventions and
Research
KAREN E. ORLANDO,$ Sr. Project Assistant, Committee on AIDS Research
and the Behavioral, Social, and Statistical Sciences
JEFFREY S. STRYKER, Study Director, Panel on Monitoring the Social
Impact of He AIDS Epidemic
CHARI,ES F. TURNER, Director, Committee on AIDS Research and tile
Behavioral, Social, and Statistical Sciences
* Served through September 1, 1989. ~ Served through July 1, 1990. ~ Served through
September 1, 1990.
1V
OCR for page R5
Liaison Representatives to the Pane! from the
U.S. Public Health Service
ZILI AMSEL, National Institute on Drug Abuse
G. STEPHEN BOWEN, Centers for Disease Control
DAVID BROWNELL, Centers for Disease Control
ISABEL FERNANDEZ, Centers for Disease Control
JACOB GAYLE, Centers for Disease Control
JANINE JASON, Centers for Disease Control
C. FREDERIC KROGER, Centers for Disease Control
SUSAN MIDDLESTADT, Centers for Disease Control
KEVIN O 'REILLY, Centers for Disease Control
DEBORAH L. RUGG, Centers for Disease Control
ELLEN STOVER, National Institute of Mental Health
RONALD W. WILSON, National Center for Heals Statistics, Centers for
Disease Control
Consultants
MICHAEL L. COHEN, School of Public Affairs, University of Maryland
SAHR J. KPUNDEH, Depamnent of Political Science, Howard University
LEAH MAZADE, National Academy Press
LAURA RUDKIN-MINIOT, Department of Sociology, Princeton University
MICHAEL A. STOTO, Division of Heals Promotion and Disease Prevention,
Institute of Medicine
v
OCR for page R6
OCR for page R7
Preface
As the second decade of the epidemic of acquired immune deficiency
syndrome (AIDS) begins, behavioral change remains our primary weapon
in the fight against the spread of the human immunodeficiency virus
(HIV) and AIDS. It is unfortunate, however, that we know relatively little
as scientists about the comparative effectiveness of different strategies
intended to encourage and facilitate protective behavioral change among
persons who are at risk of HIV transmission.
It is our strong belief that such knowledge is an urgent national priority
and that it will require a long-term commitment to a course of rigorous
(and selective) research assessing current and future AIDS prevention
programs. Such research will enable us to learn from both our failures
and our successes in inducing protective changes in sexual and ~ug-using
behaviors.
The present report provides a bluepnnt for action. It has been under-
takenat the behest of He U.S. Public Health Service, which requested
assistance in developing strategies for evaluating the three major AIDS
prevention programs sponsored by the Centers for Disease Control. This
reports was prepared by the Pane] on the Evaluation of AIDS ~terven-
lions, which was convened by the National Research Council Committee
on AIDS Research and the Behavioral, Social, and Statistical Sciences.
The panel's work extends the committee's previous assessment of ~nter-
vention needs, reported in AIDS, Sexual Behavior, and Intravenous Drug
Use (Turner, Miller, and Moses, 1989~. That report argued strongly for
a national commitment to the careful design and evaluation of AIDS
intervention strategies.
We recognize Mat the high quality evaluation research recommended
in this report will be costly, but we believe that uncertainty about program
effectiveness is even more costly. The price of continued ignorance
will be measured both in dollars spent on ineffective programs and in
deaths and disease that might have been prevented. A commitment to
1 The present volume is an expanded edition of the Panel's 1989 report. This edition incorporates new
sections on nonexperimental approaches to intervention (Chapter 6 and Appendix F) and methodolog-
ical issues in mounting evaluations (Appendixes C-E).
. .
V11
OCR for page R8
Viii ~ PR EFACE
well-conducted evaluation research must be made now, and it must be
maintained if we are to improve our understanding of how to control this
deadly epidemic.
There is good reason to believe that this nation can and will do a
better job in the future in determining which AIDS prevention programs
do change behavior and which do not. Federal officials and people who
have been on the front lines of AIDS prevention activities since the early
days of the epidemic all express a considerable desire for evaluation of
their work. The paucity of rigorous evaluations conducted to date is not
due to individuals' unwillingness to undertake the research but to a lack
of resources and expertise. It is evident Hat past links between those
who provide services and manage programs, on the one hand, and those
who conduct research and evaluation, on the over, have not been strong.
It is our hope that this volume win foster such collaboration and
thereby advance our understanding of and ability to facilitate the behav-
ioral changes required to retard Be spread of REV in the fixture.
SUSAN L. COYLE, STUDY Director
ROBERT F. BORUCH, Chair
Pane} on the Evaluation of AIDS Interventions
CHARLES F. TURNER, Director
Committee on AIDS Research and the Behavioral,
Social, and Statistical Sciences
OCR for page R9
Acknowledgments
Duling the course of this study, the panel was assisted by a number of
scientists who took time to share their insights and expertise. To those
who assisted us in our work, the pane} extends its sincere tiaras and
appreciation. The panel's work was supported by the U.S. Public Health
Service.
Note on Contributions
This report is the collective product of pane! members and staff. The
content of this report reflects the deliberations of the panel, and the
report presents the panel's recommendations. The list below identifies
the persons who shared major responsibility for preparing initial drafts
of materials for each chapter in this report. The panel reviewed all
contributions, and they have been revised and edited in light of the
panel's discussions and the comments of outside reviewers. The purpose
of the following alphabetical list, therefore, is to give credit to individuals
but not to assign final responsibility for the published text. It should also
be noted that, although the list covers major sections of this volume,
these sections frequently contain additional paragraphs or pages from
over hands.
SUMMARY: Coyle, Turner
CHAPTER I: Boruch, Coates, Coyle, Laird, Moses, Turner
CHAPTER 2: Margin, win Coyle
CHAPTER 3: Flay and Kessler, with Utts
CHAPTER 4: Boruch, Hubbard, Turner, Wailer
CHAPTER 5: Boruch, Coates, Coyle, Davis, Hubbard, Martin, Turner
CHAPTER 6: Coyle, Moses, and Turner, with Boruch, Cohen,
Hubbard, Laird
APPENDIX A: Waller
APPENDIX B: Boruch
APPENDIX C: repented from Miller, Turner, and Moses (1990),
contributors—S. L`indenbaum, J. Sorensen, C. Turner, J. Wiley,
with P. Blumstein, R. Dawes, L. Rub Moot
1X
OCR for page R10
APPENDIX D: Coyly
APPENDIX E: Coyly
APPENDIX F congas a bound pier Spud far He panel by
Rood bomb; this per is Be sole ~sponsibUi~ of ilS =~0L
OCR for page R11
Contents
SUMMARY ....
·.!
1 DESIGN AND IMPLEMENTATION OF
EVALUATION RESEARCH 15
Awes of Evaluation, 16
Evaluation Research Design, 18
The Management of Evaluation, 27
References, 32
2 MEASUREMENT OF OUTCOMES..
Program Objectives, 34
Outcomes for Evaluations of HIV Prevention Programs, 37
Evaluation Measures, 45
References, 48
3 EVALUATING MEDIA CAMPAIGNS..
Background and Objectives, 51
Formative Evaluation: What works better? 53
Efficacy Tnals: Can Me campaign make a difference? 61
Process Evaluation: What is actually delivered? 66
Outcome Evaluation: Does the campaign
make a difference? 71
References, 81
4 EVALUATING HEALTH EDUCATION
AND RISK REDUCTION PROJECTS
Background and Objectives, 83
What Services Are Delivered? 86
Do the Projects Make a Difference? 93
What Works Better? 99
References, 100
X1
....34
. .50
..... 83
OCR for page R12
Xii ~ CONTENTS
5 EVALUATING HIV TESTING AND COUNSELING
PROJECTS .......................................
Background and Objectives, 102
How Well Are Services Delivered? 103
Options for Evaluating Whether HIV Testing and
Counseling Services Make a Difference, 114
What Works Better? 116
References, 122
6 RANDOMIZED AND OBSERVATIONAL APPROACHES TO
EVALUATING THE EFFECTIVENESS OF
AIDS PREVENTION PROGRAMS.............
Overview, 125
Randomized Experimentation, 128
Designing Comparability into Nonrandomized Studies, 144
Modeling and Statistical Adjustments for Bias, 168
When Should Nonrandomized Approaches
Be Considered? 182
Interpreting Evaluation Results, 186
References, 189
Appendixes
A Collaborative Contracting Strategy ...................
B Oversight and Coordination Strategy ........................
C Methodological Issues in AIDS Surveys ...................
D Sampling and Randomization: Technical Questions about
Evaluating CDC's Three Major AIDS Prevention Programs
E Ancillaly, Emerging, and Related Projects ....
F The Use of Selection Models to Evaluate AIDS Interventions
with Observational Data .........................
INDEX ......
...... 102
124
........ 197
.200
.207
. .
.317
335
342
.365
OCR for page R13
Evaluating AIDS
Prevention Programs
Expanded Edition
OCR for page R14
