Click for next page ( R2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Committee on Planning the Assessment/Evaluation of Programs Implemented Under the U.S. Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008 Board on Global Health Board on Children, Youth, and Families

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract No. SAQMMA09M0693 (STAT-3977) between the National Academy of Sciences and Department of State. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project. International Standard Book Number-13: 978-0-309-15393-5 International Standard Book Number-10: 0-309-15393-X Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu. Copyright 2010 by the National Academy of Sciences. All rights reserved. Printed in the United States of America. The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin. Suggested citation: IOM and NRC (Institute of Medicine and National Research Council). 2010. Strategic approach to the evaluation of programs implemented under the Tom Lantos and Henry J. Hyde U.S. Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008. Washington, DC: The National Academies Press.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1

OCR for page R1
COMMITTEE ON PLANNING THE ASSESSMENT/EVALUATION OF PROGRAMS IMPLEMENTED UNDER THE U.S. GLOBAL LEADERSHIP AGAINST HIV/AIDS, TUBERCULOSIS, AND MALARIA REAUTHORIZATION ACT OF 2008 ROBERT BLACK (Chair), Bloomberg School of Public Health, Johns Hopkins University MARTHA AINSWORTH, Independent Evaluation Group, World Bank PIERRE M. BARKER, University of North Carolina at Chapel Hill and Institute for Healthcare Improvement MARY T. BASSETT, Doris Duke Charitable Foundation RONALD BROOKMEYER, School of Public Health, University of California, Los Angeles DAVID D. CELENTANO, Bloomberg School of Public Health, Johns Hopkins University ANGELA DÍAZ, Mount Sinai School of Medicine LORETTA S. JEMMOTT, University of Pennsylvania School of Nursing JENNIFER KATES, Henry J. Kaiser Family Foundation ANN KURTH, New York University College of Nursing DORA MBANYA, University of Yaoundé I and Centre Hospitalier et Universitaire, Cameroun AFFETTE McCAW-BINNS, University of the West Indies, Mona Campus, Jamaica GEETA RAO GUPTA, Bill & Melinda Gates Foundation DOUGLAS D. RICHMAN, VA San Diego Healthcare System and University of California, San Diego DEBORAH L. RUGG, The Joint United Nations Programme on HIV/AIDS (UNAIDS) DAWN K. SMITH, U.S. Centers for Disease Control and Prevention SALLY K. STANSFIELD, Health Metrics Network, World Health Organization JANE WALDFOGEL, School of Social Work, Columbia University KATHRYN WHETTEN, Sanford School of Public Policy, Duke University CATHERINE M. WILFERT, Retired, Elizabeth Glaser Pediatric AIDS Foundation Study Staff KIMBERLY A. SCOTT, Study Director BRIDGET B. KELLY, Deputy Study Director IJEOMA EMENANJO, Senior Program Associate MILA C. GONZÁLEZ, Research Associate KATE MECK, Research Assistant KRISTEN DANFORTH, Senior Program Assistant CARMEN C. MUNDACA, Intern WENDY E. KEENAN, Program Associate JULIE WILTSHIRE, Financial Associate ROSEMARY CHALK, Director, Board on Children, Youth, and Families PATRICK KELLEY, Director, Boards on Global Health and African Science Academy Development v

OCR for page R1

OCR for page R1
Reviewers This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Stefano Bertozzi, Bill & Melinda Gates Foundation Greg Bisson, University of Pennsylvania School of Medicine Thomas Coates, University of California Los Angeles, School of Medicine Lola Dare, CHESTRAD International Victor DeGruttola, Harvard School of Public Health Theresa Díaz Vargas, United Nations Children’s Fund Johanna Dwyer, Tufts University School of Medicine and Friedman School of Nutrition Science & Policy Robert Hecht, Results for Development Institute Phyllis Kanki, Harvard School of Public Health Kathy Marconi, University of Maryland University College Peter Mugyenyi, Joint Clinical Research Center, Kampala, Uganda Robert R. Redfield, University of Maryland School of Medicine Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Bernard Guyer, Johns Hopkins University Bloomberg School of Public Health and Edward Perrin, Emeritus, University of Washington. Appointed by the National Research Council and Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. vii

OCR for page R1

OCR for page R1
Acknowledgments The committee would like to acknowledge the diverse and important contributions of the many individuals whose assistance is reflected in this report. First, the Committee would like to thank the Office of the U.S. Global AIDS Coordinator and the congressional staff who provided information, guidance, and support. The committee also greatly benefited from the time and expertise of those who met with committee delegations and participated in public information- gathering sessions. These individuals are listed in full in Appendix C. There are a number of individuals who were critical for the administrative and logistical success of this project. For help with scheduling and communication, the committee would like to thank Mary Rybczynski, Rachel White, Lola Adedokun, Nancy Leonard, Polina Royzman- Tabak, Fortuna Salinas, Tanya Davis-Powell, Cindy Ogasawara, Elvira Bustamante, Audrey Palix, Aubrey Celeste C. Musngi-Anouar, Cindy Chu, and Sharon Abbruscato. The committee is also grateful to Anthony Mavrogiannis and the staff at Kentlands travel for their assistance with the travel needs of this project. The committee would also like to thank Kristin Shaw and Megan Perez for their excellent work on this project as interns at the Institute of Medicine. Carmen Mundaca also deserves special mention for her valuable research and analytic contributions during the time she devoted to the project as an intern within her doctoral training program. ix

OCR for page R1

OCR for page R1
Preface By the first decade of the 21st century, the world had been grappling with the HIV/AIDS pandemic for nearly three decades. The countries hardest hit in terms of morbidity and mortality remain in sub-Saharan Africa, home to an estimated two-thirds of people living with HIV infection. Although international aid has increased substantially since the beginning of the pandemic and national government expenditure for HIV prevention, treatment, care, and capacity building activities also increased in the most affected countries, there remains a funding gap relative to the estimated need. In 2003, Congress mandated a study to be conducted by the Institute of Medicine (IOM) to assess the progress of the implementation of the programs and aid offered in a major new U.S initiative that became known as the President’s Emergency Plan for AIDS Relief (PEPFAR). The findings and recommendations of that study informed the processes, policies, and activities of the program and the reauthorization legislation, 1 known in short as the Lantos–Hyde Act of 2008. This reauthorized legislation mandated another study by the IOM to assess the performance of United States-assisted global HIV/AIDS programs and evaluate the impact on health of prevention, treatment, and care efforts that are supported by United States funding (for a complete description of the elements to be considered in the assessment and evaluation, see the Statement of Task in Appendix A). In addition to informing Congress and the Department of State, this newly-mandated, evidence-based IOM study will provide the scientific community, program implementers, policy makers, civil society, people living with and affected by HIV/AIDS, and international stakeholders in global public health with a rigorous, non-partisan, multidisciplinary, and independent evaluation of the PEPFAR program. This report outlines the design plan for the evaluation of this evolving program that has geographically expanded to more than twice the number of countries funded at the time of the first evaluation. This new evaluation is complicated by not only its scale, but also the diversity created by the characteristics and complexities of each country in which the program operates. The dynamism as a program in operation parallel to the evaluation itself—and the challenges that presents—also add complexity. The context in which PEPFAR operates has also shifted, with a more recent emphasis on transitioning from an emergency response to a longer-term model of sustainability, promoting country ownership, and strengthening health systems. PEPFAR also has a new place within the context of international funding as part of the new U.S. Global Health Initiative (GHI). Less than a year after PEPFAR was reauthorized, the Obama Administration launched the GHI—a new 6-year (2009–2014), $63 billion government-wide effort to develop a comprehensive U.S. global health strategy (these funds include PEPFAR funds). The GHI includes and builds on the success of PEPFAR, but also includes other global health challenges in a more coordinated approach. As such, the GHI is expected to affect the way in which all U.S. global health programs operate, including PEPFAR. This plan includes an illustration of the types of questions that could be addressed in the evaluation, partitioning and elaborating the areas of interest described in the statement of task. 1 Tom Lantos and Henry J. Hyde United States Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008, Public Law 110-293, 110th Cong., 2nd sess. (July 30, 2008). xi

OCR for page R1
xii PREFACE The ability to answer these evaluation questions will depend on the availability of timely and quality data from the Department of State’s Office of the Global AIDS Coordinator and other sources such as other federal agencies, U.S. Country Teams and implementing partners, other donors, and other international stakeholders including the Global Fund to Fight AIDS, Tuberculosis, and Malaria, the World Health Organization, the Joint United Nations Programme on HIV/AIDS, and the United Nations Children’s Fund. While the evaluation questions illustrated in this plan will undergo further refinement and prioritization, they are presented here to facilitate an understanding of not only the type of data needed from the numerous sources, but also the processes and temporal complexities inherent in responding to the committee’s charge. The evaluation will be conducted between 2010 and 2012, with a report of the committee’s findings and recommendations issued in 2012. It will provide insights into the contributions of the investment made by the United States through PEPFAR to improve and save lives of men, women, adolescents, and children living with and affected by HIV/AIDS in developing countries. In the current time of global economic recessions, limited resources, and stated intentions to transition to a more sustainable and country-owned response to the pandemic, an even more urgent priority must be accorded to the identification, dissemination, and scale-up of the most effective strategies for preventing new HIV infections and meeting the myriad needs of people already living with HIV/AIDS. The committee hopes to contribute to this knowledge and understanding. The committee extends its gratitude to all those who provided information to assist in the planning committee’s work. In our initial data gathering, we have been able to forge positive relationships with major global stakeholders. Perhaps the most important milestone in this planning phase is the commitment of those major global stakeholders to share their data, analyses, and other invaluable information with the IOM, thereby modeling indispensable international collaboration to collectively address and understand critical issues and outcomes related to the pandemic. In closing, I would like to express my appreciation to the study staff for their superb work on this project and to the members of the committee for the time and energy they gave so generously to this project, for the expertise they contributed, and for their participation in robust discourse and deliberation. Robert E. Black, Chair Committee on Planning the Assessment/Evaluation of HIV/AIDS Programs Implemented Under the U.S. Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008

OCR for page R1
Contents SUMMARY OF KEY MESSAGES.......................................................................................................1 PART I: INTRODUCTION AND BACKGROUND Introduction...............................................................................................................................................7 Background...............................................................................................................................................8 PART II: PROPOSED EVALUATION APPROACH Congressional Charge and Planning Phase Approach..........................................................................19 Evaluation Goals and Conceptual Framework for Evaluation Design................................................20 Overarching Evaluation Challenges and Limitations...........................................................................27 Evaluation Methodologies .....................................................................................................................28 Data Sources and Analysis.....................................................................................................................38 Workplan ................................................................................................................................................46 PART III: ILLUSTRATIVE EVALUATION DETAILS FOR ASSESSMENT OF PEPFAR’S PERFORMANCE AND IMPACT Section 1: PEPFAR’s Knowledge Management ..................................................................................53 Section 2: Mapping PEPFAR Funding .................................................................................................61 Section 3: Prevention Services ..............................................................................................................69 Section 4: Adult and Pediatric Treatment .............................................................................................79 Section 5: Care and Support Services ...................................................................................................89 Section 6: Child and Adolescent Well-Being .......................................................................................99 Section 7: Gender-Related Vulnerability and Risk.............................................................................115 Section 8: Key Systems-Level Goals and Activities ..........................................................................121 REFERENCES.....................................................................................................................................133 APPENDIXES A Statement of Task.............................................................................................................................143 B Committee and Staff Biographies....................................................................................................145 C Public Committee Meeting Agendas...............................................................................................159 D Glossary ...........................................................................................................................................163 E Sample PEPFAR Country Timelines...............................................................................................175 F Evaluation Committee Workplan.....................................................................................................185 G Sample Data Sources Matrix ...........................................................................................................189 H Illustrative Questions for the Evaluation of PEPFAR’s Health Systems Strengthening Activities ..........................................................................................................193 xiii

OCR for page R1

OCR for page R1
Tables, Figures, and Boxes TABLES 1 Current HIV/AIDS Funding from Select Donors Received by PEPFAR Countries (as of April 2010), 64 2 Categories of PEPFAR Prevention Interventions, 71 3 Cumulative Number of People on Antiretroviral Treatment by PEPFAR Country, 79 4 PEPFAR Age Categories for Programs for Orphans and Vulnerable Children, 105 FIGURES 1 Organizational structure of OGAC (last updated October 16, 2007), 11 2 Model structure of PEPFAR Country Teams, 13 3 Program impact pathway for evaluation of PEPFAR’s effects on HIV-related health impact for children and adults, 24 4 Context for PEPFAR program implementation, 26 5 PEPFAR headquarters-level strategic information partners and headquarters-level data- gathering sources, 54 6 A public health questions approach to HIV monitoring and evaluation, 56 7 PEPFAR funding landscape, 62 8 PEPFAR FY2008 obligations to prime partners: U.S., local, or other location stratified by key sector, 67 9 Program impact pathway for evaluation of PEPFAR’s prevention programs, 73 10 Program impact pathway for evaluation of PEPFAR’s adult treatment programs, 85 11 Program impact pathway for evaluation of PEPFAR’s pediatric treatment programs, 86 12 PEPFAR care and support services, 90 13 Program impact pathway for evaluation of PEPFAR’s care and support programs, 95 14 Program impact pathway for evaluation of PEPFAR’s services for children and adolescents, 109 15 Program impact pathway for evaluation of PEPFAR’s gender efforts, 117 16 Representation of WHO’s six building blocks for effective health systems, 126 17 Program impact pathway for evaluation of PEPFAR’s health systems strengthening activities at the country level, 128 BOXES 1 PEPFAR and the U.S. Global Health Initiative, 16 2a Central U.S. Government Data Sources, 40 2b Multilateral Donor and Other International Data Sources, 42 2c Country Data Sources, 44 2d Other Data Sources (Single or Multi-Country), 45 3 Areas of Emphasis in PEPFAR II’s New Five-Year Strategy for Treatment, 84 4 Reauthorization Programming for Orphans and Vulnerable Children, 102 xv

OCR for page R1

OCR for page R1
Acronyms and Abbreviations AEI African Education Initiative AIDS acquired immune deficiency syndrome APR annual program results ART antiretroviral therapy ARV antiretroviral BCC behavior change communication CD4 cluster of differentiation 4 CDC U.S. Centers for Disease Control and Prevention COP country operational plan COPRS Country Operational Plan Reporting System CRC Committee on the Rights of the Child CSW commercial sex worker CTX cotrimoxazole DHS Demographic and Health Surveys DoS U.S. Department of State EID early infant diagnosis of HIV FY fiscal year GHI The U.S. Global Health Initiative Global Fund The Global Fund to Fight AIDS, Tuberculosis, and Malaria HAPSAT HIV/AIDS Program Sustainability Analysis Tool HIV human immunodeficiency virus IDU injecting drug user IOM U.S. Institute of Medicine IPTp intermittent preventive treatment of malaria for pregnant women ITNs insecticide-treated nets M&E monitoring and evaluation MDG Millennium Development Goal MICS Multiple Indicator Cluster Survey MSM men who have sex with men NGO non-governmental organization OECD Organisation for Economic Co-operation and Development OGAC Office of the U.S. Global AIDS Coordinator OI opportunistic infection OMB Office of Management and Budget PCR polymerase chain reaction PEP post-exposure prophylaxis PEPFAR The President’s Emergency Plan for AIDS Relief PEPFAR I The President’s Emergency Plan for AIDS Relief (2004–2008) PEPFAR II The President’s Emergency Plan for AIDS Relief (2009–2013) PHE public health evaluation PLWHA people living with HIV/AIDS PMI The President’s Malaria Initiative xvii

OCR for page R1
xviii ACRONYMS AND ABBREVIATIONS PMTCT prevention of mother-to-child transmission PrEP pre-exposure prophylaxis SI strategic information SPRs semi-annual progress results TAB Technical Advisory Board TB tuberculosis TWG Technical Working Group UNAIDS United Nations Joint Programme on HIV/AIDS UNGASS United Nations General Assembly Special Session UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government WHO World Health Organization