THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract/Grant No. SAQMMA09M0693 between the National Academy of Sciences and the U.S. Department of State. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
International Standard Book Number-13: 978-0-309-26780-9
International Standard Book Number-10: 0-309-26780-3
Library of Congress Control Number: 2013939517
Additional copies of this report are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu.
Copyright 2013 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin.
Suggested citation: IOM (Institute of Medicine). 2013. Evaluation of PEPFAR. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
COMMITTEE ON THE OUTCOME AND IMPACT EVALUATION OF GLOBAL HIV/AIDS PROGRAMS IMPLEMENTED UNDER THE LANTOS-HYDE ACT OF 2008
ROBERT E. BLACK (Chair), Johns Hopkins University, Baltimore, MD
JUDITH D. AUERBACH, Consultant, San Francisco AIDS Foundation, CA
MARY T. BASSETT, Doris Duke Charitable Foundation, New York, NY
RONALD BROOKMEYER, University of California, Los Angeles
LOLA DARE, Center for Health Sciences Training, Research and Development International, Ibadan, Nigeria
ALEX C. EZEH, African Population and Health Research Center, Nairobi, Kenya
SOFIA GRUSKIN, University of Southern California, Los Angeles
ANGELINA KAKOOZA, Makerere University College of Health Sciences, Kampala, Uganda
JENNIFER KATES, Henry J. Kaiser Family Foundation, Washington, DC
ANN KURTH, New York University, New York
ANNE C. PETERSEN, University of Michigan and Global Philanthropy Alliance, Ann Arbor
DOUGLAS D. RICHMAN, VA San Diego Healthcare System and University of California, San Diego
JENNIFER PRAH RUGER, Yale University, New Haven, CT
DEBORAH L. RUGG, United Nations Inspection and Evaluation Division, New York, NY
DAWN K. SMITH, U.S. Centers for Disease Control and Prevention, Atlanta, GA
PAPA SALIF SOW, Bill & Melinda Gates Foundation, Seattle, WA
SALLY K. STANSFIELD,1 Independent Consultant, Geneva, Switzerland
TAHA E. TAHA, Johns Hopkins University, Baltimore, MD
KATHRYN WHETTEN, Duke University, Durham, NC
CATHERINE M. WILFERT, Retired, Elizabeth Glaser Pediatric AIDS Foundation, Durham, NC
Consultants
SHARON KNIGHT, East Carolina University, Greenville, NC
KATHRYN TUCKER, Statistics Collaborative, Inc., Washington, DC
JANET WITTES, Statistics Collaborative, Inc., Washington, DC
__________________
1 Committee member through August 2012.
Staff
KIMBERLY A. SCOTT, Study Co-Director
BRIDGET B. KELLY, Study Co-Director
MARGARET HAWTHORNE, Program Officer
LIVIA NAVON, Program Officer
CARMEN CECILIA MUNDACA, Postdoctoral Fellow
IJEOMA EMENANJO, Senior Program Associate (through January 2011)
MILA C. GONZÁLEZ DÁVILA, Associate Program Officer (through August 2012)
KRISTEN DANFORTH, Research Associate
REBECCA MARKSAMER, Research Associate (from August 2012)
KATE MECK, Research Associate
COLLIN WEINBERGER, Research Associate (April 2011 through June 2012)
LEIGH CARROLL, Research Assistant (from October 2011)
TESSA BURKE, Senior Program Assistant (through May 2011)
ANGELA CHRISTIAN, Program Associate
WENDY E. KEENAN, Program Associate
JULIE WILTSHIRE, Financial Associate
KIMBER BOGARD, Board Director, Board on Children, Youth, and Families (from October 2011)
ROSEMARY CHALK, Board Director, Board on Children, Youth, and Families (through July 2011)
PATRICK KELLEY, Senior Board Director, Boards on Global Health and African Science Academy Development
Reviewers
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
PIERRE BARKER, Institute for Healthcare Improvement
CHRIS BEYRER, Johns Hopkins University
ANASTASIA TZAVARAS CATSAMBAS, EnCompass LLC
DAVID CELENTANO, Johns Hopkins University
PAUL DE LAY, Joint United Nations Programme on HIV/AIDS
WAFAA M. EL-SADR, Columbia University
KURT FIRNHABER, Right to Care
MITCHELL H. GAIL, National Institutes of Health
ROBERT GROSS, University of Pennsylvania
JOHN E. LANGE, Bill & Melinda Gates Foundation
CHEWE LUO, United Nations Children’s Fund
JONATHON LEE SIMON, Boston University
RJ SIMONDS, Elizabeth Glaser Pediatric AIDS Foundation
SHOSHANNA SOFAER, City University of New York
MIRIAM WERE, University of Nairobi
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Kristine M. Gebbie, Flinders University School of Nursing and Midwifery, and Ann M. Arvin, Stanford University. Appointed by the National Research Council and Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
Acknowledgments
The committee, project staff, and consultants are deeply appreciative of the diverse and valuable contributions made by so many who assisted with this study.
For information and support provided throughout the project, we thank the staff of the Office of the U.S. Global AIDS Coordinator, with particular gratitude to Paul Bouey and Tiffany Parker, who facilitated our engagement during the study, as well as staff from other President’s Emergency Plan for AIDS Relief (PEPFAR) implementing agencies. We are also grateful to the leadership of the U.S. missions and the PEPFAR staff in the countries visited for this evaluation, whose hospitable and gracious assistance was essential to the success of our country visits. We also thank the many other individuals in the countries visited who assisted us with logistics and support during our visits.
We are grateful to Eran Bendavid from Stanford University for his valuable consultation during the study. We appreciate the essential technical support provided to the project by Danielle Beaulieu, Jessica Case, Megan Somerday, Jeff Steen, and Neil Wohlford from Statistics Collaborative, Inc. We thank Kathryn Stadeli from University of California, San Diego, School of Medicine for her assistance with research for the study. We also thank Megan Perez and Meredith Cantwell for their excellent work on this study as interns at the Institute of Medicine as well as Wyatt Smith and Peter Dull, who provided temporary assistance during the project. We are grateful to Teresa Bergen and Diane Wellman for their diligent work as transcriptionists. We appreciate the creativity and effort of Jay Christian
and LeAnn Locher for their design work. In addition, we convey our deep gratitude and appreciation for the hard work of the many staff in various offices of the Institute of Medicine and the National Academies who lent their support to the project.
There are a number of other individuals who were crucial for the administrative and logistical success of this project. For help with scheduling and communication for the committee, we thank Sharon Abbruscato, Lola Adedokun, Philomena Agaloi, Jillian Albertolli, Michele Augustus, Nkiru Azikiwe, Anna Both, Cindy Chu, Kyle Hamilton, Jennifer Heflin, Maria Male, Sheila Mwero, Catherine Nyawire, Audrey Palix, Jessica Raback, Mary Rybczynski, Fortuna Salinas, Abir Shady, Cassie Toner, Rachel Upton, Kevin Vavasseur, Jackylene Wegoki, and Marie Young. We are also immensely grateful to Anthony Mavrogiannis and the staff at Kentlands Travel for their assistance with the complex travel needs of this project.
Finally, although we cannot name them here for reasons of confidentiality, we offer our most profound thanks to the hundreds of individuals who participated in interviews and site visits as part of the evaluation data collection effort. Their generosity with their time and their willingness to share their insights were fundamental to the evaluation; it was a privilege and an inspiration to hear directly from those whose dedication and tremendous effort underlie the successes of the response to HIV globally and in PEPFAR partner countries.
PART II: PEPFAR ORGANIZATION AND INVESTMENT
3 PEPFAR ORGANIZATION AND IMPLEMENTATION
Organization of PEPFAR at the Central/Headquarters Level
Organization of PEPFAR at the Country Level
Perspectives on Interagency Implementation
PEPFAR Implementation in the Context of the HIV Epidemic in Partner Countries
PEPFAR Implementation and the Policy Environment
4 U.S. FUNDING FOR THE PEPFAR INITIATIVE
PEPFAR’s Contribution Relative to Other Donors
Overview of the PEPFAR Funding Process
PEPFAR Funding Levels and Distribution by Programs and Partners
PEPFAR Funding by Country Characteristics
Strategic Use of PEPFAR Resources
PART III: PEPFAR PROGRAMMATIC ACTIVITY
Evolution of HIV Prevention Science
Overview of PEPFAR-Supported Prevention Programs
Prevention of Sexual Transmission
Prevention of Mother-to-Child Transmission
Blood and Medical Injection Safety
Clinical Care and Nonclinical Support Services
Summation for PEPFAR’s Support for Care and Treatment Services
Ongoing Challenges with ART Coverage
Sustainability of Care and Treatment
Funding History for PEPFAR Support for Children and Adolescents
PEPFAR’s Programs and Services for Orphans and Vulnerable Children
PEPFAR’s Programs and Child Survival
Measurement and Evaluation of Gender Efforts
9 STRENGTHENING HEALTH SYSTEMS FOR AN EFFECTIVE HIV/AIDS RESPONSE
Background and Context for Systems Development and Functioning for Health
Overview of PEPFAR’s Health Systems Strengthening Activities
PART IV: FUTURE OF U.S. GOVERNMENT INVOLVEMENT IN THE GLOBAL RESPONSE TO HIV/AIDS
10 PROGRESS TOWARD TRANSITIONING TO A SUSTAINABLE RESPONSE IN PARTNER COUNTRIES
Evolution of the U.S. Response to Global HIV
Country Ownership: A Fundamental Element of Progress Toward Sustainability
Other Key Elements for Achieving Sustainability
Key Barriers to Achieving Country Ownership and Sustainability
11 PEPFAR’S KNOWLEDGE MANAGEMENT
Program Targets and Priorities
PEPFAR Support for Epidemiological Data
PEPFAR Support for Data Use by Partner Country Stakeholders
PEPFAR-Supported Evaluation and Research Activities
Knowledge Transfer and Learning Within PEPFAR
PEPFAR’s Knowledge Dissemination External to PEPFAR
Transitioning to a Sustainable Response in Partner Countries
Tables, Figures, and Boxes
TABLES
1-1 PEPFAR HIV/AIDS Programs in 2004
1-2 Summary of PEPFAR’s Goals, Budgetary Requirements, and Targets
2-1 Country Visit Interviews by Stakeholder Type
2-2 PEPFAR Indicators Consistent Across the Duration of PEPFAR
4-1 Total PEPFAR Outlaid Funding by Reporting Year (the Year the Funding Was Expended), with Disaggregation by Budget Year (the Year the Funding Was Made Available) (in Current USD Millions)
4-2 PEPFAR Outlays by Reporting Year (the Year the Funding Was Expended), for Subsets of Countries (Current USD Millions)
4-3 PEPFAR Countries Grouped by 2009 Prevalence
4-4 Average PEPFAR Funding per PLHIV (Current USD)
4-5 PEPFAR Countries Grouped by 2004 Income Level
4-6 Average PEPFAR Funding per PLHIV (FY 2005–FY 2010) (Current USD) for Partner Countries Grouped by Income and HIV Prevalence
5-1 Interventions Included in PEPFAR Guidance Over Time for Prevention of Sexual Transmission of HIV
5-2 OGAC Indicator 2.1—Number of Individuals Reached Through Community Outreach That Promotes HIV/AIDS Prevention Through Abstinence and/or Being Faithful (in Millions)
5-3 OGAC Indicator 5.2—Number of Individuals Reached Through Community Outreach That Promotes HIV/AIDS Prevention Through Other Behavior Change Beyond Abstinence and/or Being Faithful (in Millions)
5-4 Number of HIV-Positive Pregnant Women Receiving ARV Prophylaxis for PMTCT (PEPFAR and National) (in Thousands)
6-1 Number of Individuals Who Received Counseling and Testing for HIV and Received Test Results (in Millions)
6-2 Number of Individuals Provided with Care (in Millions)
6-3 Number of HIV-Positive Adults and Children Receiving a Minimum of One Clinical Service (in Millions)
6-4 Number of HIV-Positive Patients in HIV Care Who Started TB Treatment (in Thousands)
6-5 Number of USG-Supported Service Outlets Providing Treatment for TB to HIV-Infected Individuals (in Thousands)
6-6 HIV-Positive Patients Who Were Screened for TB in HIV Care or Treatment Settings (in Millions)
6-7 Number of Registered TB Patients Who Received HIV Counseling, Testing, and Their Test Results at a USG-Supported TB Service Outlet (in Thousands)
6-8 Number of HIV-Positive Persons Receiving Cotrimoxazole Prophylaxis (in Millions)
6-9 Number of HIV-Positive Clinically Malnourished Clients Who Received Therapeutic or Supplementary Food (in Thousands)
6-10 Adult and Pediatric Treatment Guidelines Adoption by Country
6-11 Care and Treatment Budgetary Allocation Requirement: Documented Planned/Approved Funding Over Time (in USD Millions)
6-12 Number of Adults and Children with Advanced HIV Infection Receiving ART (in Millions)
6-13 Currently Enrolled Adults in ART, in Thousands (Annual, FY 2005–FY 2010)
6-14 Newly Enrolled Adults in ART by Sex (Annual, FY 2005–FY 2011) (in Thousands)
6-15 Newly Enrolled Children in ART (FY 2005–FY 2011) (in Thousands)
7-1 Tracking the Legislative Budgetary Requirement for OVC Programming (in USD Millions)
7-2 PEPFAR Age Categories for Programs for Orphans and Vulnerable Children
7-3 OVC Indicator Targets and Results (in Millions)
8-1 Inclusion of Gender in PEPFAR Guidance Documents Over Time, 2003–2012
8-2 Sex-Disaggregated Indicators Routinely Reported to OGAC
9-1 Health System Constraints with Potential Disease-Specific and Health System Responses
9-2 PEPFAR Indicators Related to Leadership and Governance (Organizations)
9-3 PEPFAR Indicators Related to Leadership and Governance (Individuals)
9-4 Total Expenditure on Health per Capita at Exchange Rate
9-5 PEPFAR Indicators Related to Strategic Information and Information Systems
9-6 PEPFAR Indicators Related to Workforce Training (FY 2004–FY 2009)
9-7 PEPFAR Indicators Related to Workforce Training (FY 2010)
10-1 OGAC-Identified Dimensions and Operational Definitions for Country Ownership
10-2 PEPFAR-identified Insights from an Internal Study Commissioned by OGAC on the Principles of Country Ownership
11-1 Key PEPFAR Targets Under Legislation and Strategy Mandates
11-2 Number of PEPFAR Indicators by Reporting Status and Year of Indicator Guidance
11-3 PEPFAR Indicators Consistent Across the Duration of PEPFAR
11-4 Level of Harmonization of Next Generation Indicators with Global Indicators
11-5 Evolution of PEPFAR-Supported Evaluation and Research Activities
11-6 Types of Knowledge Transferred in PEPFAR, Beyond Routine Reporting
11-7 Mechanisms of Knowledge Transfer in PEPFAR
C-1 Country-Level Indicators Reported During FY 2004–FY 2009
C-2 Primary Indicators for PEPFAR Next Generation Indicators (FY 2010–Present)
C-3 Overlapping Country-Level Phase 1 and Primary Phase 2 Indicators
C-4 Country Visit Interviews by Stakeholder Type
FIGURES
2-1 Program impact pathway for evaluation of PEPFAR’s effects on HIV-related health impact for children and adults
2-2 Context for PEPFAR contribution in partner countries
2-3 Overall data collection and analysis process
3-1 PEPFAR overall organization and implementation
3-2 Organizational structure of OGAC (last updated November 14, 2011)
3-3 Example structure of PEPFAR mission team
4-1 Total donor disbursements for HIV/AIDS in PEPFAR partner countries (constant 2010 USD billions)
4-2 PEPFAR overall funding flows framework
4-3 Congressional appropriations for PEPFAR, FY 2004–FY 2011 (current USD billions)
4-4 Cumulative obligations and outlays, FY 2004–FY 2011 (current USD billions)
4-5 Proportion of cumulative available PEPFAR funding by obligation and outlay status at the end of each fiscal year (bars) and the cumulative total of funding that has not been outlaid (line) (current USD billions)
4-6 Planned/approved funding for USG implementing agencies, FY 2005–FY 2011 (constant 2010 USD billions)
4-7 Proportion of planned/approved funding for PEPFAR operational plan programs, FY 2005–FY 2011
4-9 Planned/approved funding for PEPFAR country activities in current USD millions (left axis and bars) and as a percentage of total planned/approved funding (right axis and lines)
4-10 Planned/approved funding for PEPFAR HQ programs in current USD millions (left axis and bars) and as a percentage of total planned/approved funding (right axis and lines)
4-11 Planned/approved funding for multilateral partners in current USD millions (left axis and bars) and as a percentage of total planned/approved funding (right axis and lines)
4-12a Planned/approved funding by technical area (constant 2010 USD millions)
4-12b Proportion of planned/approved funding by technical area
4-13 Proportion of PEPFAR funding by origin of prime partner in 13 PEPFAR partner countries
4-14 Percentage of PEPFAR funding by type of prime partner in 13 PEPFAR partner countries
4-15 PEPFAR funding for local prime partners
4-16 PEPFAR planned/approved funding by 2009 prevalence groupings in 31 PEPFAR partner countries (current USD millions)
4-17 PEPFAR planned/approved funding by income level in 31 PEPFAR partner countries (current USD millions)
5-1 PEPFAR’s planned/approved funding over time for prevention (FY 2005–FY 2011)
5-2 AIDS diagnoses among perinatally infected persons, 1985–2010, in the United States and six U.S.-dependent areas
5-4 PEPFAR’s contribution to PMTCT coverage, 2006 to 2009 (aggregate data from 31 countries)
6-1 Implementation cascade for the continuum of care
6-2 Planned/approved funding over time for counseling and testing services
6-3 PEPFAR care and support services
6-4 Planned/approved funding over time for care and support services
6-5 Planned/approved funding over time for treatment
6-6 Total enrolled and newly enrolled individuals (adults and children) in ART (quarterly, FY 2005–FY 2011)
6-7 Number of newly enrolled adults in ART by sex (FY 2005–FY 2011)
6-8 Proportion of newly enrolled children in ART by age groups (FY 2008–FY 2011)
6-9 Twelve-month retention (alive and in care) by population and by the year ART was started in a subset of patients in nine PEPFAR partner countries
6-10 Proportion of patients on ART that remain in care on ART over time by population in a subset of patients in nine PEPFAR countries
6-11 Proportion of patients on ART that remain in care over time by year of ART initiation in a subset of patients in nine PEPFAR partner countries
6-12 Survival by population (2004–2011) in a subset of patients in nine PEPFAR partner countries
6-13 Survival by year of ART initiation (2004–2011) in a subset of patients in nine PEPFAR partner countries
6-14a Differences between men and women on ART in survival (7 countries, 165 clinics), 2004–2011
6-14b Differences between men and women on ART in baseline characteristics (7 countries, 165 clinics), 2004–2011
6-15 2006 estimated HIV prevalence and ART coverage
6-16 2009 estimated HIV prevalence and ART coverage
6-17 Number of adults (>15 years) eligible for ART in low-and middle-income countries, by region, according to WHO 2006 (CD4<200) and 2010 (CD4<350) guidelines
7-1 Planned/approved funding over time for services for children and adolescents
7-2 All-cause and AIDS deaths for children under 5 years, in select high-child-mortality-burden PEPFAR countries
8-1 Gender-based violence and HIV
8-2 HIV prevalence in MSM compared to HIV prevalence in all adults in 2010
9-1 Representation of WHO’s six building blocks for effective health systems
9-2 PEPFAR funding for HSS (country activities) (constant 2010 USD millions)
9-3 External resources for health as percent of total health expenditure, 2010
9-4 Components of a health information system (HIS)
9-5 Data needs and sources at different levels of the health care system
9-6 Select indicators related to PEPFAR’s laboratory activities
9-7 Health system building blocks represented as a house
11-1 PEPFAR funding for country-level strategic information in constant 2010 dollars and as percentage of total PEPFAR funding
11-2 Number of indicators routinely reported to OGAC by Next Generation Indicator (NGI) reporting category and guidance year
11-3 Ongoing PEPFAR Public Health Evaluation (PHE) studies, by country, December 2011
11-4 Organizations implementing ongoing PEPFAR Public Health Evaluation (PHE) studies, by implementing organizations’ country, December 2011
11-5 Implementation science awards, by country
11-6 Organizations implementing PEPFAR Implementation Science studies, by implementing organizations’ country, October 2012
11-7 Potential pathways of knowledge transfer within PEPFAR
11-8 PEPFAR-supported journal publications, by year, 2004–2011
11-9 Suggested elements of a PEPFAR comprehensive knowledge management framework
11-10 Recommended PEPFAR tiered reporting in the context of partner country and global reporting systems
C-1 Country visit qualitative data collection process
BOXES
3-1 Examples of Vulnerable Populations Identified from Country Visit Interview Data
4-1 Definitions for Selected Financial Terms
4-2 FY 2011 PEPFAR Budget Code Definitions by Technical Area
5-1 PEPFAR’s Adoption and Scale Up of Voluntary Medical Male Circumcision
5-2 Centrally Reported Next Generation Indicators for Prevention of Sexual Transmission
9-1 PEPFAR Budget Code Definitions for HSS
9-2 OGAC Definitions of Technical Assistance (TA) Related to Leadership and Governance
9-3 Select Innovative Financing Mechanisms from Committee-Collected Interview Data
9-4 Select Examples of PEPFAR-Supported Information Systems
9-6 PEPFAR’S Laboratory Systems Strengthening Initiatives Over Time
9-9 Select Examples of PEPFAR-Supported Models and Approaches to Service Integration
10-1 Select Global Accords That Influence Sustainability of HIV/AIDS Responses
10-2 Measures of Progress and Achievements in the Paris Declaration
10-3 Elements of Country Ownership from Interview Data
10-4 IOM Committee-Recognized Impediments to Country Ownership from Interview Data Analysis
10-5 OGAC’s 14 Initiatives to Address Priority Themes to Accelerate Country Ownership
10-6 USG-identified Potential Measures of Success for Country Ownership
11-1 Select PEPFAR Efforts to Align with Partner Country M&E Systems
11-2 Institutional Affiliations of Scientific Advisory Board Members, October 2012
11-3 Pathways of Knowledge Transfer in PEPFAR, Beyond Routine Reporting
11-4 “Organization X” Innovative Knowledge Transfer
The HIV/AIDS pandemic has beleaguered the world for more than three decades. The countries most affected continue to be in sub-Saharan Africa, home to an estimated two-thirds of people living with HIV. There have been major increases in international aid assistance as well as in national commitments to and investments in HIV prevention, treatment, care, and capacity building activities, yet funding remains insufficient to meet the estimated immediate and projected needs.
In 2003, in response to the devastating consequences of the HIV pandemic, the U.S. Congress funded a major new U.S. global health initiative, which became known as the President’s Emergency Plan for AIDS Relief, or PEPFAR.1 PEPFAR remains the largest bilateral initiative aimed at addressing HIV/AIDS. At the time of its initial authorization, PEPFAR was seen as a bold initiative, testing, among other strategies, whether treatment could be successfully and intensively scaled up in low-resource settings. The initial authorizing language mandated that the Institute of Medicine (IOM) assess the progress of PEPFAR implementation to help guide the future directions of this innovative program. The findings and recommendations of that IOM study, published in 2007, informed PEPFAR processes, policies, and activi-
__________________
1United States Leadership Against HIV/AIDS, Tuberculosis, and Malaria Act of 2003, P.L. 108-25, 108th Cong., 1st sess. (May 27, 2003).
ties as well as the legislation that reauthorized the initiative, known as the Lantos-Hyde Act of 2008.2
The reauthorization legislation mandated that the IOM assess the performance of U.S.-assisted global HIV/AIDS programs and evaluate the impact on health of prevention, treatment, and care efforts supported by U.S. funding (see Appendix A for the statement of task). This report is intended to provide a rigorous, evidence-based, multidisciplinary, and independent evaluation of PEPFAR to Congress and the Department of State as well as to the scientific community, program implementers, policy makers, civil society, people living with and affected by HIV/AIDS, and international stakeholders in global public health.
In response to its mandate, IOM first convened a planning committee to develop a strategic approach for conducting the evaluation. This approach, published in a 2010 report, addressed the complexities of evaluating an initiative with the scale and diversity of programs that PEPFAR supports and with the range of countries in which it operates. The dynamism of an initiative that was operating and evolving over the course of the evaluation presented additional complexity.
To carry out the evaluation, the IOM convened a diverse expert committee that included considerable overlap with the members of the planning committee. Guided by the strategic approach, the committee, IOM staff, and consultants carried out a mixed-methods approach. The qualitative data that were collected included extensive document review and more than 400 semi-structured interviews conducted from 2010 to 2012. Each member of the committee visited at least one PEPFAR partner country, and in total the evaluation team conducted 13 data collection visits to partner countries, hearing the perspectives of a wide range of stakeholders. PEPFAR headquarters and mission staff, partner country stakeholders, and global partners all generously contributed their time and experience to the committee. Quantitative data included financial data, program and clinical monitoring data, and epidemiological information. The committee struggled to find quantitative data to address some of the elements of the statement of task. Beyond the specific issues of available data to address the legislated task, however, there is also the critical imperative that PEPFAR be able to determine the key questions to ask in order to assess its own performance and effectiveness and to plan in advance for the collection of meaningful data to answer those questions and guide the ongoing evolution of PEPFAR.
__________________
2 Tom Lantos and Henry J. Hyde United States Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008, P.L. 110-293, 110th Cong., 2nd sess. (July 30, 2008).
The 2008 reauthorization of PEPFAR emphasized that the program must transition from its initial goal of providing an emergency response to longer-term goals of enhancing sustainability, promoting country ownership, and strengthening health systems. One of the clear findings that emerged from this evaluation is that as PEPFAR evolves in this way, major dilemmas are emerging that create tensions for decision making related to a country’s HIV response; these dilemmas will require attention as the program moves forward. As the HIV response becomes more country-driven, PEPFAR—and any other external donor effort—will need to focus its contributions on national efforts rather than on the direct provision of services and attribution of results. This will have consequences for program planning, implementation, and evaluation. Furthermore, focusing on country ownership will require relinquishing some control over the response, which in turn will have unknown consequences for quality and access to services; PEPFAR and its partner countries will have to grapple with these issues together.
PEPFAR has been globally transformative—changing in many ways the paradigm of global health and what can be accomplished with ambitious goals, ample funding, and humanitarian commitment to a public health crisis. As it moves forward, PEPFAR must continue to be bold in its vision, implementation, and global leadership, but now toward its aims of continuing to strengthen the capacity of partner countries to respond to the pandemic. The committee hopes that this evaluation will serve as a tool to achieve these aims.
The committee extends its gratitude to all those who provided information to assist in the evaluation. The committee has continuing deep admiration for those carrying out the difficult work of responding to the pandemic. I was privileged to serve as the chair for both the planning committee and the evaluation committee. I would like to express my appreciation to the members of both committees for the expertise and perspective they contributed, for their robust participation in discourse and deliberation, and for the immeasurable time and energy they volunteered. The IOM committee staff, very ably led by study co-directors Bridget Kelly and Kimberly Scott, have been highly professional, thoughtful, and committed to ensuring the most responsive and rigorous evaluation possible. I thank the entire staff and the committee consultants for their tireless efforts in support of the committee.
Robert E. Black, Chair
Committee on the Outcome and Impact Evaluation of Global
HIV/AIDS Programs Implemented Under the Lantos-Hyde Act of 2008
AIDS |
acquired immune deficiency syndrome |
ANC |
antenatal care |
APR |
annual program results |
ART |
antiretroviral therapy |
ARV |
antiretroviral |
AZT |
zidovudine |
BCC |
behavior change communication |
BPE |
basic program evaluation |
CBO |
community-based organization |
CCM |
country coordinating mechanism |
CD4 |
cluster of differentiation 4 |
CDC |
U.S. Centers for Disease Control and Prevention |
CGD |
Center for Global Development |
CHERG |
Child Health Epidemiology Reference Group |
CHSW |
community health or para-social worker |
COP |
country operational plan |
COPRS |
Country Operational Plan Reporting System |
CPT |
cotrimoxazole preventive therapy |
CRC |
Committee on the Rights of the Child |
CSO |
civil society organization |
CTX |
cotrimoxazole |
DAH |
development assistance for health |
DHAP |
Division of HIV/AIDS Prevention (at CDC) |
DHS |
Demographic and Health Surveys |
DoD |
U.S. Department of Defense |
DoL |
U.S. Department of Labor |
DoS |
U.S. Department of State |
EA |
expenditure analysis |
EID |
early infant diagnosis of HIV |
FBO |
faith-based organization |
FELTP |
Field Epidemiology and Laboratory Training Program |
FETP |
Field Epidemiology Training Program |
FY |
fiscal year |
GAO |
U.S. Government Accountability Office |
GBV |
gender-based violence |
GHI |
U.S. Global Health Initiative |
Global Fund |
Global Fund to Fight AIDS, Tuberculosis, and Malaria |
GMS |
Grants Management Solutions |
GNI |
gross national income |
HAPSAT |
HIV/AIDS Program Sustainability Analysis Tool |
HHS |
U.S. Department of Health and Human Services |
HIPC |
heavily indebted poor country |
HIS |
health information system |
HIV |
human immunodeficiency virus |
HMIS |
health management information system |
HQ |
headquarters |
HRH |
human resources for health |
HRSA |
Health Resources and Services Administration |
HSS |
health systems strengthening |
IeDEA |
International Epidemiological Database to Evaluate AIDS |
IGA |
income-generating activity |
IOM |
Institute of Medicine |
IPT |
isoniazid preventive therapy |
IPTp |
intermittent preventive treatment of malaria for pregnant women |
ITN |
insecticide-treated net |
LIMS |
laboratory information management system |
LTFu |
loss to follow-up |
M&E |
monitoring and evaluation |
MAT |
medication-assisted treatment |
MCC |
Millennium Challenge Corporation |
MCH |
maternal and child health |
MDG |
Millennium Development Goal |
MEPI |
Medical Education Partnership Initiative |
MERG |
Monitoring and Evaluation Reference Group |
MICS |
Multiple Indicator Cluster Survey |
MOH |
ministry of health |
MSM |
men who have sex with men |
MTCT |
mother-to-child transmission |
NAC |
National AIDS Commission/Committee/Council/Control Agency |
NAS |
National Academies of Science |
NASA |
national AIDS spending assessment |
NDOH |
National Department of Health (South Africa) |
NEPI |
Nursing/Midwifery Education Partnership Initiative |
NGI |
next generation indicator |
NGO |
nongovernmental organization |
NHA |
national health account |
NIH |
U.S. National Institutes of Health |
NRC |
National Research Council |
NSF |
National Science Foundation |
OECD |
Organisation for Economic Co-operation and Development |
OGAC |
Office of the U.S. Global AIDS Coordinator |
OI |
opportunistic infection |
OMB |
Office of Management and Budget |
OVC |
orphans and vulnerable children |
PCR |
polymerase chain reaction |
PEP |
post-exposure prophylaxis |
PEPFAR |
The President’s Emergency Plan for AIDS Relief |
PEPFAR I |
The President’s Emergency Plan for AIDS Relief (2004–2008) |
PEPFAR II |
The President’s Emergency Plan for AIDS Relief (2009–2013) |
PEQ |
priority evaluation question |
PF |
Partnership Framework |
PFIP |
Partnership Framework implementation plan |
PHE |
public health evaluation |
PI |
principal investigator |
PICT |
provider-initiated counseling and testing |
PIP |
Program Impact Pathway |
PLHIV |
people living with HIV/AIDS |
PMI |
President’s Malaria Initiative |
PMTCT |
prevention of mother-to-child transmission |
PPP |
public–private partnership |
PrEP |
pre-exposure prophylaxis |
QA |
quality assurance |
QI |
quality improvement |
RFA |
request for application |
SAB |
Scientific Advisory Board (of PEPFAR) |
SAMHSA |
Substance Abuse and Mental Health Services |
Administration |
|
SANAC |
South African National AIDS Council |
SAPR |
semi-annual program results |
SCMS |
Supply Chain Management System |
SGBV |
sexual and gender-based violence |
SI |
strategic information |
SOPA |
State of the Program Area |
STD |
sexually transmitted disease |
STI |
sexually transmitted infection |
TA |
technical assistance |
TAB |
technical advisory board |
TB |
tuberculosis |
TDR |
transmitted drug resistance |
TE |
targeted evaluation |
TWG |
technical working group |
UN |
United Nations |
UNAIDS |
Joint United Nations Programme on HIV/AIDS |
UNGASS |
United Nations General Assembly Special Session |
UNICEF |
United Nations Children’s Fund |
UNODC |
United Nations Office on Drugs and Crime |
USAID |
United States Agency for International Development |
USG |
U.S. government |
VMMC |
voluntary medical male circumcision |
WHO |
World Health Organization |
INTERVIEW CITATION ABBREVIATIONS
Country Visit Exit Synthesis: Country # + ES
Country Visit Interview: Country # + Interview # + Organization Type
Non-Country Visit Interview: “NCV” + Interview # + Organization Type
Organization Types
USG |
U.S. government |
USNGO |
U.S. nongovernmental organization |
USPS |
U.S. private sector |
USACA |
U.S. academia |
PCGOV |
partner country government |
PCNGO |
partner country nongovernmental organization |
PCPS |
partner country private sector |
PCACA |
partner country academia |
CCM |
country coordinating mechanism |
ML |
multilateral organization |
OBL |
other (non-U.S. and non-partner country) bilateral |
OGOV |
other government |
ONGO |
other (non-U.S. and non-partner country) nongovernmental organization |