Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
PART III Illustrative Evaluation Details for Assessment of PEPFARâs Performance and Impact
The IOM was mandated by Congress in the LantosâHyde Act of 2008 to conduct a study that includes an assessment of the performance of U.S.-assisted global HIV/AIDS programs and an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by U.S. funding. Part of the charge to the planning committee in developing a plan for this evaluation was to be cognizant of the requirements and charges mandated for the evaluation (see Appendix A). To augment the overview of the evaluation design presented in Part II, this part of the plan partitions and elaborates the areas of interest laid out in the congressional mandate. The guiding framework of the program impact pathway is applied to each of these areas, reflecting the committeeâs understanding of the rationale for how PEPFARâs specific inputs and activities can be plausibly linked to PEPFARâs contribution to effects on HIV-specific health outcomes and impacts. This part of the report illustrates the types of questions that will guide the evaluation of PEPFARâs activities in prevention, adult and pediatric treatment, care and support, child and adolescent wellbeing (including orphans and vulnerable children), and gender-related vulnerability and risk activities. The evaluation will also consider other fundamental activities in the areas of knowledge management and funding flows; these are considered first in this part of the report because they underlie the success of all other programmatic areas. This part of the report culminates with a discussion of cross-cutting activities related to key systems-level goals that are critical for the long term goals articulated by PEPFAR, such as health systems strengthening and transitioning to sustainability and country ownership. As described in Part II, in each of these areas the evaluation questions will be addressed using a mixed methods approach and layers of investigation and analysis, drawing on a range of available primary and secondary data sources. By applying a mix of methods, data sources, and analytical techniques, the committee will arrive at findings that can be triangulated to draw conclusions about the performance of PEPFAR and its contribution to health impact, even when any one data source is not sufficient or any one methodological approach is not feasible. The extent to which specific methods can be applied to answer the evaluation questions will depend on the timely availability of data that is of sufficient quality to lead to reliable findings. Therefore, the illustrative questions and the methods and data sources that will be used to address them will undergo further refinement and prioritization as a result of the operational planning phase activities described previously. 51
SECTION 1: PEPFARâS KNOWLEDGE MANAGEMENT The management of knowledge and information is critical to the success of any program because it serves to monitor the activities and effects of the program as well as to guide policies, priorities, and programmatic decisions. Therefore, assessing the performance of PEPFARâs knowledge management activities will be an important part of the evaluation of the performance of PEPFAR as a whole, as well as an assessment of the forward-looking mechanisms that are in place for continuous M&E of the programâs progress and appropriate responses. In PEPFAR I, the primary goal of OGACâs strategic information (SI) efforts for M&E, in partnership with implementing agencies, was to support PEPFAR through results-based planning and implementation, focusing on quality information collection, timely data management and use, evaluation of best practices, and information for decision making. In PEPFAR II, the goals have been expanded to include support of the larger PEPFAR mission. To this end, the expanded SI mission includes building the capacity of countries to improve health outcomes by increasing and strengthening the use of information for effective stewardship of programs and effective implementation of efficient, high-quality, and sustainable health systems (Bouey, 2010). During PEPFAR II, the SI goals also include improved harmonization of USG reporting needs with country-driven M&E efforts through not only strengthening country capacity and alignment with national data collection, but also through better alignment with global reporting requirements to lessen the burden on implementing partners and partner governments (OGAC, 2009h). An increased focus on both program coverage and quality will be reflected in SI efforts to identify indicators that can give an accurate picture of these two areas (OGAC, 2009d). Finally, although it continues to recognize that PEPFAR is not intended to be a research initiative, the PEPFAR Five-Year Strategy outlines the additional goals of improving the programâs efforts to contribute to the evidence base for HIV interventions and to expand the amount of publicly available data (OGAC, 2009g). This expanded research effort will prioritize the evaluation and proactive dissemination of topics that PEPFAR is in a unique position to address as well as studies that focus on methods to improve program delivery (OGAC, 2009h). It will also increase the tracking of outcomes, cost-effectiveness, innovation, and impacts in order to identify timely information regarding the programâs effectiveness and impact (OGAC, 2009h). Strategic Information Management Structurally, SI activities at the PEPFAR headquarters level are carried out by USG implementing agencies with coordination through OGAC. These headquarters-level activities draw from a wide range of data-gathering sources (see Figure 5). 53
54 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS FIGURE 5 PEPFAR headquarters-level strategic information partners and headquarters-level data- gathering sources. NOTES: CDC = U.S. Centers for Disease Control and Prevention, Census = U.S. Census Bureau, DoD = U.S. Department of Defense, HRSA = Health Resources and Services Administration of the U.S. Department of Health and Human Services, M&E = monitoring and evaluation, OGAC = Office of the U.S. Global AIDS Coordinator, SI = strategic information, TWG = technical working group, USAID = United States Agency for International Development. SOURCE: Adapted from Bouey (2010). The staff at headquarters are responsible for issuing guidance related to COP submission and reporting processes, as well as for providing technical assistance. This guidance instructs country teams on how to successfully complete their reporting requirements. In addition to annual reporting needs, guidance from headquarters can also provide information on collecting, interpreting, and updating basic epidemiologic profiles as well as information on how to develop and incorporate efforts to evaluate new initiatives (OGAC, 2009e). While this guidance focuses primarily on the processes unique to OGAC, it also offers additional information regarding data collection and target setting that has the potential for broader applicability. Within headquarters there is also a SI technical working group. These efforts at the headquarters level also support M&E activities at the country level through the development of resources such as an M&E Systems Strengthening Tool, which is designed to help partner countries prioritize their M&E needs and encourage alignment with a national M&E strategy, and a Data Quality Assessment Tool (MEASURE Evaluation, 2007; PEPFAR, 2008a). While management of data collection, storage, and analysis at the country and project level varies, the primary mechanism for reporting and aggregating these data is via an electronic, Internet-based program known as the Country Operational Plan Reporting System (COPRS). Data that are reported via COPRS are collected at the OGAC headquarters level during the relevant semi-annual or annual program reporting periods, depending on the country (OGAC, 2009d). A portion of these data is released to the public and is also communicated to Congress via an annual report. Occasionally, they are used by OGAC to produce additional topic-specific reports (e.g., Report on Gender-Based Violence and HIV/AIDS) (PEPFAR, 2010c). COPRS is
55 PEPFARâS KNOWLEDGE MANAGEMENT currently in transition due to two factors. First, the Next Generation Indicators Reference Guidance was recently released, introducing a limited number of new indicators and redefining some measures that had previously been in use. This guidance was developed in part to support PEPFARâs contribution to global efforts to harmonize reporting requirements for HIV/AIDS initiatives, which aims to reduce the reporting burden of program implementers and to allow more flexibility and increased local ownership of the design of M&E plans (OGAC, 2009d). Second, a new generation of COPRS, COPRS II, is in development and is expected to be deployed in FY2010 (OGAC, 2009e). Beyond Information Management In 2008, PEPFAR began a campaign titled âknow your epidemic/know your resultsâ aimed at using information to more closely align program activities with population needs (OGAC, 2008b). As a result, a focus was placed on developing sustainable SI systems to âcollect, analyze, critically review, disseminate, interpret, display, and strategically use data at all levelsâ (OGAC, 2008b). Continuing with this development of SI, the 2009 headquarters operational plan allocated funds for the development of a âdraft knowledge management strategy,â perhaps in response to the first IOM PEPFAR evaluation recommendation to develop a detailed, overall strategy for institutionalizing its efforts to function as a learning organization and to increase its contributions to the global knowledge base (IOM, 2007; OGAC, 2009f). Coordination among country staff and dissemination of best practices is also facilitated by the PEPFAR implementersâ meeting held annually in a PEPFAR country. In addition to sharing information across countries, this meeting includes a variety of breakout sessions dedicated to SI issues (PEPFAR, 2009a). In addition, as described above, the recent goals for PEPFAR II emphasized the important role of expanding the programâs research portfolio to contribute to the publicly available evidence base, with an emphasis on operations research to improve program delivery as well as methods for timely assessments of the programâs effectiveness and impact (OGAC, 2009h). Some research activities are already occurring in individual partner countries. For example, the Public Health Evaluations (PHE), initiated in 2007, are a PEPFAR activity intended as a source to inform policy and program-level changes. They currently serve as the primary mechanism through which PEPFAR supports research within countries, including operations research (Edgil, 2010; OGAC, 2009h). Some PHEs are single country, while others are multi-site investigations. The selection of PHE proposals (annually solicited from investigators) is performed by an interagency technical policy group charged with prioritizing areas in need of evaluation, overseeing the implementation of evaluations, and recommending approvals and levels of funding for PHEs. In doing so, priority is given to studies that are driven by locally- identified country needs as well as those that involve local institutions and investigators in the research process. 15 As of 2008 there were 195 PHE activities (Edgil, 2010), with the most recent call for proposals issued by the National Institutes of Health in April 2010. 16 15 Personal communication from OGAC, April 9, 2010. 16 Ibid.
56 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS PEPFARâs Monitoring and Evaluation Framework To evaluate PEPFARâs performance in the area of knowledge management, the committee will be guided by the public health questions approach (see Figure 6), a framework that is widely used in the global HIV/AIDS M&E community and has been adopted by OGAC (Bouey, 2010; Rugg et al., 2004; UNAIDS MERG, 2010). The committee will determine the extent to which M&E activities are meeting the goals laid out in this framework and the extent to which these activities are contributing to evaluating and improving the performance of the program and building the capacity of partner countries to use information to improve health outcomes. FIGURE 6 A public health questions approach to HIV monitoring and evaluation. SOURCE: Reprinted, with permission, from UNAIDS MERG (2010). Illustrative Questions The evaluation of PEPFARâs SI activities will be carried out at the level of headquarters and in those partner countries where PEPFAR has made major investment in M&E. In order to accomplish the assessment, the committee will use a desk review of OGAC and country-level M&E strategy and implementation procedures and data management systems since the beginning of PEPFAR (i.e., including different iterations over time). The committee will also examine M&E funding allocations and expenditures at global and country levels where available. A review of national M&E strategies, national M&E assessment reports, and reports on the HIV epidemic and response, including global reports such as United Nations General Assembly Special Session (UNGASS) documents or M&E data from UNAIDSâs National Composite Policy Index, will also provide context for the evaluation. Due to the limitations in assessing the progress of efforts over time from guidance and assessment documents, the committee will also rely on structured interviews with key SI and programmatic PEPFAR staff at all levels (OGAC, USG agencies, OGAC SI technical working groups, country staff, contractors/implementers) regarding the mechanisms and role of SI in the PEPFAR program and in informing the national HIV response. The committee may also explore structured interviews with key M&E and programmatic staff of each countryâs national AIDS program as well as other multilateral and bilateral organizations (including UNAIDS, WHO, the
57 PEPFARâS KNOWLEDGE MANAGEMENT Global Fund) about the role of PEPFAR as a partner at the local, national, and global levels. These interviews could be incorporated as a part of the committeeâs country visits or conducted by phone or video conference (or alternative self-completed questionnaire) for those countries where a country visit is not planned. The committeeâs analysis of the completeness and validity of data requested from OGAC and implementing partners for all areas of this evaluation will also inform an assessment of SI and M&E performance and progress. In order to assess PEPFARâs contribution to the global knowledge base, the committee will assess PEPFARâs participation in international M&E development processes. The committee will also assess current research efforts, such as the PHEs, and associated dissemination efforts, such as presentations from PEPFAR programs at the annual implementersâ meeting and other international conferences like the meetings of the International AIDS Society. In addition, a preliminary search of published literature conducted during this planning phase will provide the foundation for a more extensive review during the evaluation of available articles, reports, and other publications resulting from PEPFAR-funded activities. The following are examples of illustrative questions that the committee may consider in the evaluation. These questions related to knowledge management reflect a fundamental activity of the program and as such are intended to contribute to addressing all of the areas for consideration in the congressional mandate, as described in the Statement of Task (see Appendix A). To what extent has investment in M&E resulted in effective systems for PEPFAR decision making and for program management and improvement at both the headquarters and country levels? Is data collection and analysis being used for decision making about PEPFAR program priorities, implementation strategies, effectiveness, and efficiency? For example, is PEPFAR using data to support evidence-based COP planning and resource allocation? Does data collection lead to timely identification of implementation problems, and does this result in corrective action? Does data on targets determine whether programs are implemented on a large enough scale to have an impact on the epidemic? Are the data collated, analyzed, interpreted, presented, and disseminated in a manner that allows for use in decision making? What are the mechanisms used to assure the validity and quality of data? What mechanisms are in place to facilitate the translation of information produced by M&E systems into action? Are lessons learned accessible and are changes applied across the whole of the program where appropriate? Have operations research and other research activities supported by PEPFAR, such as PHEs, had an impact on service delivery and led to improved outcomes of prevention, treatment, and care programs supported by PEPFAR? Are operations and other research activities using appropriate methodologies and resulting in information that is shared across sites, programs, and countries to optimize and inform policy and program decisions? Are these research activities addressing the issues most in need of evaluation? What should the priorities be for future PHEs and other research activities?
58 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS What has been the impact of PEPFAR reporting requirements for accountability purposes (i.e., reporting to the U.S. Congress) on continuation of PEPFAR funding and on decision making for priority investments as well as program management and improvement? What progress has been made on PEPFARâs intentions to develop indicators where there are currently limited mechanisms for tracking progress, such as gender and health systems strengthening? Are there sufficient M&E mechanisms and capacity to evaluate whether the program meets new goals for sustainability and country ownership, as well as the resulting outcomes and impact of changes made to address those goals? Does OGAC draw on sources of data outside of PEPFAR to inform programmatic and policy decisions? To what extent is PEPFAR contributing to the global knowledge base? Are PEPFAR-funded activities resulting in research that is contributing to the scientific knowledge base? Are research findings, lessons learned, and best practices from PEPFAR available in the published literature? What other mechanisms are used to disseminate knowledge not only within but also beyond PEPFAR? To what extent is that process encouraged or facilitated? To what extent is PEPFAR engaging with other international stakeholders around SI activities? To what extent is PEPFAR contributing to the development of state-of-the- art practices in M&E at the global level? To what extent has PEPFAR built/is PEPFAR building capacity at the country level, including national M&E systems, 17 to support an appropriate, effective, and efficient national HIV response? What is PEPFARâs approach to supporting long-term sustainability of the national M&E system? How is PEPFAR translating the commitment of the United States to the âThird Oneâ adopted by most donorsâone national M&E system to reduce reporting burden and transactional costs of business for countries with multiple donors? To what extent is the PEPFAR M&E strategy aligned with and incorporated in the national M&E strategy/plan? What are the positive and negative effects of the headquarters-level PEPFAR M&E strategy on national M&E systems? To what extent and how are PEPFAR M&E data (program planning, routine program 17 National M&E system refers to M&E at the national, sub-national, and service-delivery levels.
59 PEPFARâS KNOWLEDGE MANAGEMENT monitoring, findings from special studies) shared with the national M&E system to ensure a coordinated HIV response and to guide program improvement? What mechanisms are used by PEPFAR for M&E capacity building and to ensure effective partnerships for technical cooperation and technology transfer? What are the effects of the PEPFAR M&E capacity building activities on national M&E system strengthening and data use for decision making? To what extent has PEPFAR built/is PEPFAR building M&E capacity within partner organizations implementing programs at the country level, including data analysis and management?
SECTION 2: MAPPING PEPFAR FUNDING All donor-specific impact evaluations are limited by the ability to attribute desired impacts to the investments of a single program given the presence of similar and complementary programs funded by other sources, so the committeeâs evaluation will focus on PEPFARâs contribution to global efforts to fight HIV/AIDS (IOM, 2008). PEPFAR inputs include policy guidance and regulations, personnel, technical assistance and training, facilities and equipment, knowledge transfer and research, and funding. While all are important to assessing PEPFARâs impact, the level and allocation of funding, as well as how the funding is used, underpins much of the evaluation. It both represents the most direct measure of the USGâs investment to address the global AIDS pandemic and provides a critical input needed for answering many of the other evaluation questions pertaining to the impact of PEPFAR. These include questions about PEPFARâs impact on specific health targets, such as reducing HIV incidence, and its impact on broader mandates and goals, such as promoting long-term sustainability, country ownership, and health systems strengthening. As such, financing is a cross-cutting aspect of the overall evaluation, embedded within each of the other main evaluation questions. To evaluate the impact of PEPFARâs financial investments on desired program impacts, the committee will develop a funding flow framework for PEPFAR financing, designed to map funds throughout the life cycle of the program. It will begin with Congress, which appropriates funding to federal agencies, and follow funding provided by federal agencies to prime partners and other implementers in an attempt to map all the way to service providers in the field and ultimately to beneficiaries. The objective of this financial framework is not to conduct a financial audit but to assess the role of U.S. funding in the context of the overall evaluation of PEPFARâs impact on health and other outcomes and to illustrate the specific points at which the USG intervenes, how it intervenes, and the level of its intervention. Figure 7 represents the committeeâs initial understanding of the landscape of PEPFAR funding flows, which may change as more information is accumulated throughout the evaluation. 61
62 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS FIGURE 7 PEPFAR funding landscape. NOTES: Dept. = department, Govt = government, HHS = U.S. Department of Human Health and Services, NGOs = non-governmental organizations, USAID = U.S. Agency for International Development, USG = U.S. government. SOURCE: Committee assessment based on a review of documents from OGAC, including PEPFAR operational plans and partner information, as well as other readily available sources.
63 MAPPING PEPFAR FUNDING The evaluation will also situate PEPFAR investments within the larger landscape of investments made by other funders to allow for an assessment of PEPFARâs relative financial contributions to global efforts to combat HIV/AIDS. This is important because nearly all of the countries that receive PEPFAR funding also receive funding for HIV/AIDS programs from other sources (see Table 1 for an illustration of the current funding landscape in PEPFAR countries). The committee will therefore develop another financial framework to situate PEPFAR within the broader funding landscape, designed to capture PEPFARâs relative contribution globally and by country. It will include other donor governments, multilateral institutions, partner country governments and households, and the private sector.
64 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS TABLE 1 Current HIV/AIDS Funding from Select Donors Received by PEPFAR Countries (as of April 2010) Multilateral Bilateral Country Global Fund World Bank CIDA DFID Angola X X X Botswana Cambodia X X X China X X Congo, Democratic Republic of the X X X Côte dâIvoire X X X Dominican Republic X X Ethiopia X X X X Ghana X X X X Guyana X X Haiti X X India X X X X Indonesia X X Kenya X X X X Lesotho X X X Malawi X X X X Mozambique X X X X Namibia X X Nigeria X X X Russia X X Rwanda X X X X South Africa X X X Sudan X X X Swaziland X X X Tanzania X X X X Thailand X Uganda X X X X Ukraine X X Vietnam X X X Zambia X X X Zimbabwe X X X NOTE: An X denotes that a country receives or has recently received approval for an HIV/AIDS grant from the Global Fund, or has at least one currently operational HIV/AIDS program funded by the World Bank (excluding IBRD-financed loans), the Canadian International Development Agency (CIDA), or the UK Department for International Development (DFID). CIDA and DFID are representative of other bilateral funders of global HIV/AIDS programming, and the UK in particular is the second largest bilateral donor to HIV/AIDS (Kates et al., 2009). SOURCE: Compiled from GFATM (2010a), World Bank (2010), CIDA (2010), and DFID (2010).
65 MAPPING PEPFAR FUNDING Finally, since the data used to populate the financial framework are critical inputs for each program area, the committee will attempt, where feasible, to use these data to answer questions surrounding program efficiency, economies of scale, costs of intermediate units of analysis (e.g., the unit cost of outputs, the costs per beneficiary), and costs per intermediate outcomes (e.g., school enrollment, change in condom use, persons on treatment, and child infections averted from PMTCT, etc.). Ultimately, the framework proposed here will be designed to provide policymakers with a direct measure of PEPFARâs impact on the global HIV/AIDS epidemic. Illustrative Questions The committee will consult a variety of data sources including PEPFAR and USG agencies (e.g., OGAC databases, COPs, partnership frameworks, prime and sub-prime partner reports, etc.), other donors and international stakeholders (UNAIDS, the World Bank, the Global Fund, Organisation for Economic Co-operation and Development [OECD], etc.), partner countries, and the relevant evaluation literature. To identify data sources and availability, the committee will need to conduct structured interviews with relevant stakeholders including USG representatives, implementers, partners, and others. Data collection and quality will be a primary potential limitation since much of the analysis is predicated on being able to obtain timely data from these sources. The complexities of mapping financing flows from multiple donors to multiple countries will also complicate the ability to access consistent and complete longitudinal data. Given the timely availability of quality data, the following are examples of illustrative questions that are feasible to address. These questions related to the flow of funding reflect a fundamental aspect of program operations and as such are intended to contribute to addressing all of the areas for consideration in the congressional mandate, as described in the Statement of Task (see Appendix A). How much PEPFAR funding is provided and what is its relative contribution to global efforts against HIV/AIDS? Where possible, the committee will use longitudinal data to analyze PEPFARâs relative contribution over time and determine whether PEPFAR has been able to leverage additional resources from other donors and national governments. To further describe the contribution of PEPFAR funding, the committee will attempt to determine the proportion of the national budget and the gross domestic product that would need to be spent on health and HIV/AIDS if partner governments were to take on these expenditures. How is PEPFAR funding provided to partner countries and USG partners? Initially, the committee will analyze COPs and partnership frameworks to identify a general mapping of funding flows and seek to characterize countries according to a sub-set of different types of models or frameworks. The committee aims to determine whether PEPFAR funding is provided through bilateral or multilateral channels and which mechanisms are used to deliver funding, such as cash transfers or commodities. To further describe funding cascades, the committee will attempt an analysis of PEPFARâs disbursement procedures to determine how
66 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS quickly funding is provided to the field once it has been appropriated by Congress, obligated to countries, and disbursed to partners. Who are the recipients of PEPFAR funding at each stage of program? As depicted in Figure 8, PEPFAR channels funding through governments, NGOs, academic institutions, and private contractors that may be based in the United States, in partner countries, or in other countries. The committee will seek to analyze the proportion of funding that flows to each type of recipient and analyze longitudinal trends. A preliminary committee analysis of FY2008 obligations to prime partners identified three main sub-types when stratified by geographic location and five sub-types when stratified by geographic location and organizational type, as follows (listed in order of share of investment): Prime Partner Funding Flow Sub-Types by Geographic Location 1. United States 2. Domestic/Partner Country 3. Other/International Prime Partner Funding Flow Sub-Types by Geographic Location and Organizational Type 1. U.S. NGO 2. U.S. Academic 3. Domestic/Partner Country NGO 4. Domestic/Partner Country Government 5. U.S. for Profit Firms The relative distribution of funds by agency origin (United States versus partner country) and by sector (government, NGO, academia, private sector) over time also will be assessed. Understanding these patterns may help elucidate the degree to which the targeting of funding supports, or potentially undercuts, the goal of increasing country ownership of HIV programming. The criteria for providing grants to local implementing partners and how they are monitored may also contribute to an assessment of whether this process contributes to the goals of increasing country ownership. In addition, the performance of the prime partner organizations in terms of accountability, administrative transparency, and good governance could be additional metrics to consider in the evaluation. Finally, assessing the distribution of funding in this way may help to provide data and information on the amount of funding that actually reaches the field after accounting for intermediaries along the way.
67 MAPPING PEPFAR FUNDING FIGURE 8 PEPFAR FY2008 obligations to prime partners: U.S., local, or other location stratified by key sector. NOTES: Totals do not equal 100 percent due to rounding. Govt = government, Intl = international, NGOs = non-governmental organization, Orgs = organizations. SOURCE: Preliminary committee analysis based on FY2008 OGAC data on obligations to prime partners available at http://www.pepfar.gov/partners/index.htm. How are PEPFAR funds distributed among different programmatic areas and interventions? How are funds from PEPFAR and the Global Fund meshed together in countries? Are the joint funds maximized for service delivery and coverage? The committee will seek to measure the proportion of funding spent in each program area, such as prevention, treatment, care and support, and health systems. Within each program area, the committee will attempt to identify the level of funding for specific interventions and analyze PEPFARâs efforts to allocate funding appropriately based on country needs and available evidence. In order to analyze whether efficiencies are being achieved, the committee
68 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS will attempt to determine the unit costs of key PEPFAR interventions and how they vary by type of intervention, country, and other variables of interest. Which populations are supported through PEPFAR funding? In order to evaluate the impact of PEPFAR funding, the committee will seek to determine the specific populations, such as women, children, and other vulnerable populations, reached with program interventions. The committee will use existing data from reliable sources to measure the number of people living with HIV/AIDS (PLWHA) and compare to the number of vulnerable people reached through PEPFAR programming. The number and types of populations supported by PEPFAR funding will be measured through quantitative analysis of PEPFAR indicators. Aggregated data reported to OGAC provide limited insight about the types of populations supported, so the committee will seek disaggregated data from other sources.
SECTION 3: PREVENTION SERVICES It has been estimated that a new HIV infection occurs globally every 12 seconds, and there is agreement that in most countries incident infections are outpacing the number of people placed on ART (Global HIV Prevention Working Group, 2010; UNAIDS and WHO, 2009; Zachariah et al., 2010). At the same time, HIV prevention budgets have been falling in several countries. Currently there is an increasing push by the global community to scale up effective, evidence-based prevention strategies with the knowledge that averting new infections is the only way to bend the curve of the epidemic. Information about new behavioral, biomedical, and structural prevention approaches is growing and individual countries, regions, or localities are being encouraged to incorporate all of the approaches into a comprehensive strategy. PEPFAR is collaborating with WHO and UNAIDS in these efforts and the programâs prevention goals, if attained, are intended to help address these deficits. This section describes the current state of the programâs prevention efforts and some of the potential questions and challenges under consideration by the committee to evaluate the extent to which PEPFAR I and II have met their prevention targets and goals. In addition to PEPFAR Iâs original prevention performance target, there was also the country target of reducing expected HIV incidence by 50 percent (IOM, 2007). In PEPFAR I, the prevention target was to be achieved through five funding and reporting subcategories: Abstinence/be faithful, Condoms and other prevention (e.g., IDU, military forces, street children), PMTCT, Blood safety, and Injection safety (IOM, 2007). The prevention activities undertaken that correspond to those funding and reporting categories were: promotion of behavior change aimed at risk avoidance and reduction, comprehensive programs for people who engage in high-risk behavior, PMTCT services, and reduction of medical transmission of HIV by ensuring safe blood supplies (OGAC, 2004). PEPFAR II promotes a transition to increasingly country-owned HIV programs that adequately address their HIV epidemics over time, including expanding prevention in both concentrated and generalized epidemics. The are several targets for PEPFAR II: to support the prevention of more than 12 million infections over the course of the entire program (since 2003), 18 to ensure that every partner country with a generalized epidemic has both 80 percent coverage of testing for pregnant women at the national level19 and 85 percent coverage of ARV drug prophylaxis and treatment of women found to be HIV-infected (OGAC, 2009g), to double the number of at-risk babies born HIV-free from the 240,000 babies of HIV-positive mothers who were born HIV-negative during the first five years of PEPFAR (OGAC, 2009g), and lastly to provide 100 percent of youth in PEPFAR prevention programs, in every partner country with a generalized epidemic, with comprehensive and correct knowledge of the ways HIV is transmitted and ways to protect themselves, consistent with Millennium Development Goals (MDGs) indicators in this area (OGAC, 2009g). The LantosâHyde Act of 2008 requires the Coordinator to establish a balanced HIV sexual transmission prevention strategy to govern expenditures for prevention activities in countries with generalized epidemics. This âbalanced fundingâ directive replaces the âone-third abstinence or be faithfulâ budget earmark requirement in the original 2003 legislation. Instead of identifying a specific requirement for the distribution of sexual prevention funds, countries are 18 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(i). 19 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iv). 69
70 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS now required to provide a compelling explanation, justified by the Coordinator, if less than 50 percent of this funding is directed toward activities promoting (a) abstinence, (b) delay of sexual debut, (c) monogamy, (d) fidelity, and (e) partner reduction. The Coordinator is also required to annually publish, and make available to the public, a report that details the progress toward implementation of this prevention strategy (OGAC, 2009e). Similarly, research and dissemination of best practices in prevention methodology have also been identified as priorities for PEPFAR II (OGAC, 2009i). In addition to the performance targets, PEPFAR II has also established additional overarching goals for prevention, such as strengthening partner country capacity to generate and use timely, accurate, and up-to-date epidemiological information (emphasizing prevention efforts targeted toward most vulnerable populations); encouraging use of innovative strategies for data mapping and service provision; and developing goals for each individual prevention category (OGAC, 2009h). Objective and Scope Evaluation of PEPFARâs efforts toward prevention and assessment of any impacts they may have had are essential to understanding if PEPFAR is meeting its targets of averting more than 12 million new infections. The committee will evaluate the extent to which PEPFAR is effectively engaging in scale-up of combinations of evidence-based HIV prevention intervention. The categories of PEPFAR prevention interventions are shown in Table 2.
71 PREVENTION SERVICES TABLE 2 Categories of PEPFAR Prevention Interventions Behavioral Structural Biomedical Administration of antiretroviral Improving awareness Increasing access to Prevention of drugs during pregnancy, at birth HIV testing and Mother To Child and acceptability of and breastfeeding antiretroviral drugs Transmission HIV testing and antiretroviral drug use during pregnancy in pregnancy Screening blood prior to transfusion Blood Safety Education about (in) Developing a national appropriate use of transfusion policy transfusion Ensuring continual access to blood screening supplies Injecting Drug Youth education to Increasing access to Methadone and buprenorphine Use reduce injecting drug drug treatment services use initiation or clean needles Sex education in Male circumcision Sexual Counseling to reduce schools to delay sexual concurrency, debut inconsistent condom use, HIV education for youth Incorporating trained Testing partners to detect Prevention in Counseling to counselors in treatment discordancy and identification and People Living increase condom use centers treatment of sexually transmitted with HIV/AIDS and reduce infections and opportunistic concurrency infections NOTE: Needle or syringe exchange programs (NSPs) have been shown to be an evidence-based method to reduce HIV and other blood-borne pathogens (IOM, 2006). The ban on domestic funding for NSPs was lifted by the U.S. Congress as part of the FY2010 appropriations process. No similar ban was codified for international assistance accounts (Consolidated Appropriations Act, 2010, P.L. 3288-2, 111th Congress, 1st Session, [January 6, 2009]). However, âforeign affairs programs, including PEPFAR, have traditionally followed the domestic policy guidance regarding NSPsâ (Personal communication from OGAC, January 8, 2010). OGAC reports that this lifted ban creates an opportunity for PEPFARâs MARP TWG to explore ways to support NSPs as part of a comprehensive package of services for IDUs and to develop a revised set of comprehensive guidance for PEPFAR programs and teams so that NSPs can augment prevention, treatment, and care activities already provided to this population (Personal communication from OGAC, January 8, 2010). Gateway Activities Targeting Prevention That Will Be Examined and Used For Analysis In addition to specific interventions, there are additional gateway activities related to PEPFARâs prevention efforts. These gateway activities do not, in and of themselves, reduce the number of new infections, but are critical to targeting prevention services and monitoring the effects and impact of interventions on the prevention goal of reducing new infections. They include HIV testing and counseling in both the community and clinic settings and surveillance activities in general and sub-populations.
72 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Challenges in Prevention Implementation Implementation of programs for HIV prevention face many challenges related to context, culture, and information. The gateway activity of surveillance and the use of spatial mapping tools (e.g. Geographical Information Systems) to describe or visually represent epidemics are critically important, as many countries do not have a clear picture of their epidemic at the national, regional, or local levels. While a sufficient understanding of the nature of the epidemic can more effectively guide programmatic decisions to target appropriate population groups, in some countries multiple types of epidemics are occurring simultaneously both within and across different regions. This poses challenges during the development of an HIV prevention strategic plan and requires more complicated combinations of activities that increase the number of logistic and coordination barriers to be managed. Finally, varying types of stigma and discrimination can restrict the ability of prevention programs to reach at-risk and vulnerable populations such as IDUs, CSWs, and women and girls. Each of these issues provide context when evaluating the areas of success and failure in PEPFARâs prevention efforts (OGAC, 2009i). Program Impact Pathway PEPFARâs prevention efforts will be assessed in accordance with a program impact pathway framework, as shown in Figure 9. The evaluation will draw on a variety of data sources, including outputs and outcomes as measured by PEPFAR countries and OGAC. For estimating the impact measure of incidence, the committee will consider emerging globally accepted proxy measures, such as prevalence in young people, as well U.S. Census Bureau and other modeling efforts. The committee may also use available data to consult with experts who model infections averted for PEPFAR in particular. Since prevalence is affected by incidence, duration (longevity should increase with more people getting on ART), and population growth, the committee will be interested to know how the modelers will parse out these relative constituents. While there is concern about relying too heavily on these statistical forecasts, the committee feels that models have utility. They will be examined taking into account their limitations, which are primarily determined by the quality of the data available to do the modeling. The evaluation may also assess the contribution that PEPFAR funds made to achieve impact at the country level by undertaking a trend analysis of incidence figures over time and comparing that to key policy or program or funding events that occurred over the same time period.
FIGURE 9 Program impact pathway for evaluation of PEPFARâs prevention programs. NOTES: ARVs = antiretroviral drugs, IDU = injecting drug user, PMTCT = prevention of mother-to-child transmission. 73
74 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions To assess whether both PEPFAR I and II have achieved their prevention goals, the committee will not only evaluate whether they met their quantitative targets, but also assess whether in doing so, they provided quality services, met the needs of the countries in which programs were implemented, and met their own overarching goals (coordination, integration, health system strengthening, and building capacity) as articulated in the strategy for PEPFAR II (OGAC, 2009g). The evaluation of PEPFARâs prevention efforts, outputs, and outcomes is subject to a variety of limitations including the quality and timely availability of quality data. In addition, as a part of OGACâs Next Generation Indicators project, many of the indicators centrally collected by OGAC that are applicable to prevention efforts were modified or revised at the end of 2009, which may affect the ability of the committee to assess temporal trends or compare original PEPFAR countries with those added more recently. Each individual topic also contains its own challenges and limitations that may limit the committeeâs analysis; these are discussed below. The following illustrative questions are intended to contribute to addressing part A, item ii and part B, items i and v of the areas for consideration in the congressional mandate.20 Are prevention strategies for a country defined based on an epidemiological analysis and evidence of what works? Has PEPFAR supported routine behavioral and epidemiological surveillance at the country level to identify sub-national variations in at-risk groups and used these data to target prevention interventions to appropriate groups (most-at-risk populations, other vulnerable populations, youth and adolescents, adults)? Is PEPFAR appropriately and efficiently incorporating emerging evidence on prevention techniques into its programmatic guidance? Surveillance strategies can include both biomedical and behavioral measures to determine disease prevalence (using HIV testing data including antenatal and at-risk populations), seroincidence (using âdetunedâ assays such as BED 21 in population surveys), and risk behavior prevalence (via Demographic and Health Surveys [DHS] or âsecond generationâ surveys). The frequency with which these types of data-gathering methods occur, the methods through which PEPFAR supports their implementation, and the dissemination and use of the data obtained will be assessed by the committee. This will be done by relying on publicly available data sets from DHS and other population surveys as well as qualitative data gathered through site visit interviews. PEPFAR indicators and programmatic guidance related to prevention efforts will be coupled with data from other sources on the nature of the national epidemics and on internationally-accepted best practices for HIV prevention. Studies on the efficacy of new 20 (A)(ii) an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by United States funding, including multilateral and bilateral programs involving joint operations (B)(i) an assessment of progress toward prevention, treatment, and care targets (B)(v) an evaluation of the impact of prevention programs on HIV incidence in relevant population groups 21 Supra., note 12.
75 PREVENTION SERVICES approaches to HIV prevention such as pre-exposure prophylaxis (PrEP), microbicides, and vaccine are under way, therefore these types of interventions would not be part of this evaluation (OGAC, 2009i). How does the distribution of funding for prevention activities compare to the epidemiology of the local epidemic? What is PEPFARâs progress toward the development and implementation of a prevention strategy and balanced funding portfolio? How is prevention funding, for countries and OGAC, aligned with elements of the prevention strategy? As a backdrop to the evaluation of outcomes and impact, the evaluation may also determine the percentage of funds spent by PEPFAR I and II (by year) on prevention overall, by subcategories, by region, and within countries. The subcategory expenditures can be examined by the committee to determine whether they match with the estimated populations at risk in the country to provide an assessment of the countryâs ability or willingness to adhere to the guidance of OGAC in âknow your epidemic/know your resultsâ and to fund correspondingly appropriate activities. This new guidance from OGAC has resulted in headquarter-suggested realignment of funds in the country when there are significant mismatches (Moloney-Kitts, 2009). In addition, the committee will attempt to determine the funding for each region and country by the prevalence by year and by sub-population. Expenditure studies should also compare the level of investment in prevention activities and the types of programs supported by country and by implementing partners, as well as whether these factors are associated with achievement of prevention objectives. Given the sensitivity surrounding programmatic financial data, the committee anticipates there will be difficulty regarding availability for a complete assessment. The committee will seek whatever data and information are available from OGAC to assess the progress on the development and implementation of the described prevention strategy and its programmatic and budgetary elements, as well as the interaction between this legislative guidance and PEPFARâs stated goal of mapping each countryâs epidemic and aligning prevention responses to the identified needs (OGAC, 2009g). Within each country, have HIV prevention programs reached the performance targets for prevention for both PEPFAR I and II? Is coverage of each service or a combination of services expanding over time? Progress toward the targets previously outlined will be assessed primarily using aggregated indicators that are reported to OGAC. These are available for the majority of the targets. However, for some areas, such as behavior change communication (BCC), additional PEPFAR indicators not reported to OGAC will also be required in order to address the question more completely. The committee will also attempt to assess PEPFARâs progress toward its stated goals in each of the specific prevention areas. For many of these goals for which no reported indicators exists at the aggregated headquarters level at OGAC, the committee will focus on qualitative primary data collection during site visits and will seek out additional quantitative data where available. Has the number of people who have been counseled and tested and received their results increased over time?
76 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Have counseling and testing services been offered in an equitable way to women (pregnant and non-pregnant) and men, to young people as well as adults, and to locally- identified vulnerable populations? To address PEPFARâs efforts toward expanding its counseling and testing services, the committee will analyze the reported PEPFAR indicators regarding the number of people who receive counseling and testing services and their results. While this indicator is disaggregated by sex, it may be insufficient to identify if programs are reaching all of the at-risk populations in need or if counseling and testing programs are successfully linking with intervention facilities. Where this is the case, the committee will pursue other sources of quantitative and qualitative data. What are the effects of the behavioral interventions (education and awareness, condom education and distribution, peer education, sex and HIV prevention education, group and individual behavioral interventions, etc.) that are being implemented? BCC programs occur in multiple settings and may be targeted to general population participants or specific at-risk groups. The committee will use indicators aggregated and reported to OGAC regarding number and types of individuals reached with BCC programs, as well as the disaggregation of these indicators where available. These indicators cover the range and frequency of behavior change programs, but do not sufficiently address the effectiveness of these programs with respect to whether the participants actually changed their behavior following the activity. To try to look longitudinally at prevalence as a reflection of the outcome of behavior change efforts, the committee will examine the utility of data from the antenatal population. Potential data sources include large international NGOs that have programs in many PEPFAR countries and therefore have very large cohorts across multiple countries. Some data may also be obtained through consultation with modelers. However, in general the committee will have limited ability to address this effectiveness issue. What has been the coverage and outcome or impact of biomedical prevention interventions, which include blood safety, PMTCT, PEP, and male circumcision? How has PMTCT been maximized with jointly operated activities between PEPFAR and the Global Fund? How has coverage expanded as result of joint efforts? Implementation and tracking of biomedical prevention programs varies significantly and targets a broad range of individuals at all ages. Data for the evaluation of biomedical prevention interventions will be obtained from aggregated OGAC level indicators and other sources at the country and local levels, as there are not reported PEPFAR indicators for blood safety, PEP, and male circumcision. Challenges associated with evaluating these interventions also include emerging knowledge and best practices that will have to be incorporated in the future and may alter the trajectory of affected programs. This information may be supplemented by qualitative data collected during site visits. Are successful linkages being developed between PEPFAR prevention services and other services where appropriate in order to better meet the needs of targeted individuals and to increase coverage and efficiencies when available? For example, are women identified
77 PREVENTION SERVICES as HIV-positive and HIV-exposed infants in PMTCT programs successfully referred to and enrolled in treatment programs? Prevention services have the potential to serve as entry points to identify those in need and increase access for treatment and other services. Optimal implementation of PMTCT programs, for example, can not only reduce perinatal transmission to 2 percent with the use of combination ART, regimens but also serve as a linkage to provide HIV-positive women with ART (Cunningham et al., 2002; Dorenbaum et al., 2002; Kuhn et al., 2008). As there are currently no centrally reported indicators for âlinkages or referrals,â the committee will rely primarily on recommended measures such as âthe number of health facilities providing antenatal care that provide both HIV and PMTCT services on siteâ as well as information collected during country site visits. The committee may also attempt to triangulate, where available, the information collected with indicators on PMTCT services coverage and ART coverage for women in specific regions, in order to gain insight into PEPFARâs progress toward its goal of 85 percent prophylaxis and treatment of pregnant women found to be HIV- positive. Has PEPFAR implemented effective prevention programs for injecting and other drug users (behavioral, biomedical, and structural interventions and approaches) in regions/countries where drug use drives the epidemic? What has been their effect? What has been the coverage and are interventions evidence-based? The committee will use the reported PEPFAR output indicator of ânumber of [IDUs] on opioid substitution therapyâ and the recommended indicator of âpercent of [IDUs] on opioid substitution therapyâ where available. However, as more information will likely be required to perform a sufficient evaluation, the committee will also pursue alternative data sources. Issues of stigma and confidentiality pose challenges to program implementation and data collection as the population of IDUs is often hidden or difficult to access. In order to increase access to services and adoption of testing and preventive behaviors, what efforts has PEPFAR made to reduce stigma through community programs or campaigns or policy initiatives? What has been their effect? Structural and policy approaches to reduce stigma and vulnerability are important components of comprehensive HIV combination prevention, but their outcomes, effects, and impact are difficult to evaluate. The committee will use PEPFAR indicators that are disaggregated by sex and testing location (to avoid obscuring the potential male dominance in results outside of maternal and child health settings) where available and supplement these with qualitative data collected during site visits. However, given the lack of data availability, the potential for analysis of interventions to reduce stigma and vulnerability may be limited.
SECTION 4: ADULT AND PEDIATRIC TREATMENT Recognizing the need for delivering life-saving treatment to the millions of people living with HIV as a global health emergency, PEPFAR I strategically focused on the rapid scale-up of HIV treatment services and an increase in the coverage of HIV-infected individuals in need of ART (IOM, 2007). As of September 30, 2009, PEPFAR has contributed to the support of treatment for 2,485,300 people, of which 8 percent are children under 15 years of age, through bilateral programs in countries (OGAC, 2010). As shown in Table 3, in 2008 PEPFAR reached the legislative 5-year target of assuring treatment for 2 million people (PEPFAR, 2009c), which included treatment for 131,500 children under 15 years of age (OGAC, 2009c). Since this contribution âhas been the major success of PEPFARâ scale-up efforts in the early years (OGAC, 2009i, p. 26), PEPFAR II (2009â2013) plans will focus on supporting a country-led response to the HIV/AIDS epidemic, especially assisting countries in identifying resources available, increasing country coverage, and prioritizing the unmet needs for ART for PLWHA. TABLE 3 Cumulative Number of People on Antiretroviral Treatment by PEPFAR Country PEPFAR Five- Year Target (2003-2008)a Country FY2005 FY2006 FY2007 FY2008 b Botswana 33,000 37,300 67,500 90,500 111,700 77,000 50,500 Côte dâIvoire 11,100 27,600 46,000 Ethiopia 210,000 16,200 40,000 81,800 119,600 Guyana 2,000 800 1,600 2,100 2,300 Haiti 25,000 4,300 8,000 12,900 17,700 Kenya 250,000 44,700 97,800 166,400 229,700 Mozambique 110,000 16,200 34,200 78,200 118,000 Namibia 23,000 14,300 26,300 43,700 56,100 67,100c Nigeria 350,000 28,500 126,400 211,500 Rwanda 50,000 15,900 30,000 44,400 59,900 South Africa 500,000 93,000 210,300 329,000 549,700 Tanzania 150,000 14,700 44,300 96,700 144,100 Uganda 60,000 67,500 89,200 106,000 145,000 Vietnam 22,000 700 6,600 11,700 24,500 Zambia 120,000 36,000 71,500 122,700 167,500 Total 2,000,000 401,200 822,000 1,358,500 2,007,800 NOTES: Numbers reflect totals of downstream (direct) and upstream (indirect) results. a Based on PEPFARâs Congressional Report to Congress of persons on antiretroviral therapy in the original 15 focus countries as of September 30, 2008. b Botswana results are attributed to the National HIV Program. Beginning FY2006, U.S. Government (USG) downstream contributions in Botswana are embedded in the upstream numbers, following a consensus reached between the USG and the Government of Botswana to report single upstream figures for each relevant indicator. c In Nigeria, it is currently unknown if the government's number of people on treatment accounts for people who are lost to follow up, therefore the total number of people on treatment had been reduced by 15 percent to account for the estimated attrition. SOURCE: PEPFAR (2009c). 79
80 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS For treatment programs, the host government may use Global Fund, other donor, or country resources to support specific components of services in some PEPFAR treatment sites, while PEPFAR supports other essential components of the treatment services. Beyond these joint operations of treatment sites by PEPFAR and other donors, other support to PEPFAR specific sites may be national or regional in nature (PEPFAR and USAID, 2007). At the same time, in countries where multiple funders (e.g., PEPFAR and the Global Fund) are present, some PEPFAR delivery sites also receive support from other bilateral and multilateral funding through their investments in the governmentâs national HIV/AIDS programs. From 2005â2009, the total number of individuals directly supported on ART that counted toward the 5-year legislative target included the estimated overlap of individuals receiving ART with support by both PEPFAR and the Global Fund. As of September 30, 2009, the overlap estimate was 1.3 million individuals (OGAC, 2010). This overlap estimate also is included in the treatment results reported by the Global Fund (2.5 million individuals receiving ART with support from the Global Fund). To estimate this overlap, PEPFAR conducts a review of the treatment and funding data with the Global Fund and WHO, on a country-by-country basis. In its review, PEPFAR and the Global Fund take into account the percentage or level of contribution to the national HIV/AIDS program in order to determine where there is likely to be overlap (GFTAM, 2009; OGAC, 2010; PEPFAR, 2010b). In FY2009, PEPFAR and the Global Fund directly supported approximately 3.6 million individuals on ART (number of unique individuals supported by PEPFAR and the Global Fund, excluding the estimated overlap). PEPFAR plans to continue to work with the Global Fund to refine attribution methodologies in relation to treatment results (PEPFAR, 2010). Globally, the efforts in expanding the availability of ARV drugs has resulted in a greater proportion of people living with HIV in need of treatment receiving ART, which is helping to lower HIV-related mortality in multiple countries and regions (UNAIDS and WHO, 2009). Particularly, there is increasing evidence of the effectiveness of ART in decreasing morbidity and mortality in PLWHA in resource-poor settings (Bussmann et al., 2008; Herbst et al., 2009; Jahn et al., 2008; Mermin et al., 2008). Access to ARV drugs in low- and middle-income countries increased 10-fold between 2003 and 2008 (UNAIDS and WHO, 2009). In 2008, ART coverage in low- and middle-income countries reached 42 percent of the 9.5 million people in need, while coverage for children was 38 percent among 730,000 children (WHO et al., 2009). Despite progress in scaling-up access to ART in these countries, the majority of PLWHA and in need of treatment are currently not receiving such services, specifically in sub-Saharan Africa, where there is the greatest need (WHO et al., 2009). As noted above, in 2008 nearly two-thirds of the children under 15 years of age living with HIV worldwide in need of ART are still not receiving treatment (WHO et al., 2009). Some of the challenges in achieving greater coverage of treatment services for children include availability of cluster of differentiation (CD4) testing and ART at primary care, antenatal, delivery, and postnatal facilities where most maternal-child health care takes place (WHO et al., 2009). Furthermore, the difficulty of early infant diagnosis of HIV (EID), mainly due to the lack of affordable and accessible diagnostic testing and monitoring and to the shortage of ART regimens for children, poses particular challenges for increasing ART coverage among HIV- exposed or HIV-infected infants (UNAIDS, 2008; WHO et al., 2009). Without treatment, HIV infection in children follows an aggressive course including a faster progression to AIDS and death than in adults (Newell et al., 2004; Violari et al., 2008). With approximately 6 percent of PLWHA being children (UNAIDS and WHO, 2009), the disease is disproportionately killing
81 ADULT AND PEDIATRIC TREATMENT more children than adults even though current WHO guidelines indicate that all HIV-infected infants and children less than 2 years of age should be started on ART immediately upon diagnosis, irrespective of CD4 count or WHO clinical stage (WHO, 2010). In 2008, 14 percent of AIDS-related deaths (280,000) were in children under 15 years of age (UNAIDS and WHO, 2009). Improved access to pediatric treatment (which includes children 0â14 years) will depend on the ability to identify women and children routinely through maternal-child care service entry points. For example, scaling up optimal implementation of PMTCT programs not only can improve treatment for women by providing HIV-positive pregnant women with ART, but also has the potential to identify and increase access for children in need of HIV treatment services. WHO revised its ART recommendations in 2009 and is now recommending that ART be initiated at a higher CD4 threshold of 350 cells/mm3 (compared to previous levels of 250 cells/mm3) for all HIV-positive patients, including pregnant women, regardless of the symptoms (WHO, 2009a, 2009b, 2009c). The 2009 WHO ART treatment guidelines for adults and adolescents, however, adds another 50 percent to those in need of ART from the 2008 figure aboveâincreasing this number to approximately 15 million people, which is almost half of the PLWHA worldwide (De Lay, 2010). In the past 6 years, PEPFAR has supported activities for building laboratory capacity (OGAC, 2006a, 2009a, 2009c), including laboratory equipment, training, and quality control, given that laboratory monitoring is used by clinicians to track key patient outcomes, including the effects of ART on the well-being of a patient (WHO, 2006b, 2007b). Both CD4 and viral load testing, when available, are now recommended under the new 2009 WHO ART guidelines as part of program M&E activities to ensure quality HIV treatment programs (WHO, 2009a, 2009b, 2009c). In many developed countries this is part of the standard patient treatment monitoring; however, there are challenges to implementing CD4 cell and viral load testing in resource-limited settings, including both test cost and lack of infrastructure. These challenges may result in countries making determinations about how high a priority current ART programs should put on making these tests available vis-Ã -vis expanding the number of patients on treatment (Phillips and van Oosterhout, 2010). There is some evidence that the use of viral load monitoring in resource-limited settings improves patient outcomes as much as it does in high- resource settings (Gupta et al., 2009). The effect of routine CD4 cell monitoring on mortality was modest in a randomized trial in Uganda and Zimbabwe, and thus it requires additional evaluation (Dart Trial Team, 2009). The expanding goals of ART are to drive and maintain HIV-1 RNA plasma levels below limits of detection, preserve and restore immune function, and prevent or delay the clinical progression of disease (Hammer, 2010). Minimizing drug toxicities and prevention of drug resistance are also important goals of ART, and so adherence to treatment is one of the crucial variables of effective ART programs. A continuous supply chain for drug access, tolerable and convenient treatment regimens, and counseling efforts can enhance adherence to ART (and ultimately provide a normal life expectancy including improving quality of life). It is expected that with the 2009 WHO ART guidelines, tolerability and convenience of the initial regimen will improve by withdrawing stavudine from the recommended list of nucleosides (WHO, 2009a, 2009b). Earlier initiation of treatment is another variable as it confers the potential benefits of reducing the mortality associated with treatment of late stage disease, diminishing transmission including transmission from mother-to-child in the peripartum and during breast-feeding, diminishing drug toxicity, and reducing treatment failure with its attendant development of drug
82 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS resistance (Emery et al., 2008; Granich et al., 2010; Johansson et al., 2010; Kuhn et al., 2008; Kuhn, 2009; Lawn et al., 2008). Furthermore, treatment also confers other potential population health benefits such as reducing secondary transmission of HIV and TB (Corbett et al., 2003; Girardi et al., 2000; Granich et al., 2009; Lima et al., 2008; Montaner et al., 2006; von Linstow et al., 2010; Wood et al., 2009). HIV drug resistance in treated patients is a measure of regimen efficacy and patient adherence; HIV drug resistance in treatment naïve patients is a measure of transmitted drug resistance. Consequently, monitoring for HIV drug resistance is important in order to assess program efficacy and future challenges with regard to different treatment regimens, maintaining a supply chain for drug access, providing care and counseling, and monitoring of treatment and its outcomes (WHO, 2006b). Under PEPFARâs laboratory program, PEPFARâs investments have provided support for the monitoring of drug resistance, which included âestablishing HIV drug resistance testing capacityâ and training of people from PEPFAR countries âon monitoring the emergence of drug resistance mutations in antiretroviral drug treated populationsâ (OGAC, 2009c, p. 128). Objective and Scope The IOM evaluation of treatment programs under PEPFAR will assess the effects of these programs on long-term outcomes such as retention of people on treatment and increasing coverage and quality of services, as well as PEPFARâs contribution to improving the health of individuals and decreasing population level mortality due to HIV/AIDS. This will include consideration of some health systems components. PEPFAR treatment programs, for purposes of budgeting and performance targets, include activities that directly or indirectly support the provision of ART. This includes the training of clinicians and other providers, clinical examinations, clinical monitoring, related laboratory services, and community-adherence activitiesâas well as procurement and in-freight delivery of ARV drugs (OGAC, 2009e). Like ARV drug procurement, all procurement for post-exposure prophylaxis for rape victims will be included under treatment activities. However, distribution, supply chain logistics, pharmaceutical management, and related systems-strengthening inputs do not fall under PEPFARâs treatment activities and will be assessed and evaluated under PEPFARâs health systems strengthening activities (see section 8 on Key Systems-Level Goals and Activities). The evaluation of treatment programs will focus on PEPFARâs progress in achieving direct support of more than 4 million people on treatment (OGAC, 2009i)âthe PEPFAR II programmatic targetâand an assessment of the overall performance of treatment programs, including efforts to integrate with other health services and to address gender-specific aspects of HIV/AIDS treatment (see section 7 on Gender-Related Vulnerability and Risks). The committee might also find it necessary to consider some of the emerging issues identified by PEPFARâs TWG on Adult and Pediatric Treatment during the evaluation and assessment of treatment programs under PEPFAR. One of the main challenges that country programs face is continuing to scale up services for those in need while maintaining current numbers of people on treatment. Furthermore, according to PEPFARâs reauthorization, countries need to provide care and treatment services to HIV-infected children in proportion to the pediatric HIV burden in each country. 22 Its stated treatment and care goals for children in relation to allocation of funds across all treatment service categories over time may be valuable metrics for program âsustainability.â 22 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(v).
83 ADULT AND PEDIATRIC TREATMENT Although the 2009 WHO guidelines on ART for adults, adolescents, and pregnant women substantially expand the population for whom treatment is indicated, PEPFARâs new Five-Year Strategy indicates that priority for access will focus on increasing access primarily within the âsickest,â pregnant women, and patients with HIV and TB co-infection (OGAC, 2009i) (see Box 3). How to address treatment needs for the many eligible patients who may remain untreated is an important challenge. The evaluation committee can learn from the findings of detailed scholarly evaluations conducted by local researchers and the scientific communities of the partner countries related to overall access to treatmentâconducted as a consequence of specific in-country decisions made and intended for future country leadership to learn from their historic experiences. Further, many challenges remain in the pediatric population, including issues with early identification and diagnosis and low access of ART in children less than 2 years old. This is compounded especially by inadequate funding for pediatric treatmentâin several countries, UNITAID 23 and the Clinton Foundation are the sole purchasers of ART formulations and 2nd regimes for children, EID commodities, cotrimoxazole (CTX), and other drugs for opportunistic infections (OIs) (OGAC, 2009c; UNITAID-CHAI, 2010). UNICEF and UNAIDS have determined that about $6 billion of the $25 billion, needed by 2010 to enable countries to reach universal access goals, 24 is required to attain the universal goals specific to women and children (UNICEF, 2009). In 2010â2011, UNITAID and the Clinton Foundation will deliver the last shipment of some of these commodities; therefore, countries will need to make up for the gap (OGAC, 2009c). Since the United States alone cannot sustain ART for the millions of PLWHA, PEPFARâs focus under its new strategy is to support countries in discussing what resources are needed to respond to the HIV/AIDS epidemic, how to prioritize the large unmet need for treatment of adults, adolescents, and children, and how to identify resources for the gapâPEPFAR plans to support these efforts in a five-year plan through partnership frameworks with partner countries. 23 UNITAID is an international facility dedicated to purchasing drugs for HIV/AIDS, malaria, and tuberculosis, primarily for people in low-income countries. UNITAID leverages its funds, received through airline ticket taxes or regular multi-year budget contributions from member countries, to reduce the price of quality diagnostics and medicines as well as to accelerate the development and availability of these products in low- and middle-income countries (UNITAID, 2010). 24 At the United Nations General Assembly High-Level Meeting on AIDS in 2006, countries committed to work toward âuniversal access to comprehensive [HIV] prevention programmes, treatment, care and support by 2010â (United Nations General Assembly, 2006, p. 3).
84 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS BOX 3 Areas of Emphasis in PEPFAR IIâs New Five-Year Strategy for Treatment In partnership with country governments, PEPFAR is continuing the scale up of treatment to directly support more than 4 million people in its next phase. PEPFAR is working with countries to reach a threshold of 85 percent ARV prophylaxis or treatment of pregnant women found to be HIV-infected, in order to optimize maternal health and maximize HIV-free infant survival. In generalized epidemics, PEPFAR is working to reach a target of 65 percent early infant diagnosis, and support treatment for pediatric populations at a level commensurate with their representation in a larger country epidemic. Through country- and global-level efforts, PEPFAR is creating increased sustainability and capacity in treatment efforts and supporting countries in mobilizing and coordinating resources from multiple donors. PEPFAR is working with countries and international partners to expand identification and implementation of efficiencies in treatment, while ensuring continued expansion of measures to maintain adherence, quality, and retention in care. As part of the U.S. Global Health Initiative, PEPFAR is integrating its treatment programs with prevention and care portfolios, other health programs, and larger development efforts. SOURCE: OGAC (2009i). Program Impact Pathway The evaluation approach for those activities under PEPFAR specific to adult and pediatric treatment will follow the impact pathway framework (see Figures 10 and 11). This approach is intended to illustrate how PEPFARâs investments and other laboratory commodities (inputs) for the delivery of ART services increase availability of ARVs and access to treatment interventions (outputs). Subsequently, the coverage of those in need of and eligible for treatment and the delivery of quality treatment services for HIV patients (outcomes) are expected to ultimately improve health of individuals and decrease population-level mortality (impacts). Time is required for the effects of the PEPFAR program on outcomes for a chronic disease like HIV/AIDS to become apparent and, more importantly for this evaluation, to accrue the systematic collection of valid and relevant data. Given the timely availability of quality data, the committee has developed examples of illustrative questions that it could address during the evaluation of PEPFAR. The committee developed a separate impact pathway for evaluating pediatric treatment considering the differences with adult treatment, including different points of entry for pediatric treatment services and socio-cultural barriers associated with the effective delivery of pediatric treatment by clinicians and non-clinicians, as well as the availability of drug formulations for children or appropriate use of adult preparations when substitutions are necessary (see Figure 11).
FIGURE 10 Program impact pathway for evaluation of PEPFARâs adult treatment program. NOTE: CD4 = cluster of differentiation 4. 85
86 FIGURE 11 Program impact pathway for evaluation of PEPFARâs pediatric treatment programs. NOTES: ARV = antiretroviral drugs, ART = antiretroviral therapy, CD4 = cluster of differentiation 4, MCH = maternal and child health, mo = months old, yrs = years old.
87 ADULT AND PEDIATRIC TREATMENT Illustrative Questions Most of the limitations in evaluating the impact of treatment programs under PEPFAR and assessing their performance have to do with the fact that all the essential indicators gathered by OGAC from all partners are aggregated and limited. Age disaggregation is exceedingly important, especially to assess PEPFARâs performance in treating infants under 2 years old since ART, about one-third of HIV-infected infants will die by age 1 year and 50 percent by age 2 years (Newell et al., 2004). Meaningful data disaggregation will occur at the country level and, if available, are probably accessible through multilateral organizations such as UNAIDS and UNICEF, but without attribution to specific donorsâthese data will have to be requested. DHS and Multiple Indicator Cluster Survey (MICS data) will also need to be accessed in order to provide country context. The following are examples of illustrative questions that the committee could address. These questions are intended to contribute to addressing part A, item ii and part B, items i and iv of the areas for consideration in the congressional mandate. 25 Is treatment funding (solo and joint PEPFAR and Global Fund) and provision of services being efficiently (equitably) distributed among countries, contractors, and implementers? What is the number and proportion of adults and children with advanced HIV infection (i.e., CD4 <200 for adults) currently receiving ART? What is the number and proportion of adults and children receiving treatment according to the updated WHO guidelines (i.e., all newborns, adults <350, all pregnant women)? To what extent does the current methodology adequately or appropriately determine PEPFAR and Global Fund overlap in numbers? What is the financial sustainability of existing and expanded coverage with treatment services, specifically ART? To what extent do PEPFAR activities or PEPFAR/Global Fund joint activities assist countries in determining and projecting costs for existing and expanded coverage with treatment services including ART? To what extent are CD4 measurements being utilized to initiate treatment and monitor treatment? PEPFARâs NGO sub-partners, especially the Track 1.0 partners for treatment and care as well as the Presidential initiative for PMTCT that pre-dates PEPFAR I (but was subsequently subsumed within PEPFAR), have longitudinal reporting for as long as 10 years, and the ability to disaggregate by facility, as well as by many other parameters. Longitudinal, individual patient 25 (A)(ii) an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by United States funding, including multilateral and bilateral programs involving joint operations (B)(i) an assessment of progress toward prevention, treatment, and care targets (B)(iv) an evaluation of the impact of treatment and care programs on 5-year survival rates, drug adherence, and the emergence of drug resistance PREPUBLICATION COPY: UNCORRECTED PROOFS
88 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS data are available only from very few of sub-partners, but do include large numbers of people. The most detailed information will therefore represent specific facilities in selected countries. What are the rates of drug resistance in treated and naïve populations? How are these data used for policymaking decisions about selection of treatment regimens? PEPFFAR supports the WHOâs strategy on HIV drug resistance surveillance and monitoring, and has provided support to build drug resistance testing capacity in PEPFAR countries (OGAC, 2009c). Globally, the WHO HIV drug resistance threshold survey method is being implemented in resource-limited countries to determine transmitted drug resistance levels as part of the surveillance of transmitted HIV drug resistance. The results of these surveys are published in scientific journals. The committee will seek to collect these published papers in order to assess the effects of treatment programs on the emergence of drug resistance. What percent of adults, children, and pregnant women who initiated ART are alive and on treatment at 1 year and annually thereafter? Although treatment adherence is considered essential to successful therapy as well as to extending regimens and reducing medication and hospitalization costs, none of the OGAC indicators measure it. However, in the provision of treatment at the facility, district, and sub- partner levels, adherence has been monitored because it is necessary for the delivery of therapy. Some of the sub-partners, interested in quality of care, have routinely monitored adherence (albeit with multiple and not always standardized or validated definitions). Monitoring is usually done via return visits, pill counts, clinical assessment, and self-report. The proportion of the population on second-line regimens is another measure, although without genotyping/phenotyping it is not always clear whether regimen failure is due to non-adherence. Learning what the individual countries require will guide the committeeâs effort to answer questions on adherence. Currently, assessing the impact of ART programs on mortality at the population level will be limited due to the lack of vital statistical data or the delay in generating vital statistics in many of the PEPFAR countries. Many low- and middle-income countries have inadequate vital or civil registration systems and where the systems exist, they are usually not comprehensive and the cause of death is misreported or underreported (WHO et al., 2009). âSurveys conducted sometimes use verbal autopsies to retrospectively assess HIV-attributable mortality in a population; however, this approach often lacks baseline data for assessing how access to ART affects HIV-related mortality. Some countries have also counted burials of deceased people in the age group of 15â49 yearsâ (WHO et al., 2009). Therefore, although an evaluation of the impact of treatment on 5-year survival of HIV-infected adults and children receiving ART has been requested in the congressional mandate, any survival data will be limited and mortality data greater than one- to two-year survival will not be available for most PEPFAR treatment programs.
SECTION 5: CARE AND SUPPORT SERVICES Care and support services are an important component of PEPFAR and other donor programs for HIV/AIDS. PEPFAR defines care and support services as âthe wide range of services other than ART offered to people living with HIV/AIDS (PLWHA) and other affected persons, such as family membersâ (OGAC, 2009a, p. 16). During FY2009, PEPFAR directly supported care and support for nearly 11 million people affected by HIV/AIDS, including 7 million HIV-infected individuals and approximately 4 million orphans and vulnerable children (OGAC, 2010). Care and support includes clinical, psychological, social, spiritual, and preventive services that may be provided in facility-, community-, or home-based settings. The LantosâHyde Act of 2008 charges PEPFAR II to support care for 12 million people infected with or affected by HIV/AIDS (including 5 million children orphaned or made otherwise vulnerable by HIV/AIDS), âwith an emphasis on promoting a comprehensive, coordinated system of services to be integrated throughout the continuum of care.â 26 Efforts to integrate care and support services with broader health and development programs, such as voluntary family planning and reproductive health services, are a key component of PEPFAR II (OGAC, 2009g). As outlined in the new Five-Year Strategy (2009â2013), PEPFAR has adopted a âwoman- and girl-centered approachâ to delivering services, which âtakes into account the realities of womenâs and girlsâ lives as shaped by gender norms, service availability, and larger structural factors,â and is working to ensure that other marginalized populations have equal access to services (OGAC, 2009h, p. 6, 2009i). Components of PEPFAR Care and Support Services The latest PEPFAR-issued guidance for care and support services, âGuidance for United States Government In-Country Staff and Implementing Partners for a Preventive Care Package for Adults,â was released in 2006 and describes a menu of preventive care services for adults (OGAC, 2006b). A similar menu of services for children (0â14 years) was also released in 2006 and is described in the Child and Adolescent Well-Being section of this report. These preventive care services (see Figure 12) are intended to promote health and quality of life for PLWHA, slow the progression of AIDS, and reduce HIV-related complications and mortality. The preventive care services for adults include five of the thirteen interventions considered essential by WHO for adults and adolescents living with HIV in resource-limited settings (five additional interventions are delivered through PEPFAR prevention and treatment services) (OGAC, 2006b; WHO, 2008). 26 Supra., note 6 at §101(a), 22 U.S.C. §7611(a)(4)(C). 89
90 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS FIGURE 12 PEPFAR care and support services. NOTES: HPV = human papillomavirus; ITNs = insecticide-treated nets; PCP = Pneumocystis jiroveci pneumonia; STI = sexually transmitted infections; TB = tuberculosis.
91 CARE AND SUPPORT SERVICES Although the type of interventions in a preventive care menu will differ by region or country according to the capacity of implementing partners, some recommended components include: Screening, Prophylaxis, and Treatment for Tuberculosis TB and HIV co-infection is very common and has devastating consequences for PLWHA. HIV infection increases the risk of TB 10-fold, and in sub-Saharan Africa and Asia, TB is the leading cause of death for PLWHA (WHO, 2008). WHO guidelines recommend counseling (education) and regular TB screening for all PLWHA, and preventive therapy (isoniazid) for HIV-infected patients with latent TB (WHO, 2008). In order to identify people who have HIV infection, PEPFAR efforts have focused on the scale-up of same-day HIV testing at TB clinics. However, even in countries where this scale-up has been effective, a large number of TB patients identified as HIV-positive are lost to follow up after referrals to HIV/AIDS care and treatment programs, and efforts to screen, diagnosis, and treat all HIV patients for TB have been less successful (OGAC, 2009c). Further, lack of laboratory capacity has hindered efforts to scale up isoniazid preventive therapy for PLWHA with latent TB infection, as many countries lack the ability to rule out active TB, a prerequisite for isoniazid prophylaxis (OGAC, 2009c). The new Five-Year Strategy commits PEPFAR to scale up efforts to screen, diagnose, and if necessary, treat all HIV patients for TB, while expanding linkages and referrals to ensure that all TB patients are tested for HIV and if positive, referred to treatment (OGAC, 2009g, 2009h). Prophylactic Drugs for Opportunistic Infections CTX is a broad-spectrum antimicrobial agent that prevents Pneumocystis jiroveci pneumonia (formerly Pneumocystis carinii pneumonia), toxoplasmosis, and malaria (WHO, 2006a). WHO recommends that all adults with HIV receive CTX prophylaxis indefinitely as a cost-effective method to significantly reduce morbidity and mortality, but country-level policies vary according to the burden of HIV and other diseases, as well as the capacity and infrastructure of health systems (WHO, 2008). WHO also recommends that all infants born to HIV-infected mothers receive CTX (WHO, 2006a). Cryptococcal disease is common and often treatable in PLWHA, but many countries lack the infrastructure and human capacity for diagnosis (OGAC, 2009c; WHO, 2008). PEPFAR has provided limited training and laboratory capacity building for diagnosis, but without early recognition, mortality from cryptococcal disease is high (OGAC, 2009c). Where cryptococcal disease is common and diagnostic capacity exists, WHO recommends consideration of antifungal prophylaxis (fluconazole or itraconazole) for severely immunocompromised PLWHA (WHO, 2008). Currently, there is limited availability of antifungal prophylaxis, but PEPFAR is working with its Supply Chain Management System and Pfizer, which runs a fluconazole donation program, to increase access to drugs for treatment and prevention (OGAC, 2009c). Improved Screening and Treatment of Opportunistic Infections (including Cervical Cancer) Screening and diagnosis of OIs may be limited by human and laboratory capacity in many countries. Cervical cancer is almost always caused by certain types of human papilloma virus. Cancer-causing strains of human papilloma virus are common in women with HIV and
92 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS increase the risk of cervical cancer (OGAC, 2009c). WHO recommends that where available, women with HIV should be screened for cervical cancer annually (WHO, 2008). As part of a comprehensive approach to OIs, PEPFAR is currently supporting âpilot programs which provide screening and treatment to prevent cervical cancer in HIV-positive womenâ (OGAC, 2009a, p. 55). These pilot programs are using the âsee and treatâ approach, which includes visual inspection with acetic acid, visual inspection with Lugolâs iodine, and direct visual inspection (Kaur and Singh, 2010). Increased Access to Safe Drinking Water to Prevent Waterborne Illnesses and the Promotion of Basic Hygiene and Sanitation to Reduce Exposure to Pathogens In many developing countries, poor infrastructure and lack of safe management of human waste increase the risk of waterborne and enteric pathogens, many of which cause diarrhea (WHO, 2008). Diarrhea affects 90 percent of PLWHA, and interventions to improve water, sanitation, and hygiene, such as provision of safe water storage vessels and education regarding hand-washing, can greatly reduce diarrhea-related morbidity (OGAC, 2009c). The latest USG âFramework for Addressing Water Challenges in the Developing World,â which guides USAID and DoS efforts, encourages the incorporation of these interventions into all HIV/AIDS programs (USAID and DoS, 2009). PEPFARâs preventive care package also includes water purification systems, which in combination with other health interventions, may keep PLWHA healthy and delay the need for treatment (OGAC, 2009a). Prevention of Malaria WHO recommends the integration of malaria and HIV services with a particular focus on prevention (WHO, 2008). As previously mentioned, CTX may reduce malaria-related morbidity and mortality in PLWHA. Insecticide-treated nets (ITNs), when used properly and regularly, are cost-effective and greatly reduce exposure to malaria infection. WHO also recommends intermittent preventive therapy (IPTp), which can reduce the risk of malaria and its consequences, for HIV-positive pregnant women who are not taking CTX (WHO, 2008). The Presidentâs Malaria Initiative (PMI) is a USG interagency initiative to reduce malaria in 15 focus countries, 9 of which have significant PEPFAR programs (PMI, 2009). PMI is working to expand coverage of effective malaria prevention and treatment interventions, including ITNs, indoor residual spraying with insecticides, IPTp, and artemisinin-based combination therapy (PMI, 2009). PEPFAR routinely links to PMI, and their efforts overlap in ITN distribution and education programs as well as coordination of lab services (OGAC, 2009a). Food and Nutrition Support Services HIV infection may cause or intensify malnutrition by reducing appetites, increasing energy needs, and impairing nutrient absorption in PLWHA (OGAC, 2009c). Proper nutrition supports the immune system, preventing OIs. Nutritional and micronutrient supplementation may reduce HIV-related morbidity and mortality and improve outcomes for patients on ART (OGAC, 2009i). Through its Food by Prescription programs, PEPFAR targets clinically malnourished children and adults with HIV infection, pregnant and lactating women and their infants in PMTCT programs, and orphans and other vulnerable children (regardless of HIV status) for food
93 CARE AND SUPPORT SERVICES and nutrition care and support, including nutrition assessment and counseling services, specialized food products, and micronutrient supplementation (OGAC, 2009c). Although PEPFAR does not support direct food distribution to families, the new Five-Year Strategy emphasizes linkages and referrals of those in need to the new USG Global Hunger and Food Security Initiative, Title II programs, and other initiatives such as the World Food Program (OGAC, 2009c, 2009g). Health, Dignity, and Prevention Programming for PLHWA and Their Families Health, Dignity, and Prevention programming includes efforts related to health promotion and education, reducing stigma for PLWHA, and preventing HIV transmission from infected to non-infected people. The components of the preventive care package recommended by PEPFAR address health promotion, and prevention efforts are discussed in this reportâs prevention section. PEPFAR is working with country governments to develop âpolicies that address the drivers of the epidemic in country and provide equitable access to quality services for marginalized populationsâ and expanding âlinkages to multiple primary and specialty health servicesâ which âincreases community-level access to quality care and reduces the stigma associated with HIVâ (OGAC, 2009i, pp. 15, 27). Palliative Care, Including Management of Pain and Other Symptoms PEPFAR defines palliative care as an âholistic approach to providing services that includes a focus on pain and symptom management and on improving quality of life,â which is consistent with the WHO definition (OGAC, 2009c, p. 25). Palliative care, including pain management and end-of-life care, enables PLWHA to lead happier, more productive lives and reduces the burden of care on families. In many countries, restrictive policy environments prohibit effective pain management programs, and access to strong pain medications such as opioids is limited (OGAC, 2009i). Up to 80 percent of those with advanced HIV infection experience pain, and pain management programs can greatly improve quality of life for PLWHA (OGAC, 2009c). PEPFARâs new strategy calls for continued efforts to âsupport policy changes that ensure pain management is included both in guidelines and actual clinical services for PLWHAs,â as well as increased efforts to âstrengthen commodity systems, train providers, and expand access to opioids for pain managementâ (OGAC, 2009i, p. 19). Economic Strengthening and Support Activities Recognizing that a lack of economic assets increases vulnerability to HIV infection, PEPFAR supports economic strengthening and support activities that âsupply, protect, or grow physical, natural, financial, human, and social assetsâ (OGAC, 2009h, p. 17). These activities may include microfinance and microcredit programs to expand access to financial services, vocational training to offer alternatives to transactional sex, and income-generating activities, such as communal gardens (that may also provide food). Objective and Scope
94 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS With regard to care and support services, the committee is charged to evaluate the impact on health of care efforts supported by USG funding and assess progress toward care targets. Complicating this task, country-level guidelines for ART eligibility differ and make program- wide comparisons difficult. The committee is also charged to evaluate impact of care programs on 5-year survival rates among people not yet eligible for ARV treatment. Data on 5-year survival rates is very limited for any population; unless these data are being collected and can be accessed, it will not be possible to assess PEPFARâs impact on 5-year survival rates among people not yet eligible for ART. Program Impact Pathway To assess whether PEPFAR has achieved targets regarding care and support services, the committee will examine PEPFARâs care and support activities and use output and outcome indicators (e.g., number of people accessing care services, percent of HIV-positive patients provided CTX prophylaxis) where available. To evaluate the impact of care services on health, the committee will use a combination of output and outcome indicators to determine the extent of delivery of PEPFAR-funded services in-country. Figure 13 depicts the committeeâs understanding of some of PEPFARâs care and support activities and some potential outputs and outcomes to be measured during the evaluation.
FIGURE 13 Program impact pathway for evaluation of PEPFARâs care and support programs. NOTES: ART = antiretroviral therapy; HBC = home-based care workers; HCW = heath care workers; ITNs = insecticide-treated nets; OIs = opportunistic infections; TB = tuberculosis. 95
96 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions Care is composed of multiple components, and the complexities of measurement are increased by what is included or has changed over time during PEPFAR (i.e., 14 of the essential PEPFAR indicators of care have changed significantly, have been dropped, or are new). 27 The lack of consistently measured data may result in data gaps that limit the possibility of examining longitudinal trends. Given the timely availability of quality data, the following are examples of illustrative questions that the committee could address. These questions are intended to contribute to addressing part A, item ii and part B, items i and iv of the areas for consideration in the congressional mandate. 28 What populations are accessing care and support services? The number and types of populations accessing care will be measured through quantitative analysis of PEPFAR indicators. Aggregated data reported to OGAC provide limited insight about the types of populations accessing care, so the committee will seek disaggregated data from other sources. To reflect PEPFARâs new women- and girl-centered approach to delivering services and the emphasis on equity across marginalized populations, the committee will seek data disaggregated by sex and population type (IDUs, MSM, etc.). Measurement of the number of people receiving care services is complicated by the potential of âdouble countingâ (people accessing more than one service may be counted more than once). What is the access to prophylaxis for, diagnosis of, and treatment of OIs? How has treatment for TB been maximized with jointly operated activities between PEPFAR and the Global Fund? How has coverage expanded as result of joint efforts? What has been the effect on grantee performance to improve access? There are several PEPFAR indicators that measure the number and percentage of HIV- positive patients receiving CTX, isoniazid preventive therapy, and TB screening and treatment (OGAC, 2009d). In addition to the aggregated data reported to OGAC, the committee will seek additional data sources to provide information regarding testing and treatment for malaria and sexually transmitted infections, treatment for pneumonia and diarrhea, vaccinations for human papilloma virus, and more. The committee will also seek information regarding human and laboratory capacity building to increase diagnostic capabilities for OIs. How is PEPFAR supporting access to and distribution of ITNs? How is this measured? There are no PEPFAR indicators regarding ITNs. The committee will seek information from PMI or implementing NGOs or contractors that receive PEPFAR funding to distribute ITNs. 27 Essential indicators are those for which OGAC requires PEPFAR Country Teams to track data to monitor PEPFARâs progress (OGAC, 2009d). 28 (A)(ii) an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by United States funding, including multilateral and bilateral programs involving joint operations (B)(i) an assessment of progress toward prevention, treatment, and care targets (B)(iv) an evaluation of the impact of treatment and care programs on 5-year survival rates, drug adherence, and the emergence of drug resistance
97 CARE AND SUPPORT SERVICES How is PEPFAR supporting access to safe drinking water, basic hygiene, and sanitation? How is this measured? Is there any discernible effect of this access for improved health or a decrease in OIs diagnosed and needing treatment? There are no PEPFAR indicators regarding safe drinking water, basic hygiene, and sanitation. The committee will seek data and information regarding access to safe water, ideally multi-year data to examine trends, from countries, implementing agencies, partners, sub-partners, and international stakeholders. Some data may be available from implementing NGOs or contractors that receive PEPFAR funding to conduct activities regarding basic hygiene and sanitation. How is PEPFAR incorporating palliative care or pain management into care services? What is the proportion of need compared to availability of services, particularly analgesics and opioids? Are PEPFAR teams following WHO guidelines for provision of analgesics including opioids? PEPFAR indicators measure the percentage of ART sites that have pain management programs and the progress toward incorporating pain management into national HIV/AIDS strategies, but the committee will seek additional information from other sources. The committee will also seek information from the Palliative Care TWG at OGAC about proposed activities for and guidance to Country Teams to develop or change policies in the country related to drug procurement, drug security, and scope of practice for dispensing drugs to patients. What economic support and strengthening activities does PEPFAR support? How is this measured? There is one PEPFAR indicator that measures the number of eligible adults and children provided with economic support or strengthening services, but the committee will need to seek additional information regarding the types of economic support and strengthening activities and their effectiveness from other sources. Who is receiving therapeutic or supplementary food and through what activities? Is there a discernible effect on ART adherence or mortality? PEPFAR indicators measure the number of people receiving food or nutrition services and the percent of HIV-positive clinically malnourished patients receiving food. The committee will use aggregated data reported to OGAC and seek disaggregated data from organizations and initiatives to which PEPFAR links or refers HIV-positive patients for nutritional support.
SECTION 6: CHILD AND ADOLESCENT WELL-BEING The HIV/AIDS pandemic has produced devastating effects on the lives of millions of children and adolescents 29 worldwide endangering their development, life course, and survival. When HIV affects parents and other adult caregivers, it destroys their families and deprives them of care and protection by weakening communities and social support networks, welfare systems, and economies. Moreover, millions of children and adolescents are currently directly affected due to infection with HIV. In 2008, children under 15 years of age were estimated to be nearly 16 percent of incident cases, and the number of children under 15 years of age living with HIV/AIDS worldwide was approximately two million, with an additional number of adolescents among the 31 million adults (age 15 and over) living with the disease (UNAIDS and WHO, 2009). In 2007, the number of children and adolescents aged 0â17 years old who have lost one or both parents as a result of the AIDS epidemic 30 was estimated to be approximately 15 million worldwideâin sub-Saharan Africa alone, the number was estimated to be approximately 12 million children (UNAIDS, 2008). Combining the needs of children and adolescents infected with HIV with the needs of orphans, as well as other children and adolescents made vulnerable due to HIV/AIDS, gives a full perspective on the burden of the epidemic in these populations. Therefore, understanding the needs of all children and adolescents affected and made vulnerable by HIV/AIDS is a vital step in the response to the HIV/AIDS pandemic (JLICA, 2009). In addition to the need for access to HIV/AIDS services, there are also other critical developmental and societal factors influencing the health and psychosocial well-being of children and adolescents affected by HIV/AIDS. For example, many children and adolescents with sick and dying parents have to become the breadwinners and primary caregivers of their households (Cluver et al., 2007). When a parent dies, some of the effects related to the grieving process, as well as the deprivation and life changes that occur because of this loss, might affect the health and well-being of the lost parentâs children. Parental loss might result in trauma, relocation, loss of a breadwinner, residence in poorer households, and living with less closely related caregivers, which can lead to other effects including poorer access to adequate nutrition, shelter, and health care, and lack of educational support. These are all mediating factors of psychosocial well-being (Cluver and Orkin, 2009; Nyamukapa et al., 2008). Children and adolescents, in settings where HIV is highly stigmatized, have to cope with higher levels of psychosocial stressors associated with the loss of a parent due to HIV/AIDS than both children orphaned by other causes and non-orphaned children (Cluver and Gardner, 2007; Cluver and Orkin, 2009). Children and adolescents living within communities that experience a high HIV 29 The committee uses âchildren and adolescentsâ as a general term without a specific age definition, recognizing that the ages used to categorize children and adolescents vary by data source and organization. In particular, the age categories vary for terms like adolescents, youth, and young people. For example, adolescents are defined by WHO as young men and women 10â19 years of age and young people refers to men and women 10â24 years of age (WHO, 1999, 2006c). United Nations defines youth as men and women 15â24 years of age and young people refers to men and women 10â24 years of age (WHO, 1999, 2006c). Within PEPFAR, defined age ranges vary by programmatic area. Throughout this section, the specific age ranges used by PEPFAR or by the cited data source are indicated whenever feasible. 30 In 2001, a consensus was reached among members of the UNAIDS Reference Group on Estimates Modelling and Projection, and international researchers on the definition of orphans due to HIV/AIDS. An âAIDS orphanâ was defined as a child who has at least one parent who has died due to AIDS and a dual (or double) âAIDS orphanâ as a child whose mother and father have both died, at least one due to AIDS (UNAIDS Reference Group on Estimates Modelling and Projections, 2002). 99
100 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS burden are at a greater risk of these forms of vulnerabilities, and are at a greater risk of physical and sexual abuse, sexual exploitation, homelessness, and exposure to HIV (UNAIDS et al., 2002, 2004). In addition to younger children, the vulnerabilities of youth between the ages of 15â24 years have been recognized by the international community (UNGASS, 2001; WHO et al., 1997). This developmental period is an important transition period, and youth are vulnerable due to age-specific changes that are physical (their physical and cognitive abilities), psychological (how they think about themselves), and social (their relationships and roles, expectations, economic security, and citizenship). These changes have implications for how they understand information and what influences them, how they think about the future and make decisions in the present, and how they perceive risk and their sexual behavior (Dick, 2009). In 2007, an estimated 45 percent of incident cases in people aged 15 years and older were found among youth aged 15â24 years. Overall, in 2008, a total of approximately 5 million youth aged 15â24 years were living with HIV in low- and middle-income countries (UNICEF, 2009), and in sub-Saharan Africa, youth, and in particular young women, are disproportionately vulnerable and therefore at greater risk of HIV infection (Gouws et al., 2008; Napierala- Mavedzenge et al., 2010; UNAIDS and WHO, 2009; UNICEF, 2009). Beyond biological susceptibility to HIV, socio-cultural factors that contribute to the vulnerability of young women to sexually transmitted HIV infection include entrenched gender roles, unbalanced power relations, sexual violence (such as coerced sex), unsafe sex with older men, and a lack of skills and information about how to protect themselves and access services (UNAIDS, 2009a) (see section on Gender-Related Vulnerability and Risk). The International Convention on the Rights of the Child,31 guides the international communityâs efforts to protect the rights of children under the age of 18 years to survival, development, and access to health services, including a focus on reversing the HIV epidemic in children and mitigating its negative effects on their health and well-being through the MDGs, the UNGASS on HIV/AIDS, and the UNGASS on Children (UNICEF, 2007). The progress of countries in achieving the standards and goals outlined in these documents is monitored by the Committee on the Rights of the Child (CRC), primarily through country reports that may provide valuable information for this evaluation on the environment in which PEPFARâs efforts to improve child and adolescent well-being are operating within a specific country or region. Efforts by multilateral and bilateral stakeholders to support policy on orphans and vulnerable children and adolescents affected by HIV/AIDS have resulted in the development of the âFramework for the Protection, Care, and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS,â which PEPFAR has adopted (IOM, 2007). This framework document lays out five key strategies to improve the well-being of children: (1) strengthening the capacity of families, (2) mobilizing and supporting community-based responses, (3) ensuring access for orphans and vulnerable children to essential services, (4) ensuring that governments protect the most vulnerable children through improved policy and 31 The Convention on the Rights of the Child is the first legally binding international instrument to incorporate the full range of civil, cultural, economic, political, and social rights of children. The Convention gives UNICEF the responsibility of promoting the rights of children by supporting the Committee on the Rights of the Child (CRC). As the oversight body, the CRC monitors the progress of State Parties in setting and meeting the standards outlined in the Convention. UNICEF provides technical assistance to governments on the implementation of the Convention and the development of implementing reports, which are required to be submitted by State Parties to the CRC 2 years after acceding to the Convention and every 5 years thereafter. The CRC convenes three times a year to review the Statesâ reports (OHCHR, 2007; United Nations, 1990; United Nations Treaty Collection, 2010).
101 CHILD AND ADOLESCENT WELL-BEING legislation, and (5) raising awareness at all levels through advocacy and social mobilization to create a supportive environment (UNAIDS et al., 2002, 2004; UNICEF, 2004). PEPFAR-Supported Interventions for Children and Adolescents PEPFAR currently supports services for children and adolescents through its three main programmatic areasâprevention, care, and treatment. Additionally, in keeping with the âFramework for the Protection, Care, and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS,â PEPFAR supports programs specifically identified for orphans and vulnerable children and adolescents (hereinafter referred to as OVC programs or programming 32 ). In FY2009, PEPFAR supported care for 3,620,140 million children and adolescents through its OVC programming, and provided pediatric treatment for 201,500 individuals under 15 years of age (about 8 percent of the total number of people receiving ART with direct PEPFAR support) (OGAC, 2010). In addition, 273,100 individuals were trained or retained on the care and support of orphans and vulnerable children and adolescents (OGAC, 2010). Other PEPFAR-supported capacity building during this fiscal year to address the needs of children included the training or retaining of 62,100 healthcare workers on PMTCT services (including 15,597 PMTCT service facilities) (OGAC, 2010). The LantosâHyde Act of 2008 underscores children and adolescent needs as part of the USG commitment of preventing 12,000,000 new HIV infections worldwide and increasing the number of individuals with HIV/AIDS receiving ART. In particular, it states that programs supported by PEPFAR need to âprovide care and treatment services to children with HIV in proportion to their percentage within the HIV-infected population of a given partner country.â33 Additionally, PEPFAR II performance targets for the care and support of PLWHA include providing care and support for five million children and adolescents orphaned or made otherwise vulnerable by HIV/AIDS. 34 In order to achieve this target, the new Five-Year Strategy states that PEPFAR will continue to address child-development issues through child-focused programming that targets the full range of needs at different developmental stages (OGAC, 2006e, 2009i, p. 22) (see Box 4). Within the scope of care and support activities for PLWHA, the act directs that âat least 10 percentâ 35 of PEPFAR funds supporting country activities for prevention, treatment, and care go to OVC programs in order to mitigate the impact of HIV/AIDS for millions of children and adolescents living in affected communitiesâan earmark that was preserved in the reauthorization legislation. In FY2009, these funds amounted to approximately $320 million,36 with an additional $44 million provided in funding for pediatric care and support activities (OGAC, 2009f). Finally, the LantosâHyde Act of 2008 also highlights a new emphasis on the transition to young adulthood. The reauthorization legislation states that PEPFARâs annual report to Congress will include a description of the strategies, goals, programs, and interventions that âaddress the needs and vulnerabilities of youth populationsâ and ââexpand access among young men and women to evidence-based HIV/AIDS health care services and HIV prevention programs.â 37 To 32 For the purpose of brevity, the acronym OVC will be used to describe programs targeting eligible children and adolescents under PEPFARâs programs for orphan and vulnerable children. 33 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(v). 34 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iii). 35 Supra., note 1 at §403(b), 22 U.S.C. §7673(b). 36 Approved funds in FY2009 for supporting OVC programsâ activities found in PEPFAR operational plans. 37 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151-b2(f)(2)(D)(ix).
102 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS reflect the new priorities in programming that address the age-specific needs and vulnerabilities of adolescents and young people in particular, PEPFAR II is committing through its new Five- Year Strategy to support countries in pursuing these objectives (see Box 4). BOX 4 Reauthorization Programming for Orphans and Vulnerable Children Years 1â2: Support countries to define, map, and plan a prioritized, multisectoral response to the needs of OVC populations and sub-populations within a country. Work with partner countries to identify gaps in capacity, including gaps in coordination among ministries overseeing education, food and nutrition, social welfare, and health. Establish training, mentoring, and technical assistance programs in partnership with governments in order to increase the number of professional staff in all agencies who can address cross-cutting OVC needs. Work with countries to increase support for family-based care by establishing and strengthening linkages between clinical and home- and community-based care. Scale up and ensure robust monitoring of existing high-impact OVC programs and support countries in developing, implementing, and evaluating innovative OVC pilot programming. Help countries ensure that policies for most-at-risk populations have adequate coverage and referrals for youth subpopulations. Support countries in developing a case management capability to assist the transition of young adults from OVC services into society and careers. Years 3â5: Work with countries to engage in periodic and targeted surveys and other evaluations to determine impact of OVC programming. Ensure that countries have programs through which OVC can access livelihood development opportunities, including vocational training and microenterprise development training, to support themselves and their families. SOURCE: OGAC (2009i). Prevention The LantosâHyde Act of 2008 emphasizes the need to intensify efforts to prevent HIV as a priority within the USG-supported response to the global HIV epidemic. 38 The majority of new incident cases in infants and young children occur from transmission in utero, during delivery, or post-partum as a result of breastfeeding (UNAIDS and WHO, 2009). In 2008, an estimated 1.4 million pregnant women living with HIV in low- and middle-income countries gave birth (WHO et al., 2009). PMTCT, also described previously in the section on Prevention, is an evidence- based prevention intervention that can reduce perinatal HIV transmission to approximately 2 percent if appropriately delivered under optimal circumstances (Cunningham et al., 2002; 38 Ibid., §4, 22 U.S.C. § 7603(3)(A).
103 CHILD AND ADOLESCENT WELL-BEING Dorenbaum et al., 2002; Kuhn et al., 2008). UNAIDS estimated that 200,000 cumulative new HIV infections have been averted in the past 12 years through the provision of ARV drugs for prophylaxis to HIV-positive pregnant women (UNAIDS and WHO, 2009). Similar to other bilateral and multilateral stakeholders, such as the Global Fund, PEPFAR II strategic plans include expanding access to PMTCT services âas a mechanism to both prevent transmission of HIV to children and support expanded access to care and related services for pregnant womenâ (OGAC, 2009i, p. 9). PEPFAR II targets, aligned to follow the targets for PMTCT of the Declaration of Commitment on HIV/AIDS, commits to providing âat least 80 percent of the target population with access to counseling, testing, and treatment to prevent the transmission of HIV from mother-to-child.â 39 In FY2009, PEPFAR reported that 509,800 HIV-positive pregnant women received ARV prophylaxis, and estimated approximately 96,862 infant HIV infections averted (OGAC, 2010). In addition, PEPFAR supports other prevention interventions that are âmedically accurate, age-appropriate, and targeted to the needs based upon behaviorâ (OGAC, 2009i, p. 12; PEPFAR, 2008b). PEPFARâs activities aimed at prevention of sexually transmitted HIV target at-risk youth (OGAC, 2009e), and PEPFAR is supporting prevention programming for young people (10â24 years old) (OGAC, 2009a). Country activities that support âefforts to expand HIV counseling and testing, which are entry points to care and treatment,â also serve orphans and vulnerable children and adolescents (OGAC, 2009a, p. 52). Treatment PEPFAR also serves the needs of children and adolescents through its treatment programs including appropriate pediatric formulations for HIV-infected infants and eligible children (ages 0â14) (see section on Adult and Pediatric Treatment). In addition to improving clinical outcomes and increasing survival in HIV-infected children and adolescents on ART, overall population improvements in access to ART may also have benefits for children and adolescents. Recent studies suggest that improved availability and coverage of ART may reduce the number of children and adolescents who will be orphaned due to HIV in the next couple of years (Mermin et al., 2008; Stover et al., 2008; UNAIDS, 2008; UNAIDS and WHO, 2009). In the FY2009 report to Congress, PEPFAR estimated that the number of orphans averted through PEPFAR-supported treatment programs was approximately 1.6 million through September 30, 2008. However, the availability of data and methods for estimating the effects of ART in protecting children and adolescents from orphanhood continues to be a challenge as more information is gathered on ART coverage and the current needs of child and adolescent populations infected or affected by HIV/AIDS (UNAIDS Reference Group on Estimates Modelling and Projections, 2002, 2007). Care and Support PEPFAR commits to supporting comprehensive care interventions other than ART to meet the new targets of providing care and support for 12 million people infected with or affected by HIV/AIDS, including 5 million children and adolescents orphaned or made otherwise vulnerable by HIV/AIDS, 40 (see section on Care and Support Services). In addition to PEPFAR- 39 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iv). 40 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iii).
104 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS supported pediatric care and support activities described here, the specific OVC programs described later are also among the HIV/AIDS care interventions that PEPFAR supports (OGAC, 2006e, p. 3). PEPFARâs âGuidance for United States Government In-Country Staff and Implementing Partners for a Preventive Care Package for Children Aged 0â14 Years Old Born to HIV-Infected Mothers - #1,â which was released in April 2006, describes PEPFAR-supported activities for children born to HIV-infected mothers (HIV-exposed children), including children in whom an HIV diagnosis has been confirmed (OGAC, 2006c). Although prioritization and selection of the activities in the preventive care package for children (0â14 years old) is country-specific, PEPFAR recommends the following components: diagnosis of HIV infection in infants and young children, childhood immunizations, prevention of serious infections, and provision of nutritional care. PEPFARâs activities include efforts to increase early identification of HIV exposure and infection status in children. In the FY2009 annual report to Congress, PEPFAR indicated its support in âexpanding polymerase chain reaction (PCR) testing to identify the presence of HIVâ including âcountry-level policy change to allow PCR-based dried blood spot testing,â which is thought to reduce the cost and burden of infant diagnosis (OGAC, 2009a, p. 49). PEPFAR also supports clinical pediatric care for children through various sitesâfacilities and community- or home-based settingsâincluding the prevention and treatment of OIs, TB, and other diseases like malaria and diarrhea through the provision of pharmaceuticals, ITNs, safe water interventions, and related laboratory services (OGAC, 2009a, 2009e). Moreover, PEPFAR supports palliative care interventions for children and adolescents, including pain and symptom relief using age-appropriate interventions and methods of administration for pediatric palliative care (ages 0â14) (OGAC, 2006d). This includes psychological, social, and spiritual support services intended to alleviate the burden of families caring for family members living with HIV/AIDS, particularly for children and adolescents who often are forced to drop out of school to become the breadwinners of the household and care for the ill parent (OGAC, 2009a). Other critical factors in the care of children are nutrition and growth; therefore, the PEPFAR-supported preventative pediatric care package recommends services that address nutritional needs, such as therapeutic or supplementary feeding, replacement feeding, and provision of micronutrient supplements (OGAC, 2006c). Through the new Five-Year Strategy, PEPFAR II emphasizes the need for governments âto expand coverage, and access to quality basic care packagesâ for people diagnosed with HIV, including HIV-infected children and adolescents, âparticularly through integration of care with other health and development programmingâ (OGAC, 2009i, p. 18). This is especially significant within the current efforts to guarantee the health and survival of children through referrals and follow-up of HIV-infected children for immunizations, as in the case of many PEPFAR-supported care and PMTCT programs linking services to maternal-child health programs (OGAC, 2009c). Programs for Orphans and Vulnerable Children In its guidance document, PEPFAR defines orphans and vulnerable children as children who are âeither orphaned or made more vulnerable because of HIV/AIDS.â Further, it defines an orphan as a child who âhas lost one or both parents to HIV/AIDSâ (OGAC, 2006e, p. 2). A vulnerable child is defined as a child being âmore vulnerable because of any or all of the following factors that result from HIV/AIDS: is HIV-positive; lives without adequate adult
105 CHILD AND ADOLESCENT WELL-BEING support (e.g., in a household with chronically ill parents, a household that has experienced a recent death from chronic illness, a household headed by a grandparent, and/or a household headed by a child); lives outside of family care (e.g., in residential care or on the streets); or is marginalized, stigmatized, or discriminated againstâ (OGAC, 2006e, p. 2). Because children and adolescents can differ greatly in their needs and individual vulnerabilities, in the past 6 years, PEPFAR has supported âchild-centered, family-focused, community-based, and government- supportedâ OVC programming for these groups in need of further care to facilitate age- appropriate development (OGAC, 2006e, 2009i, p. 22). Therefore, PEPFARâs OVC programs target different age groups of orphans and vulnerable children and adolescents from age 0 to 17 years (see Table 4). TABLE 4 PEPFAR Age Categories for Programs for Orphans and Vulnerable Children Age Stage <2 years Infancy 2â4 Early Childhood/Toddler 5â11 Middle Childhood 12â17 Late Childhood/Adolescence SOURCE: OGAC (2006e). In addition to these age groups, the expanded program goals under the new Five-Year Strategy highlight the importance of refocusing OVC program efforts to also address the needs of several neglected subset populations such as young adults, and in particular the vulnerability of girls (see the Gender section of this report) (OGAC, 2009g). According to this strategic plan, PEPFAR will help countries to develop initiatives that target âadolescent and young adults as they transition from OVC programs into society and careersâ (OGAC, 2009i, p. 23). PEPFAR guidance provides an operational definition and guiding principles for OVC programming decisions; nonetheless âeach community will need to prioritize those children most vulnerable and in need of further careâ (OGAC, 2006e, p. 2). PEPFAR countries use the Child Status Index tool (MEASURE Evaluation, 2009), developed by Duke University and MEASURE Evaluation with USG support, to assist in-country planning of OVC programming based on these core elements and 12 measurable factors that approximate a standard for child health and well-being (OGAC, 2009a). PEPFARâs OVC programming guidance identifies important elements of a childâs and adolescentâs life in seven core areas that are based on the principles of the âFramework for the Protection, Care, and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDSâ (OGAC, 2006e). These factors, described in more detail below, include food and nutritional support, shelter and care, protection, health care, psychosocial support, education and vocational training, and economic opportunity and/or strengthening. PEPFAR also supports OVC programs âthat link OVC services with HIV-affected families [through] linkages with PMTCT, palliative care, treatmentâ (OGAC, 2009e, p. 137). Further, PEPFAR-supported activities under OVC programs include services that directly support orphans and vulnerable children and adolescents as well as their caregivers, families, and community members (OGAC, 2009e). In addition to activities at the level of patient care and the level of the caregiver, PEPFAR also supports activities at the systems level. As emphasized in the new strategic plan, system-wide OVC program activities are aimed at building local, regional, and national capacity to enhance the structures and networks that support healthy child development. This includes
106 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS efforts by PEPFAR to assist countries in coordinating among ministries overseeing education, social welfare, and health in order to develop policy and program responses that lead to comprehensive and effective care for orphans and vulnerable children and adolescents (OGAC, 2009i). Food and nutritional support OVC programs include nutritional assessments and counseling, therapeutic or supplementary feeding and micronutrient supplementation for HIV-infected children based on national and appropriate international guidelines, and replacement feeding and support for children born to HIV-infected mothers (OGAC, 2006e). Orphans and vulnerable children are one of the priority groups identified by PEPFAR to receive food and nutritional support. In FY2008, PEPFAR reported that 814,800 orphans and vulnerable children received support for food and nutritional supplementation in the 15 focus countries (OGAC, 2009a). PEPFAR-supported efforts to provide food and nutritional support use linkages with non-HIV funding mechanisms, such as USAID Title II (Food for Peace program41 ) and the United Nations World Food Program as well as programs in partner countries (OGAC, 2006g). In FY2010 PEPFARâs COP Guidance introduced new budget codes for food and nutrition commodities, policy, tools, and service delivery in order to capture information on country activities under this cross-cutting issue (OGAC, 2009a, 2009e). Currently, the only indicator for OVC programs required by OGAC captures the number of eligible individuals under 18 years old who received food and nutrition services (OGAC, 2009d). Shelter and care Because of the growing number of orphans and vulnerable children and adolescents globally, it is necessary to enhance the capacity of families and communities to care for these children. PEPFAR funds can support shelter and care interventions such as: identifying potential caregivers prior to a guardianâs death, family tracing and fostering, providing access to temporary shelter for children in transition, supporting child- or adolescent-headed households in taking care of their homes, supporting access to programs that incentivize adoption or the provision of foster care, and strengthening community-based and family-based care models for children (OGAC, 2006e). Protection PEPFAR OVC programs addressing this core area may focus on interventions such as: health care and social services facilitating basic birth registration and identification, the provision of community-based assistance to orphans and vulnerable children for inheritance claims, the removal of children from abusive situations into safe temporary or permanent placements, and the strengthening of local community structures that are responsible for monitoring and protecting orphans and vulnerable children (OGAC, 2006e). Health care Core services for orphans and vulnerable children address (1) the general health needs of this group, (2) health care for HIV-positive children, and (3) prevention of HIV. OVC programs are required to disaggregate health requirements and interventions by age, and they 41 The Food for Peace program (Public Law 480, also renamed Food for Peace Act of 2008) is the principal mechanism through which the USG implements its international food assistance. Title II of the Food for Peace Act, which authorizes the vast majority of U.S. international food assistance, is managed by the USAID Office of Food for Peace. USAID Peace's implementing partners include private voluntary organizations registered with USAID, local and international nongovernmental organizations, and the United Nations World Food Program (USAID, 2009a).
107 CHILD AND ADOLESCENT WELL-BEING should facilitate access to primary health care for orphans and vulnerable children (OGAC, 2006e). âGeneral healthâ interventions include: referrals to child health care, the provision of support for survivors of abuse, the training of caregivers to monitor childrenâs health, and capacity building of public and private health providers (OGAC, 2006e). PEPFAR programs provide health care to HIV-positive children, including HIV-exposed children, through direct access to health providers, or through referrals to prevention and treatment services. Psychosocial support Children and adolescents affected by HIV/AIDS suffer anxiety, fear, grief, and trauma with the illness or death of a parent. PEPFAR programs are intended to address the psychosocial and life skills needs of orphans and vulnerable children, including gender- sensitive life skills and experiential learning opportunities; improved links between children affected by HIV/AIDS and their communities; rehabilitation for children who are living outside of family care; and referral to counseling where available and appropriate, particularly for HIV- positive youth (OGAC, 2006e). Education and vocational training Partnerships with the education sector on national and local levels provide an important opportunity to ensure that children and adolescents affected by HIV/AIDS have access to education. Linkages with other programs, such as the USG basic education program or the African Education Initiative (AEI) implemented through USAID, can help expand educational opportunities. For children and adolescents, the USG is providing $400 million through AEI to train 500,000 teachers and provide scholarships for 300,000 children and adolescents, particularly young girls (OGAC, 2009a). Other activities within this core area that are supported by PEPFAR funding include activities that encourage access for orphans and vulnerable children into early childhood development programs, vocational training, activities to integrate orphaned and vulnerable children into community social life, and anti-stigma education. PEPFARâs efforts in this core area also extend to interagency activities, such as the Interagency Education Steering Committee and other strategic planning for the expansion of education wraparound programs that target HIV-infected children and adolescents, as well as those made vulnerable by HIV/AIDS (OGAC, 2009a). Economic opportunity and strengthening PEPFAR programs fund economic strengthening services so that caregivers can meet their responsibilities to ill family members or receive orphaned children into the household. For instance, PEPFAR supports activities that promote the entrepreneurism of caregivers of orphans and vulnerable children through microfinance programs, small-business development, and programs for community-based asset building (OGAC, 2008a). Programs also provide orphans and vulnerable children and adolescents with training and other skills to improve their future economic opportunities (OGAC, 2006e). Objective and Scope The IOM evaluation will assess PEPFARâs progress toward meeting its programmatic targets and strategic goals for children and adolescents, including efforts to increase the number of HIV-infected children receiving treatment and the number of orphans and vulnerable children and adolescents receiving care and support services. Since PEPFAR provides services for children and adolescents through prevention, treatment, and care strategies for children and adolescents, as well as through specific OVC programming, activities in all of these areas, to the
108 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS extent that they serve children and adolescents, will be a part of the assessment of PEPFARâs effects on the well-being of children and adolescents. The evaluation committee, in its review of PEPFAR-supported programs aimed at the age-specific needs of children and adolescents infected and affected by HIV/AIDS, will consider the appropriateness of programmatic guidance and activities in providing effective care and support to improve the health and psychosocial well-being of these populations. This includes assessing the effect of services provided to children and adolescents in each of the programmatic areas, as well as efforts in supporting family-centered programs and community-led initiatives that respond to the needs of orphans and other vulnerable children and adolescents in PEPFAR countries. The evaluation committee will consider how PEPFARâs efforts to guide countries to implement international standards of care for children and adolescents, including those orphaned or made vulnerable by HIV/AIDS, have resulted in measurable effects on the well-being of children and adolescents in PEPFAR countries. This will include a review of PEPFAR-supported initiatives such as the development and implementation of national plans for orphans and vulnerable children and other policies related to improving survival and healthy development of children and adolescents. Program Impact Pathway An impact pathway (Figure 14) summarizes how the IOM evaluation committee proposes to examine the strength of evidence establishing plausible causal links between PEPFAR programs for children and adolescents and their intended impacts. This impact pathway reflects that significant impacts of PEPFAR services in one setting (such as health) may emerge in other service settings (such as education or child welfare). The impact pathway framework will help the committee understand the changes at each stage, in order to describe the relationship between the processes of interventions and their effects. Given the timely availability of data, the evaluation committee will examine whether PEPFAR-funded activities have had an effect on the well-being of children and adolescents through an assessment of mediating output and outcome indicators or intermediate measures of child and adolescent well-being as defined by the committee.
FIGURE 14 Program impact pathway for evaluation of PEPFARâs services for children and adolescents. 109
110 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions The limitations in evaluating the effects of PEPFARâs activities on the well-being of children and adolescents include differences in definitions with respect to the age categories of children, adolescent, youth, and young people (UNAIDS Reference Group on Estimates Modelling and Projections, 2002). While PEPFAR defines age-specific categories in some areas, such as pediatric treatment and OVC programming (Table 4), PEPFAR, country-level, and international data-collecting systems do not all use consistent age categories. However, international efforts to harmonize indicators for M&E of interventions targeting these populations, including orphans and vulnerable children under the age of 18 years, will provide some data for the evaluation of PEPFAR activities. Another potential limitation is the limited availability of age-disaggregated data for many key outcome indicators. The evaluation committee will review program-level data and country- level data to seek sources of age-disaggregated data. However, although countries are increasingly collecting age- and sex-disaggregated data systematically from programs, in particular from those strategically targeting children and adolescents, in many places there still are not enough data to assess and evaluate these programs (UNICEF, 2009). This will be an important issue not only for this evaluation but also during the implementation of PEPFAR II, in particular within the context of implementation plans for partnership frameworks, which under the reauthorization legislation need to contain age- and sex- disaggregated data for âan identification of the intended beneficiaries [â¦] including information on orphans and vulnerable children, to the maximum extent practicable.â 42 In their latest review of PEPFAR-supported care and support activities for children and adolescents, PEPFARâs Pediatric TWG indicated that data on current progress in care and support are not disaggregated by age, HIV exposure, or infection status (OGAC, 2009c). Moreover, they indicate that international indicators, such as UNAIDS globally harmonized indicators, provide only data on scale-up and coverage of particular priority program elements and limited data on receipt of care and support services by HIV-infected children (OGAC, 2009c). Finally, OVC programs may be offered within different settings in which eligible children and adolescents may receive multiple services. This means that there is a risk of a single child being counted several times by different implementing partners, which can make it difficult to determine if targets expressed as ânumber of children receiving servicesâ are being met (OGAC, 2009d). The lack of unique identifiers for each participant in PEPFAR activities constitutes a major methodological challenge for program evaluations in this context. The following are examples of questions that the committee will consider in the evaluation. These questions are intended to contribute to addressing part B, items v, vi, and vii of the areas for consideration in the congressional mandate. 43 How are services supported by PEPFAR tailored to the cultural context for the children and adolescents presenting with needs? Specifically, how are these activities tailored or 42 Supra., note 6 at §301(d)(2), 22 U.S.C. §2151-b2(e)(2)(E)(iv). 43 (B)(v) an evaluation of the impact of prevention programs on HIV incidence in relevant population groups (B)(vi) an evaluation of the impact on child health and welfare of interventions authorized under this Act on behalf of orphans and vulnerable children (B)(vii) an evaluation of the impact of programs and activities authorized in this Act on child mortality
111 CHILD AND ADOLESCENT WELL-BEING targeted to specific age cohorts among populations of children and adolescents orphaned and made vulnerable by HIV/AIDS? The committee will try to assess the cultural sensitivity and age-appropriateness of PEPFAR programs as they attempt to address culturally entrenched beliefs and practices that increase the risk-taking behavior of children and adolescents orphaned or made vulnerable by HIV/AIDS. The committee will rely on qualitative data to ensure that interventions for children are contextually relevant and that PEPFAR program interventions are responsive to variances in high and low HIV prevalence areas. What criteria are used by PEPFAR countries to establish priorities and determine the balance among prevention, treatment, and care interventions for HIV-infected infants, children, and adolescents including those orphaned or made vulnerable by HIV/AIDS? How effective are multi-sector strategic planning and implementation for child and adolescent intervention efforts? Is PEPFAR engaging social, behavioral, and prevention scientists in guiding programs, strategies, and analysis targeting infants, children, and adolescents? According to PEPFARâs stated goals, treatment as well as care and support programming targeting HIV-infected children need to be consistent with the pediatric HIV burden in a community but also with the countryâs continuing sustainability of PEPFAR programs. In order to address these goals, the committee will need to understand how funding priorities are determined for these three main program areas with regard to children and adolescents. The committee will seek to understand the different funding allocation decisions within these programmatic areas, as well as changes in funding over time, through a review of PEPFARâs operational plans and a sample of the COPs. Key informant interviews during country visits will be a complementary source of information. What is the responsiveness and adequacy of psychosocial care services to meet the needs of eligible HIV-positive children and adolescents orphaned or made vulnerable by HIV/AIDS? PEPFAR recommends countries collect data on the number of orphans and vulnerable children provided with psychological, social, or spiritual support. The committee will rely on qualitative data to assess the âresponsiveness and adequacyâ of these services as programs do not provide relevant quality data to address these specific issues. To address PEPFARâs efforts in this core OVC program area, the committee will try to evaluate the responsiveness, screening, and referral of children and adolescents to appropriate psychosocial support providers at different stages of need. Efforts will also be made to evaluate the engagement of expert guidance by country programs in determining best practices for addressing the psychosocial needs of children and adolescents orphaned or made vulnerable by HIV/AIDS. To what extent have PEPFAR-supported OVC programs had an effect on the educational enrollment of orphans and vulnerable children and adolescents?
112 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Are PEPFAR efforts supporting policy or programs aimed at mitigating the consequences of children and adolescents, particularly young women, having increasing socioeconomic responsibilities in the household due to the illness of a parent or both parents living with HIV/AIDS? To what extent are programs averting orphanhood and over what period of time? What economic support programs are in place to generate income for adult non-parental caregivers to support orphans and vulnerable children and adolescents in their households? The committee will review available data to assess other cross-cutting socioeconomic interventions such as vocational and technical training, small-business development, and household economic-strengthening workshops. The committee will need to access data disaggregated by age and sex in order to assess the effects of PEPFARâs economic strengthening programs on the socioeconomic needs of these groups at different developmental stages, particularly in young women. What activities does PEPFAR support for in-service training and pre-service training for child and adolescent service providers and facilities? The long term benefits of PEPFAR on the well-being of children and adolescents, including those orphaned or made vulnerable by HIV, will largely depend on the capacity of country programs to assume responsibility for lessening the negative effects of HIV/AIDS in the communities where they live. Thus, the committee will assess qualitative and quantitative data to evaluate whether PEPFAR programs are investing in building the capacity of district and national authorities and communities to respond to the needs of these vulnerable groups. What is the current state of child protection legislation in the community? Are activities supported by PEPFAR contributing to changes in implementation and enforcement? Some of the PEPFAR-supported activities include training workshops on childrenâs rights, as well as will writing and succession planning for children, their guardians, and for PEPFAR program staff. PEPFAR programs also link children, adolescents, and their guardians to appropriate legal services. The committee will seek to assess the effects of PEPFAR activities on current or proposed legislative reforms that incorporate child and adolescent protection measures and support protection programs mainly through primary collection of qualitative data during country visits and a review of publicly available documents. To what extent have PEPFAR policy initiatives contributed to changes in national health priorities for children and adolescents? Are PEPFAR efforts resulting in the development and adoption of national strategic plans for orphans and vulnerable children? What has been the progress of the country programs to implement the plans?
113 CHILD AND ADOLESCENT WELL-BEING A primary measure will be whether PEPFAR programs for children and adolescents orphaned or made vulnerable by HIV/AIDS are an integral part of national HIV/AIDS strategies and plans. The committee will review a range of policy documents (see Box 2a-d) and conduct interviews to assess whether these strategies and/or planning activities are designed to engage different sectors or agencies within the national government dedicated to child and adolescent well-being. The committee will seek any available data from OGAC and governments such as partnership framework implementation plans.
SECTION 7: GENDER-RELATED VULNERABILITY AND RISK The global HIV/AIDS epidemic cannot be fully addressed without considering the ways in which societal gender norms, which can be described as âhow societies define acceptable and customary roles, responsibilities, and behavior of women, girls, men, and boys,â contribute to womenâs and menâs vulnerability in the epidemic (AIDSTAR-One, 2009, p. 1). For example, gender inequalities can limit the ability of women to exert control over their sexual choices, while societal norms may encourage men to engage in riskier sexual behavior. Norms can inhibit access for both men and women to obtain testing and treatment due to, for example, fear of reprisal by a partner or the nature of service provision. In addition, socially defined responsibilities may lead to an increased burden, typically for women, associated with caring for individuals affected by HIV and AIDS (UNIFEM, 2006). A focus on gender inequality and how to reduce it thus has been recognized to be an essential ingredient of HIV prevention, treatment, and care. Current priorities among stakeholders in the global HIV/AIDS community in reducing gender-related vulnerabilities are focused on a variety of country-level strategies. These include the passage and implementation of legislation to protect women, girls, and other vulnerable groups (such as MSM) from gender-based violence, bias, discrimination, and stigma; programs to address male norms; and efforts to empower women through increasing their income generation opportunities (AIDSTAR-One, 2009; UNAIDS, 2010a; UNIFEM, 2006). This section provides a brief background on gender-related programming in the context of PEPFAR and outlines the committeeâs approach to assessing PEPFARâs efforts toward addressing these complex issues. However, it is important to note that unlike other aspects of addressing HIV and AIDS, issues associated with gender-related risk and vulnerability cut across all of PEPFARâs implementation, and do not currently comprise their own unique programmatic or funding category. In addition, while ongoing efforts to highlight the differential needs of men and women have been underway since the beginning of the program, addressing issues associated with gender norms, violence, and stigma are still in the early stages in many PEPFAR countries. PEPFARâs Efforts on Gender to Date The original legislation authorizing the creation of PEPFAR and the first Five-Year Strategy acknowledged the importance of addressing the unique vulnerability of women and girls, most notably through prevention efforts, but did not outline any specific targets for addressing gender-related issues (OGAC, 2004). Recognizing the centrality of gender inequality in the AIDS epidemic, PEPFAR subsequently convened a gender TWG in 2005 and submitted a report to Congress on gender-based violence and other gender programming activities in 2006, which emphasized the need to address challenges beyond those associated with access to care, including gender norms and linking services to reduce HIV risk among victims of gender-based violence (OGAC, 2006f). The LantosâHyde Act of 2008 expanded the programâs mandate with respect to women and called for the development of a plan to address the particular vulnerability of women and girls and improve research in areas related to gender programming. Working from the reauthorization legislation, the gender TWG also emphasized the following five strategic areas to guide program development and implementation (OGAC, 2009c): 115
116 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS 1. Increasing gender equitable access to prevention activities and services 2. Reducing violence against women, coercion, and the exploitation of women and girls by sex trafficking, rape and sexual abuse and providing post-rape prophylaxis 3. Addressing male norms and behaviors 4. Increasing womenâs legal rights and protections 5. Increasing womenâs access to income and productive resources The recently released new PEPFAR Five-Year Strategy reaffirmed the programâs commitment to gender equity in its prevention, treatment, and care services and to scaling up programs to address gender-based violence (OGAC, 2009g). It also commits the program to work with countries in establishing initial targets and reporting mechanisms to track outcomes for activities targeted at gender needs (OGAC, 2009h). Objective and Scope The IOM evaluation committee will examine the progress made toward incorporating the ideals laid out by the reauthorization legislation and Five-Year Strategy into program guidance, planning, and budgeting processes. The committee will also assess the extent to which PEPFAR has increased its gender programming to include activities aimed at reducing gender-based violence, addressing male norms, empowering women through behavior change and income generating activities, and improving equity in access to prevention, treatment, and care services. Program Impact Pathway The following impact pathway (Figure 15) illustrates the committeeâs understanding of how activities consistent with PEPFARâs stated overarching gender goals could lead to outputs, outcomes, and health impacts in the countries in which PEPFAR operates. As described in the previous section, efforts to address gender-related issues are still being developed in many countries and are not incorporated in a systematic manner into country program planning. Thus, the activities listed do not represent PEPFAR-wide initiatives, but rather serve as examples of programs that might be undertaken, or are already underway as pilots in some PEPFAR countries. Due to these realities, the committeeâs assessment will focus primarily on the process of developing gender-related activities and, where possible, the outputs achieved from their implementation. However, the ability to access data on outputs is expected to vary substantially by country and may not be feasible in many settings. The committee may also pursue data from external sources (including from multilateral partners such as UNAIDS), existing studies, and qualitative analysis during country visits.
FIGURE 15 Program impact pathway for evaluation of PEPFARâs gender efforts. 117 NOTES: PHE = public health evaluations; PMTCT = prevention of mother-to-child transmission.
118 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions The following are illustrative examples of the types of questions related to gender activities that the committee may assess given the timely availability of quality data. These questions are intended to contribute to addressing part B, item vii of the areas for consideration in the congressional mandate. 44 There are currently no PEPFAR reported, required, or recommended indicators that are specific to measuring efforts toward changes in gender norms, vulnerability related to gender, or empowerment. However, a number of prevention, treatment, and care indicators are required to be disaggregated by sex and/or age, which can provide information related to access rates and equity in service provision. In addition, due to potential variability in the definition and understanding of gender-related activities across programs, there will likely be substantial variation both within and between countries. As this represents an evolving area of programming, case studies may provide useful information at this stage. Are women and girls accessing prevention, treatment, and care programs in equal numbers to men and boys? Sex-disaggregated PEPFAR indicators (such as the number of people accessing prevention, treatment, and care programs) will be used where available. The committee may also assess whether these services are being delivered in a comparable and appropriate manner to women and men, using qualitative data collected during country visits. Has PEPFAR made special efforts to reduce the sexual risk of adolescent girls and young women (e.g., through early marriage, transactional, or cross-generational sex, etc.)? Has PEPFAR made efforts to address the sexual risk of older women (e.g., through wife inheritance by relatives of late brothers, cleansing rites, etc.)? Have these efforts been successful? As with assessing male norms, the committee may use structured interviews with key informants from the OGAC, program, and project levels to determine the incorporation of efforts to reduce womenâs sexual risk into program planning and implementation. When available at the country level, the committee will pursue access to a limited number of recommended PEPFAR indicators, including those related to cross-generational sex. In addition, there are a number of global level indicators collected by organizations and survey efforts (including UNAIDS, national demographic and health surveys, and behavioral surveillance surveys) that provide information regarding this topic (Measure DHS, 2010). The committee will assess the feasibility of incorporating information from these types of sources to provide contextual data on the countries in which PEPFAR operates. How many programs integrate or are linked to programs that seek to economically empower women? What has been their impact in improving womenâs income, savings, or access to credit? 44 (B)(iii) an assessment of efforts to address gender-specific aspects of HIV/AIDS, including gender related constraints to accessing services and addressing underlying social and economic vulnerabilities of women and men
119 GENDER-RELATED VULNERABILITY AND RISK Currently, activities that promote economic empowerment and income generation are included under the umbrella of care services provided by PEPFAR, and indicators such as ânumber of people provided with a minimum of one care serviceâ disaggregated by sex may provide some information as to the existence of these programs. However, in order to gain a more comprehensive picture of the success and coverage of these types of activities, the committee will seek other sources of data at the project level where available. How many programs has PEPFAR supported to address male norms and behaviors? What criteria are used to determine whether such programs should be a part of a COP? How effective have these programs been in fostering more gender equitable attitudes among men? The availability of activity type data at the OGAC level will determine the committeeâs ability to assess the efforts of PEPFAR toward addressing male norms. Interviews with OGAC, country, and project level staff may be used to determine how these types of activities are addressed in the program planning and reporting process. However, there are no PEPFAR indicators for measuring impact of programs to change male behavior or norms. Thus, the evaluation committee will use project level data where it exists. How many country programs include programs to reduce gender-based violence? How and to what extent have they succeeded in reducing incidence of violence? There are no PEPFAR indicators specific to measuring gender-based violence, so the committee will use country data and project level data where it is available. Data may also be gathered from key informants who run these programs. In addition, the committee will attempt to follow-up at the OGAC level on the âReport on Gender-Based Violence and HIV/AIDSâ that was submitted to Congress in 2006, to assess the potential of updating the information presented. Has PEPFAR made efforts to influence national policy and legal environments that affect the vulnerability of women and girls? How have they been successful? What impact have these efforts had on decreasing this vulnerability? Efforts toward positively influencing policy and legal reform efforts will be assessed through interviews with country team and OGAC staff. In addition, for the countries for which they have been completed, the newly formed Partnership Frameworks may be reviewed for the presence of gender-related strategies as they are intended to reflect national political and legal priorities. Given the often protracted nature of policy implementation and enforcement, an evaluation of the potential impact of these efforts may not be feasible.
SECTION 8: KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Reauthorization Legislation Shifts Priority to Sustainability The LantosâHyde Act of 2008 specifically required PEPFARâs strategy to âinclude a longer-term estimate of the projected resource needs to progress toward greater sustainability and country ownership of HIV/AIDS programs [â¦] during the 10-year period beginning on October 1, 2013.â 45 The new PEPFAR Five-Year Strategy stated that management of the response to the disease and its effects must increasingly be led by countries, with support from bilateral and multilateral partners. Countries also need to increasingly own the process of monitoring, evaluating, and responding to the unique characteristics of the epidemic in their country (OGAC, 2009g). This section therefore outlines how the evaluation committee will explore the role of PEPFAR programs in facilitating improvements to health systems in partner countries and the readiness of PEPFAR and partner countries to increasingly share responsibility for managing the response to the epidemic and move to greater sustainability and country ownership. Definitions Neither the authorizing legislation nor the PEPFAR strategy defines sustainability. For the purposes of this evaluation, one of several definitions proposed by the OECD Development Assistance Committee may be used; it defines sustainability as âthe continuation of benefits from a development intervention after major development assistance has been completedâ (DAC Network on Development Evaluation, 2002). While continuation of benefits into the future is the ultimate goal, a number of intermediate outputs or outcomes can be posited to improve sustainability: Affordability: The extent to which countries can bear the cost of programs. Efficiency/cost-effectiveness: âA measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to resultsâ (DAC Network on Development Evaluation, 2002, p. 21). Country capacity: The ability of government, the private sector, and civil society to âplan, manage, implement, and account for results of policies and programsâ (High-Level Forum, 2005, p. 5). 46 Country âownershipâ: A situation in which âpartner countries exercise effective leadership over their development policies and strategies, and coordinate development actionsâ(High- Level Forum, 2005, p. 3). Coordination and harmonization 47 with donors and governments: The extent to which donors âimplement, where feasible, common arrangements at country level for planning, funding (e.g., joint financial arrangements), disbursement, monitoring, evaluating and reporting to government on donor activities and aid flows.â (High-Level Forum, 2005, p. 6). 45 Supra., note 6 §101(a), 22 U.S.C. §7611(a)(29). 46 The Paris Declaration does not specify whose capacity within countries this defines, but it is inferred to be the governmentâs capacity. Thus, this proposed definition is somewhat broader. 47 As the extent to which PEPFAR has contributed to harmonization is being evaluated by the U.S. Government Accountability Office, it will not be explicitly addressed in this proposed evaluation. 121
122 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Partnership Frameworks to Promote Sustainable Approaches The LantosâHyde Act of 2008 permitted the USG to establish framework documents (Partnership Frameworks) with countries to promote a more sustainable approach of the USGâs global efforts against HIV/AIDS, malaria, and TB âthat is characterized by strengthened country capacity, ownership, and leadershipâ (OGAC, 2009b, p. 3). Further, these 5-year joint strategic frameworks between the USG and partner governments are meant to intensify focus on cooperation through âtechnical assistance and support for service delivery, policy reform, and coordinated financial commitmentsâ (OGAC, 2009b, p. 3). At the end of the 5-year time frame, the expectation is that âcountry governments will be better positioned to assume primary responsibility for the national response to HIV/AIDS in terms of management, strategic direction, performance monitoring, decision making, coordination, and where possible, financial support and service deliveryâ (OGAC, 2009b, p. 3). The axiom of âdo no harmâ has been adopted by OGAC for continued support of existing implementing partner service delivery systems to continue to provide quality services while this transition to county ownership occurs over time. There are additional expectations of transparency, accountability, and engagement of multiple stakeholders in the country. As for policy reform and financial commitments, the Partnership Frameworks are supposed to emphasize policy areas identified by the government and civil society that require additional or focused attention and overall accountability for resources and appropriate budgeting in HIV programs. They also provide a capacity building opportunity for the USG to assist countries in managing multiple funding sources. Some countries, based on their resources, are expected to increase their financial contributions over time. The Partnership Frameworks also provide an opportunity for the USG to provide technical assistance to countries for improved monitoring and tracking of overall health spending (including HIV/AIDS) from different sources, including financial monitoring and reporting systems (OGAC, 2009b). Although the reauthorization legislation did not define sustainability per se, PEPFAR did define how to promote sustainability in the Partnership Frameworks and Partnership Framework Implementation Plans guidance issued in September 2009âbased on the principles of the Three Ones, the Paris Declaration, and the Monterey Accord of 2002 (activities that facilitate financing for development): âFor purposes of the Partnership Frameworks, promoting sustainability means supporting the partner government in growing its capacity to lead, manage, and ultimately finance its health system with indigenous resources (including its civil society sector), rather than external resources, to the greatest extent possibleâ (OGAC, 2009b, p. 4). The Partnership Frameworks are distinct from the annual work plans for USG-supported intervention, the COPs. However, COPs are expected to reflect the Partnership Framework principles and the transition strategy outlined in the Partnership Framework Implementation Plans. The implementation plans have minimal required elements, including âan analysis of how the existing portfolio of USG- supported, NGO-implemented programs will transition to the partner government, remain NGO- based, or be terminated within the 5-year timeframeâ (OGAC, 2009b, p. 6) and âa description of the approach to supporting increased country ownership, baseline data, specific strategies for achieving the 5-year goals and objectives, and a monitoring and evaluation planâ (OGAC, 2009b, p. 8). It also seems reasonable that cost efficiencies for the future national response can be identified by both PEPFAR and the partner government during the implementation of the Partnership Frameworks and by the end of the 5-year performance period, when countries are
123 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES expected to assume primary responsibility for the national HIV/AIDS response, even if the partner government does not assume full responsibility of financing its health system. The goals and objectives would be measureable goals for the USG and all partners in the Partnership Framework. As such, the Partnership Frameworks are expected to identify indicators to assess progress toward meeting the goals, objectives, and programmatic and financial commitmentsâwith an eye toward international efforts to harmonize indicators. PEPFAR- specific reporting systems are expected to be transitioned to nationally- and country-owned systems. This step would be in full support of the Third Oneâone agreed HIV/AIDS country- level M&E system (OGAC, 2009b). Lastly, the guidance also states that country governments should be developing the âcapacity to support all relevant components ⦠of a multi-sector health systemâ (OGAC, 2009b, p. 4). These components are the six areas of the WHO six-building block framework for effective health systems (see Figure 16) that has been endorsed by PEPFAR. Program and Health System Interaction and Integration The most widely accepted definition of a health system was proposed by the WHOââall organizations, people, and actions whose primary intent is to promote, restore, or maintain healthâ (WHO, 2007a, p. 2). A health system therefore includes both public and private (for- profit and not-for-profit) providers, which may be either community- or facility-based. Identifying both well-planned synergies and unintended antagonisms with national health systems that may have resulted from the implementation of a program like PEPFAR requires a systems model, as interventions in one part of the health system may have impacts in others. The WHO Positive Synergies working group took such an approach, resulting in several recommendations that are relevant to PEPFAR: prioritize health system strengthening, agree on and track health system strengthening indicators, align resource allocation between global health initiatives and country health systems, generate more reliable data for the costs and benefits of strengthening health systems, and commit to increased national and global health financing that is more predictable to support sustainable and equitable growth of health systems (Samb et al., 2009). While global HIV prevalence stabilized in 2000 (Bongaarts et al., 2008; UNAIDS, 2008), the absolute burden of HIV treatment demand will continue to grow each year for the foreseeable future. (UNAIDS, 2008; UNAIDS and WHO, 2009). With newly-infected people still outpacing both AIDS-related deaths and numbers of people being put on ART, it is increasingly clear that treatment alone will not be a sufficient response to control the epidemic (Bertozzi et al., 2009). Achieving sustainable HIV programs not only requires health system strengthening, but also successful scale-up of effective HIV prevention strategies to avert continued growth in the HIV treatment burden in health systems that are already overburdened. As previously mentioned in the Mapping of PEPFAR Funding section, scaling-up does not necessarily mean just increased spending, so understanding unit costs of HIV-program delivery may help elucidate variation in health system capacity, efficiencies, and quality. Also program and data management and health information systems are needed to more completely assess population impact rather than just process measures.
124 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Capacity Building As has been noted, the capacity to scale up programs in low-income countries is limited by resource constraints, including lack of trained health workers and fragile health systems. PEPFAR II therefore added an additional target of training 140,000 new health workers to support the capacity of countries to improve âthe overall quality of their services ⦠and build capacity to plan, manage and sustainably finance their health systemsâ (OGAC, 2009h). Investing in healthcare workers and health systems increases the likelihood of more people receiving prevention and treatment services for HIV and of achieving a broader health impact (PEPFAR Reauthorization Action Team, 2010). Joint Activities Between PEPFAR and the Global Fund As seen in the statement of task in Appendix A, Congress mandates the committee to evaluate the impact on health of prevention, treatment, and care efforts that are supported by U.S. funding, including multilateral and bilateral programs involving joint operations. Further discussions among IOM staff, OGAC, and congressional staff 48 clarified âmultilateral and bilateral programs involving joint operationsâ to mean programs operated in conjunction with bilateral funding through PEPFAR and the Global Fund (Bressler, 2009; Marsh, 2009). In the preliminary research of the planning committee, this includes financing of ART and procurement and supply management of ARVs and other commodities for its Voluntary Pooled Procurement (discussed in the Adult and Pediatric Treatment section of this report), as well as prevention and care activities (PMTCT and TB treatment, respectively discussed in the Prevention and Care and Support Services sections). It also includes time-limited, outcome-oriented technical assistance through centrally-funded grants from OGAC to Global Fund recipients with active grants (which are not necessarily PEPFAR recipients). The USG Global Fund Technical Support Advisory Panel advises OGAC headquarters on these technical assistance activities. Begun in 2005 under the Grants Management Solutions project, this technical support is intended to (1) improve the functioning of Global Fund grants, (2) strengthen local capacity, and (3) alleviate specific bottlenecks to address under-performing Global Fund grants. The main areas identified with inadequate or poor performance were organizational development (including governance and leadership), financial management, procurement and supply management, and M&E (Coleman, 2007; PEPFAR, 2009b). These four areas correspond to four of the six blocks in the WHO building-block model for effective health systems (see Figure 16). There are stated goals for improvement for each of the four areas of technical assistance. In July 2008, a USG Global Fund Technical Support Evaluation was conducted, which showed achievements through 2008 in each of the areas from the $12 million available in FY2005 technical support through the $31 million available in FY2007 (Coleman, 2010). The committee will examine the 2008 evaluation, including its methods and findings and will also request available country/recipient progress reports as well as copies of the grant requests from recipient countries. 48 Supra., note 10.
125 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Evaluation Strategy As part of an evaluation of the impact of PEPFAR, it is necessary to determine partner country readiness to make this transition to sustainability. To assess partner country readiness, the evaluation committee aims to examine the partnership frameworks with partner countries. In addition, the committee will assess PEPFAR efforts and synergy with other global stakeholders for capacity building and technical assistance, including activities similar to those identified by the International Health Partnership, such as donor funding harmonization and collaboration between international actors and developing countries to develop and implement national health plans. The committee will also establish working definitions, define data sources, and identify evaluation methodologies for the key parameters of sustainability that include country ownership and local capacity building, including health system strengthening and healthcare workforce expansion. However, it has been noted that âlittle [global] consensus has emerged to provide uniform guidanceâ to indicators that can track these activities, although the development of these indicators is an area of intense international interest and activity. PEPFAR defines indicator and reporting requirements in health systems strengthening âto reflect a more narrow scope of interest tied to PEPFARâs focus on HIVâ (OGAC, 2009d, p. 199)âresulting in selection of two indicators to be reported centrally to OGAC that reflect laboratory and health workforce strengthening. Evaluating data for other parameters will require metrics not currently available from routine data sources or the PEPFAR indicator database (OGAC, 2009d), including extra- health sector factors such as good governance (Dybul, 2009), to permit the committee to assess performance of current or past PEPFAR activities. The committee will also examine Partnership Frameworks and Implementation Plans to assess what the countries and OGAC are responsible for, how it is being measured or tracked, and how the processes evolve for PEPFAR-related responsibilities and activities to be transitioned to country leadership for sustainable programs and positive impacts on individual and population health. Health System Frameworks PEPFARâs increased focus on health systems is shared with other global health initiatives, including The Global Alliance for Vaccines and Immunisation (2010) and the Global Fund, as well as the World Bank and the WHO. The committee will benefit from and incorporate deliberations among these agencies that will address both how to define health systems and how to measure progress toward strengthened health systems (Frenk, 2010; Shakarishvili, 2009). Adopting a common conceptual framework is a key requirement for the evaluation of health systems strengthening and is important to framing such measures. The most widely adopted framework, proposed by the WHO in 2007, is based on operational âbuilding blocksâ of the health system (Figure 16)âservices, workforce, information, commodities and technologies, financing, and leadership/governance (WHO, 2007a). These building blocks serve several functions including describing what a health system should have the capacity to do in each of these blocks to lead to improved health, system responsiveness, improved efficiency, and social and financial risk protection. It also lends itself well to a descriptive set of activities that may be undertaken through PEPFAR programs. The importance of âsystems thinkingâ has also been emphasized, because health systems are complex, dynamic, and non-linear systems whose
126 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS function is dependent on the interplay of all of its components, including âthe spaces in betweenâ (Atun and Menabde, 2008; de Savigny and Taghreed, 2009). FIGURE 16 Representation of WHOâs six building blocks for effective health systems. SOURCE: Adapted from WHO, 2007a.
127 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Program Impact Pathway The committee will utilize an impact pathway (Figure 17) to assess PEPFAR inputs, output, and outcomes where measurable, within the context of the WHO building blocks framework for health systems strengthening. The pathway can also help assess the same block elements for technical assistance and other activities related to country readiness for assuming increased to total responsibility for their HIV/AIDS response. Although a useful evaluation tool, the linear nature of an impact pathway is a simplified representation of the reality of PEPFAR programs and their impact, and the committee found this to be particularly challenging in the area of systems-level activities. Activities at the systems level are intended to result in an outcome or impact on the health system, but these are also critical inputs to all other programmatic areas. Thus, the functioning of the health system can be both a starting and end point for a pathway. The committee grappled with illustrating the complexities, dynamism, and non-linear nature of health systems with this linear pathwayâanalogous to forcing a âsquare peg in a round holeâ âbut ultimately recognized the utility of the impact pathway to evaluate the process by helping to frame the areas of inquiry and the measures that may be undertaken to assess health system strength. The findings will be interpreted in light of the more complex realities when the evaluation committee draws conclusions and makes recommendations. As previously mentioned, PEPFAR II (as well as the GHI) has adopted the WHO six building block framework to assess its capacity building and programmatic impact on health systems, categorizing contribution to (1) core HIV activities, (2) secondary benefits or intentional spillover effects of PEPFAR activities on other programs, and (3) targeted leveraging including partnerships. The six building block elements are denoted in the figure with blocks in all uppercase letters. The committeeâs working definition of an intervention that strengthens health systems is one that improves the activities and processes within the six health system building blocks, and manages interactions within the âbuilding blocksâ over time to achieve more equitable and sustained improvements in population health outcomes. These effects may be either short-term (e.g., better trained workforce delivering higher quality care) or longer-term (e.g., higher-quality care results in a healthier population that is economically more productive), thus growing resources that feed back into the causal chain as an input to the health system. Additional outcomes of an effective health system (e.g., social and financial risk protection for beneficiaries accessing and utilizing services or increased block or system efficiency) can result in increased responsiveness and potentially expanded service coverage when the activities within and among the blocks dynamically interact (see Figure 17). As previously mentioned, interventions in one part of the health system may have impacts in others. Since any intervention may affect the entire system, evaluation is, by definition, context-specific and requires a mix of evaluation strategies, both quantitative and qualitative. Although improved health is an important impact of health systems strengthening, additional measures regarding health system efficiency and equitable distribution of benefits may likewise be important to track as they are developed and adopted by international working groups.
128 FIGURE 17 Program impact pathway for evaluation of PEPFARâs health systems strengthening activities at the country level. NOTES: Upper case headers indicate the WHOâs six building blocks. All of the elements listed in the pathway can be categorized under one of the six building blocks, but due to space limitations, select elements were chosen to illustrate blocks within the pathway. ARV = antiretroviral drugs; ART = antiretroviral therapy; CD4 = cluster of differentiation 4; EQA/PT = external quality assurance/proficiency testing; M&E = monitoring and evaluation.
129 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Illustrative Questions The main questions that the committee will consider in the evaluation include the potential positive and negative impact of PEPFAR activities on country-level health system functioning, with regards to both HIV and non-HIV programs. Examples of the kinds of questions the committee may ask that specifically link the WHO six building blocks to our results chain framework include the following. These questions are intended to contribute to addressing part B, item ii of the areas for consideration in the congressional mandate. 49 Because health systems are a fundamental aspect of all program activities, these questions are also intended to contribute to addressing all of the areas for consideration described in the Statement of Task (see Appendix A). Finance: To what extent has PEPFAR funding and technical assistance for sound public finance systems resulted in more efficient and equitable care in HIV and non-HIV health systems? To what extent do the joint funds from the Global Fund support the six building blocks? To what extent have the technical support funds from PEPFAR to the Global Fund improved the performance of grant recipients in general and how is this measured? For the four main areas of technical assistance which are among the six WHO building blocks and how are they measured? Commodities and Procurement: How have PEPFAR technical assistance and training affected HIV laboratory diagnostic capacity, pharmacy infrastructure, and supply chain management for reagents and drugs? Information Systems: What is the evidence that PEPFAR-supported health information systems are resulting in higher functioning, quality-driven health systems performance? How will new and existing information officers be trained to meet country needs to strengthen higher functioning and increased quality? Service Delivery: What elements of more efficient, equitable, and effective service delivery should be expected to result in improved population health over time with better integration of HIV- and non-HIV health care and use of continuous quality improvement methods? Leadership and Governance: To what extent will partnership frameworks, jointly funded Global Fund activities, and other expressions of country ownership lead to improved and accountable governance as well as transfer of oversight, management, guidance, and financing for HIV-related services in health systems? Health Workforce: How will the pre-service education target of 140,000 new workers affect health system equity of access (e.g., rural versus urban) and health system strengthening such as increased skill capacity and retention of workforce? Are there other types of workers, (e.g., social service workers and program managers) who also need to be trained? What proportion of newly trained new health or social workers is 49 (B)(ii) an assessment of the effects on health systems, including on the financing and management of health systems and the quality of service delivery and staffing
130 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS retained one year later? How does PEPFAR support the countryâs existing health resources development plan or help develop those where absent? The committee will also assess progress toward the goals of increased country ownership to transition the HIV/AIDS response to national governments for long-term sustainability. Although there are high-volume discussions within the global development assistance community in this area, there appear to be few to no meaningful metrics to measure increased ownership or country readiness, no accepted logic models to describe and illustrate the transition process, and no indicators for when a country has sufficiently strengthened governance/ leadership or built and operated a financially-sound public finance system to implement and oversee a national health plan. Illustrative questions for this area of the evaluation include the following. Country ownership and sustainability cut across all PEPFAR activities and as such these questions are intended to contribute to addressing all of the areas for consideration in the congressional mandate, as described in the Statement of Task (see Appendix A). How is country ownership defined by the country? By PEPFAR? By relevant global stakeholders in aid development? Are there differences? Have they been reconciled? Do the Partnership Frameworks represent an âagreementâ on the definition between the country and PEPFAR? To what extent have PEPFAR capacity-building, technical assistance, and financing activities to countries contributed to country readiness for transitioning the knowledge management, decision making responsibility, financing, and accountability/oversight of the PEPFAR-funded HIV/AIDS response to the national government? How is it being measured? Are plans in place for future activities needed to improve or increase country readiness if PEPFAR were absent? How will achievements or milestones by country and/or PEPFAR be measured in the Partnership Frameworks? Will these measures adequately reflect country readiness for program, policy, and financial transitions? Is the country transitional process explained adequately? Will it be standard for all countries or adapted for each country? Will the transition be parceled over time? Which pieces might be transitioned over time, how, and why? What are the achievements and lessons learned from the Technical Support Grants to the Global Fund from PEPFAR? How are they measuring whether bottlenecks have been opened or bypassed? What has been the impact on the functioning of programs? How do Global Fund and PEPFAR measure their relative and unique contributions to programs they jointly fund or operate? How are access, equity, and quality measured for these programs? A more extensive listing of illustrative questions for this complex issue that the committee may attempt to address, given the timely availability of quality data, can be found in Appendix H. These may help guide assessment of whether and how PEPFAR has helped
131 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES countries to mount a stronger response to their HIV epidemics, plan for transition to country ownership including effective leadership and oversight of multilateral donor activities, and focus on affordable care that meets population health needs. Sustainability and its associated elements must be considered for all of PEPFARâs support. PEPFAR activities will be evaluated relative to their contribution of inputs and support of processes, and the resulting outputs and outcomes. These should ultimately result in the desired outcome of stronger health systems that can more adequately respond not only to HIV but to other serious causes of morbidity and mortality, as well as any emerging challenges to the health and safety of their respective populations.