Strategic Approach to the Evaluation of Programs Implemented Under the Tom Lantos and Henry J. Hyde U.S. Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008 (2010)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

PART III Illustrative Evaluation Details for Assessment of PEPFAR’s Performance and Impact 



The IOM was mandated by Congress in the Lantos–Hyde Act of 2008 to conduct a study that includes an assessment of the performance of U.S.-assisted global HIV/AIDS programs and an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by U.S. funding. Part of the charge to the planning committee in developing a plan for this evaluation was to be cognizant of the requirements and charges mandated for the evaluation (see Appendix A). To augment the overview of the evaluation design presented in Part II, this part of the plan partitions and elaborates the areas of interest laid out in the congressional mandate. The guiding framework of the program impact pathway is applied to each of these areas, reflecting the committee’s understanding of the rationale for how PEPFAR’s specific inputs and activities can be plausibly linked to PEPFAR’s contribution to effects on HIV-specific health outcomes and impacts. This part of the report illustrates the types of questions that will guide the evaluation of PEPFAR’s activities in prevention, adult and pediatric treatment, care and support, child and adolescent wellbeing (including orphans and vulnerable children), and gender-related vulnerability and risk activities. The evaluation will also consider other fundamental activities in the areas of knowledge management and funding flows; these are considered first in this part of the report because they underlie the success of all other programmatic areas. This part of the report culminates with a discussion of cross-cutting activities related to key systems-level goals that are critical for the long term goals articulated by PEPFAR, such as health systems strengthening and transitioning to sustainability and country ownership. As described in Part II, in each of these areas the evaluation questions will be addressed using a mixed methods approach and layers of investigation and analysis, drawing on a range of available primary and secondary data sources. By applying a mix of methods, data sources, and analytical techniques, the committee will arrive at findings that can be triangulated to draw conclusions about the performance of PEPFAR and its contribution to health impact, even when any one data source is not sufficient or any one methodological approach is not feasible. The extent to which specific methods can be applied to answer the evaluation questions will depend on the timely availability of data that is of sufficient quality to lead to reliable findings. Therefore, the illustrative questions and the methods and data sources that will be used to address them will undergo further refinement and prioritization as a result of the operational planning phase activities described previously. 51 



SECTION 1: PEPFAR’S KNOWLEDGE MANAGEMENT The management of knowledge and information is critical to the success of any program because it serves to monitor the activities and effects of the program as well as to guide policies, priorities, and programmatic decisions. Therefore, assessing the performance of PEPFAR’s knowledge management activities will be an important part of the evaluation of the performance of PEPFAR as a whole, as well as an assessment of the forward-looking mechanisms that are in place for continuous M&E of the program’s progress and appropriate responses. In PEPFAR I, the primary goal of OGAC’s strategic information (SI) efforts for M&E, in partnership with implementing agencies, was to support PEPFAR through results-based planning and implementation, focusing on quality information collection, timely data management and use, evaluation of best practices, and information for decision making. In PEPFAR II, the goals have been expanded to include support of the larger PEPFAR mission. To this end, the expanded SI mission includes building the capacity of countries to improve health outcomes by increasing and strengthening the use of information for effective stewardship of programs and effective implementation of efficient, high-quality, and sustainable health systems (Bouey, 2010). During PEPFAR II, the SI goals also include improved harmonization of USG reporting needs with country-driven M&E efforts through not only strengthening country capacity and alignment with national data collection, but also through better alignment with global reporting requirements to lessen the burden on implementing partners and partner governments (OGAC, 2009h). An increased focus on both program coverage and quality will be reflected in SI efforts to identify indicators that can give an accurate picture of these two areas (OGAC, 2009d). Finally, although it continues to recognize that PEPFAR is not intended to be a research initiative, the PEPFAR Five-Year Strategy outlines the additional goals of improving the program’s efforts to contribute to the evidence base for HIV interventions and to expand the amount of publicly available data (OGAC, 2009g). This expanded research effort will prioritize the evaluation and proactive dissemination of topics that PEPFAR is in a unique position to address as well as studies that focus on methods to improve program delivery (OGAC, 2009h). It will also increase the tracking of outcomes, cost-effectiveness, innovation, and impacts in order to identify timely information regarding the program’s effectiveness and impact (OGAC, 2009h). Strategic Information Management Structurally, SI activities at the PEPFAR headquarters level are carried out by USG implementing agencies with coordination through OGAC. These headquarters-level activities draw from a wide range of data-gathering sources (see Figure 5). 53 

54 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS FIGURE 5 PEPFAR headquarters-level strategic information partners and headquarters-level data- gathering sources. NOTES: CDC = U.S. Centers for Disease Control and Prevention, Census = U.S. Census Bureau, DoD = U.S. Department of Defense, HRSA = Health Resources and Services Administration of the U.S. Department of Health and Human Services, M&E = monitoring and evaluation, OGAC = Office of the U.S. Global AIDS Coordinator, SI = strategic information, TWG = technical working group, USAID = United States Agency for International Development. SOURCE: Adapted from Bouey (2010). The staff at headquarters are responsible for issuing guidance related to COP submission and reporting processes, as well as for providing technical assistance. This guidance instructs country teams on how to successfully complete their reporting requirements. In addition to annual reporting needs, guidance from headquarters can also provide information on collecting, interpreting, and updating basic epidemiologic profiles as well as information on how to develop and incorporate efforts to evaluate new initiatives (OGAC, 2009e). While this guidance focuses primarily on the processes unique to OGAC, it also offers additional information regarding data collection and target setting that has the potential for broader applicability. Within headquarters there is also a SI technical working group. These efforts at the headquarters level also support M&E activities at the country level through the development of resources such as an M&E Systems Strengthening Tool, which is designed to help partner countries prioritize their M&E needs and encourage alignment with a national M&E strategy, and a Data Quality Assessment Tool (MEASURE Evaluation, 2007; PEPFAR, 2008a). While management of data collection, storage, and analysis at the country and project level varies, the primary mechanism for reporting and aggregating these data is via an electronic, Internet-based program known as the Country Operational Plan Reporting System (COPRS). Data that are reported via COPRS are collected at the OGAC headquarters level during the relevant semi-annual or annual program reporting periods, depending on the country (OGAC, 2009d). A portion of these data is released to the public and is also communicated to Congress via an annual report. Occasionally, they are used by OGAC to produce additional topic-specific reports (e.g., Report on Gender-Based Violence and HIV/AIDS) (PEPFAR, 2010c). COPRS is 

55 PEPFAR’S KNOWLEDGE MANAGEMENT currently in transition due to two factors. First, the Next Generation Indicators Reference Guidance was recently released, introducing a limited number of new indicators and redefining some measures that had previously been in use. This guidance was developed in part to support PEPFAR’s contribution to global efforts to harmonize reporting requirements for HIV/AIDS initiatives, which aims to reduce the reporting burden of program implementers and to allow more flexibility and increased local ownership of the design of M&E plans (OGAC, 2009d). Second, a new generation of COPRS, COPRS II, is in development and is expected to be deployed in FY2010 (OGAC, 2009e). Beyond Information Management In 2008, PEPFAR began a campaign titled “know your epidemic/know your results” aimed at using information to more closely align program activities with population needs (OGAC, 2008b). As a result, a focus was placed on developing sustainable SI systems to “collect, analyze, critically review, disseminate, interpret, display, and strategically use data at all levels” (OGAC, 2008b). Continuing with this development of SI, the 2009 headquarters operational plan allocated funds for the development of a “draft knowledge management strategy,” perhaps in response to the first IOM PEPFAR evaluation recommendation to develop a detailed, overall strategy for institutionalizing its efforts to function as a learning organization and to increase its contributions to the global knowledge base (IOM, 2007; OGAC, 2009f). Coordination among country staff and dissemination of best practices is also facilitated by the PEPFAR implementers’ meeting held annually in a PEPFAR country. In addition to sharing information across countries, this meeting includes a variety of breakout sessions dedicated to SI issues (PEPFAR, 2009a). In addition, as described above, the recent goals for PEPFAR II emphasized the important role of expanding the program’s research portfolio to contribute to the publicly available evidence base, with an emphasis on operations research to improve program delivery as well as methods for timely assessments of the program’s effectiveness and impact (OGAC, 2009h). Some research activities are already occurring in individual partner countries. For example, the Public Health Evaluations (PHE), initiated in 2007, are a PEPFAR activity intended as a source to inform policy and program-level changes. They currently serve as the primary mechanism through which PEPFAR supports research within countries, including operations research (Edgil, 2010; OGAC, 2009h). Some PHEs are single country, while others are multi-site investigations. The selection of PHE proposals (annually solicited from investigators) is performed by an interagency technical policy group charged with prioritizing areas in need of evaluation, overseeing the implementation of evaluations, and recommending approvals and levels of funding for PHEs. In doing so, priority is given to studies that are driven by locally- identified country needs as well as those that involve local institutions and investigators in the research process. 15 As of 2008 there were 195 PHE activities (Edgil, 2010), with the most recent call for proposals issued by the National Institutes of Health in April 2010. 16 15 Personal communication from OGAC, April 9, 2010. 16 Ibid. 

56 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS PEPFAR’s Monitoring and Evaluation Framework To evaluate PEPFAR’s performance in the area of knowledge management, the committee will be guided by the public health questions approach (see Figure 6), a framework that is widely used in the global HIV/AIDS M&E community and has been adopted by OGAC (Bouey, 2010; Rugg et al., 2004; UNAIDS MERG, 2010). The committee will determine the extent to which M&E activities are meeting the goals laid out in this framework and the extent to which these activities are contributing to evaluating and improving the performance of the program and building the capacity of partner countries to use information to improve health outcomes. FIGURE 6 A public health questions approach to HIV monitoring and evaluation. SOURCE: Reprinted, with permission, from UNAIDS MERG (2010). Illustrative Questions The evaluation of PEPFAR’s SI activities will be carried out at the level of headquarters and in those partner countries where PEPFAR has made major investment in M&E. In order to accomplish the assessment, the committee will use a desk review of OGAC and country-level M&E strategy and implementation procedures and data management systems since the beginning of PEPFAR (i.e., including different iterations over time). The committee will also examine M&E funding allocations and expenditures at global and country levels where available. A review of national M&E strategies, national M&E assessment reports, and reports on the HIV epidemic and response, including global reports such as United Nations General Assembly Special Session (UNGASS) documents or M&E data from UNAIDS’s National Composite Policy Index, will also provide context for the evaluation. Due to the limitations in assessing the progress of efforts over time from guidance and assessment documents, the committee will also rely on structured interviews with key SI and programmatic PEPFAR staff at all levels (OGAC, USG agencies, OGAC SI technical working groups, country staff, contractors/implementers) regarding the mechanisms and role of SI in the PEPFAR program and in informing the national HIV response. The committee may also explore structured interviews with key M&E and programmatic staff of each country’s national AIDS program as well as other multilateral and bilateral organizations (including UNAIDS, WHO, the 

57 PEPFAR’S KNOWLEDGE MANAGEMENT Global Fund) about the role of PEPFAR as a partner at the local, national, and global levels. These interviews could be incorporated as a part of the committee’s country visits or conducted by phone or video conference (or alternative self-completed questionnaire) for those countries where a country visit is not planned. The committee’s analysis of the completeness and validity of data requested from OGAC and implementing partners for all areas of this evaluation will also inform an assessment of SI and M&E performance and progress. In order to assess PEPFAR’s contribution to the global knowledge base, the committee will assess PEPFAR’s participation in international M&E development processes. The committee will also assess current research efforts, such as the PHEs, and associated dissemination efforts, such as presentations from PEPFAR programs at the annual implementers’ meeting and other international conferences like the meetings of the International AIDS Society. In addition, a preliminary search of published literature conducted during this planning phase will provide the foundation for a more extensive review during the evaluation of available articles, reports, and other publications resulting from PEPFAR-funded activities. The following are examples of illustrative questions that the committee may consider in the evaluation. These questions related to knowledge management reflect a fundamental activity of the program and as such are intended to contribute to addressing all of the areas for consideration in the congressional mandate, as described in the Statement of Task (see Appendix A). To what extent has investment in M&E resulted in effective systems for PEPFAR decision making and for program management and improvement at both the headquarters and country levels? Is data collection and analysis being used for decision making about PEPFAR program priorities, implementation strategies, effectiveness, and efficiency? For example, is PEPFAR using data to support evidence-based COP planning and resource allocation? Does data collection lead to timely identification of implementation problems, and does this result in corrective action? Does data on targets determine whether programs are implemented on a large enough scale to have an impact on the epidemic? Are the data collated, analyzed, interpreted, presented, and disseminated in a manner that allows for use in decision making? What are the mechanisms used to assure the validity and quality of data? What mechanisms are in place to facilitate the translation of information produced by M&E systems into action? Are lessons learned accessible and are changes applied across the whole of the program where appropriate? Have operations research and other research activities supported by PEPFAR, such as PHEs, had an impact on service delivery and led to improved outcomes of prevention, treatment, and care programs supported by PEPFAR? Are operations and other research activities using appropriate methodologies and resulting in information that is shared across sites, programs, and countries to optimize and inform policy and program decisions? Are these research activities addressing the issues most in need of evaluation? What should the priorities be for future PHEs and other research activities? 

58 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS What has been the impact of PEPFAR reporting requirements for accountability purposes (i.e., reporting to the U.S. Congress) on continuation of PEPFAR funding and on decision making for priority investments as well as program management and improvement? What progress has been made on PEPFAR’s intentions to develop indicators where there are currently limited mechanisms for tracking progress, such as gender and health systems strengthening? Are there sufficient M&E mechanisms and capacity to evaluate whether the program meets new goals for sustainability and country ownership, as well as the resulting outcomes and impact of changes made to address those goals? Does OGAC draw on sources of data outside of PEPFAR to inform programmatic and policy decisions? To what extent is PEPFAR contributing to the global knowledge base? Are PEPFAR-funded activities resulting in research that is contributing to the scientific knowledge base? Are research findings, lessons learned, and best practices from PEPFAR available in the published literature? What other mechanisms are used to disseminate knowledge not only within but also beyond PEPFAR? To what extent is that process encouraged or facilitated? To what extent is PEPFAR engaging with other international stakeholders around SI activities? To what extent is PEPFAR contributing to the development of state-of-the- art practices in M&E at the global level? To what extent has PEPFAR built/is PEPFAR building capacity at the country level, including national M&E systems, 17 to support an appropriate, effective, and efficient national HIV response? What is PEPFAR’s approach to supporting long-term sustainability of the national M&E system? How is PEPFAR translating the commitment of the United States to the “Third One” adopted by most donors—one national M&E system to reduce reporting burden and transactional costs of business for countries with multiple donors? To what extent is the PEPFAR M&E strategy aligned with and incorporated in the national M&E strategy/plan? What are the positive and negative effects of the headquarters-level PEPFAR M&E strategy on national M&E systems? To what extent and how are PEPFAR M&E data (program planning, routine program 17 National M&E system refers to M&E at the national, sub-national, and service-delivery levels. 

59 PEPFAR’S KNOWLEDGE MANAGEMENT monitoring, findings from special studies) shared with the national M&E system to ensure a coordinated HIV response and to guide program improvement? What mechanisms are used by PEPFAR for M&E capacity building and to ensure effective partnerships for technical cooperation and technology transfer? What are the effects of the PEPFAR M&E capacity building activities on national M&E system strengthening and data use for decision making? To what extent has PEPFAR built/is PEPFAR building M&E capacity within partner organizations implementing programs at the country level, including data analysis and management? 



SECTION 2: MAPPING PEPFAR FUNDING All donor-specific impact evaluations are limited by the ability to attribute desired impacts to the investments of a single program given the presence of similar and complementary programs funded by other sources, so the committee’s evaluation will focus on PEPFAR’s contribution to global efforts to fight HIV/AIDS (IOM, 2008). PEPFAR inputs include policy guidance and regulations, personnel, technical assistance and training, facilities and equipment, knowledge transfer and research, and funding. While all are important to assessing PEPFAR’s impact, the level and allocation of funding, as well as how the funding is used, underpins much of the evaluation. It both represents the most direct measure of the USG’s investment to address the global AIDS pandemic and provides a critical input needed for answering many of the other evaluation questions pertaining to the impact of PEPFAR. These include questions about PEPFAR’s impact on specific health targets, such as reducing HIV incidence, and its impact on broader mandates and goals, such as promoting long-term sustainability, country ownership, and health systems strengthening. As such, financing is a cross-cutting aspect of the overall evaluation, embedded within each of the other main evaluation questions. To evaluate the impact of PEPFAR’s financial investments on desired program impacts, the committee will develop a funding flow framework for PEPFAR financing, designed to map funds throughout the life cycle of the program. It will begin with Congress, which appropriates funding to federal agencies, and follow funding provided by federal agencies to prime partners and other implementers in an attempt to map all the way to service providers in the field and ultimately to beneficiaries. The objective of this financial framework is not to conduct a financial audit but to assess the role of U.S. funding in the context of the overall evaluation of PEPFAR’s impact on health and other outcomes and to illustrate the specific points at which the USG intervenes, how it intervenes, and the level of its intervention. Figure 7 represents the committee’s initial understanding of the landscape of PEPFAR funding flows, which may change as more information is accumulated throughout the evaluation. 61 

62 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS FIGURE 7 PEPFAR funding landscape. NOTES: Dept. = department, Govt = government, HHS = U.S. Department of Human Health and Services, NGOs = non-governmental organizations, USAID = U.S. Agency for International Development, USG = U.S. government. SOURCE: Committee assessment based on a review of documents from OGAC, including PEPFAR operational plans and partner information, as well as other readily available sources. 

63 MAPPING PEPFAR FUNDING The evaluation will also situate PEPFAR investments within the larger landscape of investments made by other funders to allow for an assessment of PEPFAR’s relative financial contributions to global efforts to combat HIV/AIDS. This is important because nearly all of the countries that receive PEPFAR funding also receive funding for HIV/AIDS programs from other sources (see Table 1 for an illustration of the current funding landscape in PEPFAR countries). The committee will therefore develop another financial framework to situate PEPFAR within the broader funding landscape, designed to capture PEPFAR’s relative contribution globally and by country. It will include other donor governments, multilateral institutions, partner country governments and households, and the private sector. 

64 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS TABLE 1 Current HIV/AIDS Funding from Select Donors Received by PEPFAR Countries (as of April 2010) Multilateral Bilateral Country Global Fund World Bank CIDA DFID Angola X X X Botswana Cambodia X X X China X X Congo, Democratic Republic of the X X X Côte d’Ivoire X X X Dominican Republic X X Ethiopia X X X X Ghana X X X X Guyana X X Haiti X X India X X X X Indonesia X X Kenya X X X X Lesotho X X X Malawi X X X X Mozambique X X X X Namibia X X Nigeria X X X Russia X X Rwanda X X X X South Africa X X X Sudan X X X Swaziland X X X Tanzania X X X X Thailand X Uganda X X X X Ukraine X X Vietnam X X X Zambia X X X Zimbabwe X X X NOTE: An X denotes that a country receives or has recently received approval for an HIV/AIDS grant from the Global Fund, or has at least one currently operational HIV/AIDS program funded by the World Bank (excluding IBRD-financed loans), the Canadian International Development Agency (CIDA), or the UK Department for International Development (DFID). CIDA and DFID are representative of other bilateral funders of global HIV/AIDS programming, and the UK in particular is the second largest bilateral donor to HIV/AIDS (Kates et al., 2009). SOURCE: Compiled from GFATM (2010a), World Bank (2010), CIDA (2010), and DFID (2010). 

65 MAPPING PEPFAR FUNDING Finally, since the data used to populate the financial framework are critical inputs for each program area, the committee will attempt, where feasible, to use these data to answer questions surrounding program efficiency, economies of scale, costs of intermediate units of analysis (e.g., the unit cost of outputs, the costs per beneficiary), and costs per intermediate outcomes (e.g., school enrollment, change in condom use, persons on treatment, and child infections averted from PMTCT, etc.). Ultimately, the framework proposed here will be designed to provide policymakers with a direct measure of PEPFAR’s impact on the global HIV/AIDS epidemic. Illustrative Questions The committee will consult a variety of data sources including PEPFAR and USG agencies (e.g., OGAC databases, COPs, partnership frameworks, prime and sub-prime partner reports, etc.), other donors and international stakeholders (UNAIDS, the World Bank, the Global Fund, Organisation for Economic Co-operation and Development [OECD], etc.), partner countries, and the relevant evaluation literature. To identify data sources and availability, the committee will need to conduct structured interviews with relevant stakeholders including USG representatives, implementers, partners, and others. Data collection and quality will be a primary potential limitation since much of the analysis is predicated on being able to obtain timely data from these sources. The complexities of mapping financing flows from multiple donors to multiple countries will also complicate the ability to access consistent and complete longitudinal data. Given the timely availability of quality data, the following are examples of illustrative questions that are feasible to address. These questions related to the flow of funding reflect a fundamental aspect of program operations and as such are intended to contribute to addressing all of the areas for consideration in the congressional mandate, as described in the Statement of Task (see Appendix A). How much PEPFAR funding is provided and what is its relative contribution to global efforts against HIV/AIDS? Where possible, the committee will use longitudinal data to analyze PEPFAR’s relative contribution over time and determine whether PEPFAR has been able to leverage additional resources from other donors and national governments. To further describe the contribution of PEPFAR funding, the committee will attempt to determine the proportion of the national budget and the gross domestic product that would need to be spent on health and HIV/AIDS if partner governments were to take on these expenditures. How is PEPFAR funding provided to partner countries and USG partners? Initially, the committee will analyze COPs and partnership frameworks to identify a general mapping of funding flows and seek to characterize countries according to a sub-set of different types of models or frameworks. The committee aims to determine whether PEPFAR funding is provided through bilateral or multilateral channels and which mechanisms are used to deliver funding, such as cash transfers or commodities. To further describe funding cascades, the committee will attempt an analysis of PEPFAR’s disbursement procedures to determine how 

66 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS quickly funding is provided to the field once it has been appropriated by Congress, obligated to countries, and disbursed to partners. Who are the recipients of PEPFAR funding at each stage of program? As depicted in Figure 8, PEPFAR channels funding through governments, NGOs, academic institutions, and private contractors that may be based in the United States, in partner countries, or in other countries. The committee will seek to analyze the proportion of funding that flows to each type of recipient and analyze longitudinal trends. A preliminary committee analysis of FY2008 obligations to prime partners identified three main sub-types when stratified by geographic location and five sub-types when stratified by geographic location and organizational type, as follows (listed in order of share of investment): Prime Partner Funding Flow Sub-Types by Geographic Location 1. United States 2. Domestic/Partner Country 3. Other/International Prime Partner Funding Flow Sub-Types by Geographic Location and Organizational Type 1. U.S. NGO 2. U.S. Academic 3. Domestic/Partner Country NGO 4. Domestic/Partner Country Government 5. U.S. for Profit Firms The relative distribution of funds by agency origin (United States versus partner country) and by sector (government, NGO, academia, private sector) over time also will be assessed. Understanding these patterns may help elucidate the degree to which the targeting of funding supports, or potentially undercuts, the goal of increasing country ownership of HIV programming. The criteria for providing grants to local implementing partners and how they are monitored may also contribute to an assessment of whether this process contributes to the goals of increasing country ownership. In addition, the performance of the prime partner organizations in terms of accountability, administrative transparency, and good governance could be additional metrics to consider in the evaluation. Finally, assessing the distribution of funding in this way may help to provide data and information on the amount of funding that actually reaches the field after accounting for intermediaries along the way. 

67 MAPPING PEPFAR FUNDING FIGURE 8 PEPFAR FY2008 obligations to prime partners: U.S., local, or other location stratified by key sector. NOTES: Totals do not equal 100 percent due to rounding. Govt = government, Intl = international, NGOs = non-governmental organization, Orgs = organizations. SOURCE: Preliminary committee analysis based on FY2008 OGAC data on obligations to prime partners available at http://www.pepfar.gov/partners/index.htm. How are PEPFAR funds distributed among different programmatic areas and interventions? How are funds from PEPFAR and the Global Fund meshed together in countries? Are the joint funds maximized for service delivery and coverage? The committee will seek to measure the proportion of funding spent in each program area, such as prevention, treatment, care and support, and health systems. Within each program area, the committee will attempt to identify the level of funding for specific interventions and analyze PEPFAR’s efforts to allocate funding appropriately based on country needs and available evidence. In order to analyze whether efficiencies are being achieved, the committee 

68 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS will attempt to determine the unit costs of key PEPFAR interventions and how they vary by type of intervention, country, and other variables of interest. Which populations are supported through PEPFAR funding? In order to evaluate the impact of PEPFAR funding, the committee will seek to determine the specific populations, such as women, children, and other vulnerable populations, reached with program interventions. The committee will use existing data from reliable sources to measure the number of people living with HIV/AIDS (PLWHA) and compare to the number of vulnerable people reached through PEPFAR programming. The number and types of populations supported by PEPFAR funding will be measured through quantitative analysis of PEPFAR indicators. Aggregated data reported to OGAC provide limited insight about the types of populations supported, so the committee will seek disaggregated data from other sources. 

SECTION 3: PREVENTION SERVICES It has been estimated that a new HIV infection occurs globally every 12 seconds, and there is agreement that in most countries incident infections are outpacing the number of people placed on ART (Global HIV Prevention Working Group, 2010; UNAIDS and WHO, 2009; Zachariah et al., 2010). At the same time, HIV prevention budgets have been falling in several countries. Currently there is an increasing push by the global community to scale up effective, evidence-based prevention strategies with the knowledge that averting new infections is the only way to bend the curve of the epidemic. Information about new behavioral, biomedical, and structural prevention approaches is growing and individual countries, regions, or localities are being encouraged to incorporate all of the approaches into a comprehensive strategy. PEPFAR is collaborating with WHO and UNAIDS in these efforts and the program’s prevention goals, if attained, are intended to help address these deficits. This section describes the current state of the program’s prevention efforts and some of the potential questions and challenges under consideration by the committee to evaluate the extent to which PEPFAR I and II have met their prevention targets and goals. In addition to PEPFAR I’s original prevention performance target, there was also the country target of reducing expected HIV incidence by 50 percent (IOM, 2007). In PEPFAR I, the prevention target was to be achieved through five funding and reporting subcategories: Abstinence/be faithful, Condoms and other prevention (e.g., IDU, military forces, street children), PMTCT, Blood safety, and Injection safety (IOM, 2007). The prevention activities undertaken that correspond to those funding and reporting categories were: promotion of behavior change aimed at risk avoidance and reduction, comprehensive programs for people who engage in high-risk behavior, PMTCT services, and reduction of medical transmission of HIV by ensuring safe blood supplies (OGAC, 2004). PEPFAR II promotes a transition to increasingly country-owned HIV programs that adequately address their HIV epidemics over time, including expanding prevention in both concentrated and generalized epidemics. The are several targets for PEPFAR II: to support the prevention of more than 12 million infections over the course of the entire program (since 2003), 18 to ensure that every partner country with a generalized epidemic has both 80 percent coverage of testing for pregnant women at the national level19 and 85 percent coverage of ARV drug prophylaxis and treatment of women found to be HIV-infected (OGAC, 2009g), to double the number of at-risk babies born HIV-free from the 240,000 babies of HIV-positive mothers who were born HIV-negative during the first five years of PEPFAR (OGAC, 2009g), and lastly to provide 100 percent of youth in PEPFAR prevention programs, in every partner country with a generalized epidemic, with comprehensive and correct knowledge of the ways HIV is transmitted and ways to protect themselves, consistent with Millennium Development Goals (MDGs) indicators in this area (OGAC, 2009g). The Lantos–Hyde Act of 2008 requires the Coordinator to establish a balanced HIV sexual transmission prevention strategy to govern expenditures for prevention activities in countries with generalized epidemics. This “balanced funding” directive replaces the “one-third abstinence or be faithful” budget earmark requirement in the original 2003 legislation. Instead of identifying a specific requirement for the distribution of sexual prevention funds, countries are 18 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(i). 19 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iv). 69 

70 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS now required to provide a compelling explanation, justified by the Coordinator, if less than 50 percent of this funding is directed toward activities promoting (a) abstinence, (b) delay of sexual debut, (c) monogamy, (d) fidelity, and (e) partner reduction. The Coordinator is also required to annually publish, and make available to the public, a report that details the progress toward implementation of this prevention strategy (OGAC, 2009e). Similarly, research and dissemination of best practices in prevention methodology have also been identified as priorities for PEPFAR II (OGAC, 2009i). In addition to the performance targets, PEPFAR II has also established additional overarching goals for prevention, such as strengthening partner country capacity to generate and use timely, accurate, and up-to-date epidemiological information (emphasizing prevention efforts targeted toward most vulnerable populations); encouraging use of innovative strategies for data mapping and service provision; and developing goals for each individual prevention category (OGAC, 2009h). Objective and Scope Evaluation of PEPFAR’s efforts toward prevention and assessment of any impacts they may have had are essential to understanding if PEPFAR is meeting its targets of averting more than 12 million new infections. The committee will evaluate the extent to which PEPFAR is effectively engaging in scale-up of combinations of evidence-based HIV prevention intervention. The categories of PEPFAR prevention interventions are shown in Table 2. 

71 PREVENTION SERVICES TABLE 2 Categories of PEPFAR Prevention Interventions Behavioral Structural Biomedical Administration of antiretroviral Improving awareness Increasing access to Prevention of drugs during pregnancy, at birth HIV testing and Mother To Child and acceptability of and breastfeeding antiretroviral drugs Transmission HIV testing and antiretroviral drug use during pregnancy in pregnancy Screening blood prior to transfusion Blood Safety Education about (in) Developing a national appropriate use of transfusion policy transfusion Ensuring continual access to blood screening supplies Injecting Drug Youth education to Increasing access to Methadone and buprenorphine Use reduce injecting drug drug treatment services use initiation or clean needles Sex education in Male circumcision Sexual Counseling to reduce schools to delay sexual concurrency, debut inconsistent condom use, HIV education for youth Incorporating trained Testing partners to detect Prevention in Counseling to counselors in treatment discordancy and identification and People Living increase condom use centers treatment of sexually transmitted with HIV/AIDS and reduce infections and opportunistic concurrency infections NOTE: Needle or syringe exchange programs (NSPs) have been shown to be an evidence-based method to reduce HIV and other blood-borne pathogens (IOM, 2006). The ban on domestic funding for NSPs was lifted by the U.S. Congress as part of the FY2010 appropriations process. No similar ban was codified for international assistance accounts (Consolidated Appropriations Act, 2010, P.L. 3288-2, 111th Congress, 1st Session, [January 6, 2009]). However, “foreign affairs programs, including PEPFAR, have traditionally followed the domestic policy guidance regarding NSPs” (Personal communication from OGAC, January 8, 2010). OGAC reports that this lifted ban creates an opportunity for PEPFAR’s MARP TWG to explore ways to support NSPs as part of a comprehensive package of services for IDUs and to develop a revised set of comprehensive guidance for PEPFAR programs and teams so that NSPs can augment prevention, treatment, and care activities already provided to this population (Personal communication from OGAC, January 8, 2010). Gateway Activities Targeting Prevention That Will Be Examined and Used For Analysis In addition to specific interventions, there are additional gateway activities related to PEPFAR’s prevention efforts. These gateway activities do not, in and of themselves, reduce the number of new infections, but are critical to targeting prevention services and monitoring the effects and impact of interventions on the prevention goal of reducing new infections. They include HIV testing and counseling in both the community and clinic settings and surveillance activities in general and sub-populations. 

72 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Challenges in Prevention Implementation Implementation of programs for HIV prevention face many challenges related to context, culture, and information. The gateway activity of surveillance and the use of spatial mapping tools (e.g. Geographical Information Systems) to describe or visually represent epidemics are critically important, as many countries do not have a clear picture of their epidemic at the national, regional, or local levels. While a sufficient understanding of the nature of the epidemic can more effectively guide programmatic decisions to target appropriate population groups, in some countries multiple types of epidemics are occurring simultaneously both within and across different regions. This poses challenges during the development of an HIV prevention strategic plan and requires more complicated combinations of activities that increase the number of logistic and coordination barriers to be managed. Finally, varying types of stigma and discrimination can restrict the ability of prevention programs to reach at-risk and vulnerable populations such as IDUs, CSWs, and women and girls. Each of these issues provide context when evaluating the areas of success and failure in PEPFAR’s prevention efforts (OGAC, 2009i). Program Impact Pathway PEPFAR’s prevention efforts will be assessed in accordance with a program impact pathway framework, as shown in Figure 9. The evaluation will draw on a variety of data sources, including outputs and outcomes as measured by PEPFAR countries and OGAC. For estimating the impact measure of incidence, the committee will consider emerging globally accepted proxy measures, such as prevalence in young people, as well U.S. Census Bureau and other modeling efforts. The committee may also use available data to consult with experts who model infections averted for PEPFAR in particular. Since prevalence is affected by incidence, duration (longevity should increase with more people getting on ART), and population growth, the committee will be interested to know how the modelers will parse out these relative constituents. While there is concern about relying too heavily on these statistical forecasts, the committee feels that models have utility. They will be examined taking into account their limitations, which are primarily determined by the quality of the data available to do the modeling. The evaluation may also assess the contribution that PEPFAR funds made to achieve impact at the country level by undertaking a trend analysis of incidence figures over time and comparing that to key policy or program or funding events that occurred over the same time period. 

FIGURE 9 Program impact pathway for evaluation of PEPFAR’s prevention programs. NOTES: ARVs = antiretroviral drugs, IDU = injecting drug user, PMTCT = prevention of mother-to-child transmission. 73 

74 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions To assess whether both PEPFAR I and II have achieved their prevention goals, the committee will not only evaluate whether they met their quantitative targets, but also assess whether in doing so, they provided quality services, met the needs of the countries in which programs were implemented, and met their own overarching goals (coordination, integration, health system strengthening, and building capacity) as articulated in the strategy for PEPFAR II (OGAC, 2009g). The evaluation of PEPFAR’s prevention efforts, outputs, and outcomes is subject to a variety of limitations including the quality and timely availability of quality data. In addition, as a part of OGAC’s Next Generation Indicators project, many of the indicators centrally collected by OGAC that are applicable to prevention efforts were modified or revised at the end of 2009, which may affect the ability of the committee to assess temporal trends or compare original PEPFAR countries with those added more recently. Each individual topic also contains its own challenges and limitations that may limit the committee’s analysis; these are discussed below. The following illustrative questions are intended to contribute to addressing part A, item ii and part B, items i and v of the areas for consideration in the congressional mandate.20 Are prevention strategies for a country defined based on an epidemiological analysis and evidence of what works? Has PEPFAR supported routine behavioral and epidemiological surveillance at the country level to identify sub-national variations in at-risk groups and used these data to target prevention interventions to appropriate groups (most-at-risk populations, other vulnerable populations, youth and adolescents, adults)? Is PEPFAR appropriately and efficiently incorporating emerging evidence on prevention techniques into its programmatic guidance? Surveillance strategies can include both biomedical and behavioral measures to determine disease prevalence (using HIV testing data including antenatal and at-risk populations), seroincidence (using “detuned” assays such as BED 21 in population surveys), and risk behavior prevalence (via Demographic and Health Surveys [DHS] or “second generation” surveys). The frequency with which these types of data-gathering methods occur, the methods through which PEPFAR supports their implementation, and the dissemination and use of the data obtained will be assessed by the committee. This will be done by relying on publicly available data sets from DHS and other population surveys as well as qualitative data gathered through site visit interviews. PEPFAR indicators and programmatic guidance related to prevention efforts will be coupled with data from other sources on the nature of the national epidemics and on internationally-accepted best practices for HIV prevention. Studies on the efficacy of new 20 (A)(ii) an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by United States funding, including multilateral and bilateral programs involving joint operations (B)(i) an assessment of progress toward prevention, treatment, and care targets (B)(v) an evaluation of the impact of prevention programs on HIV incidence in relevant population groups 21 Supra., note 12. 

75 PREVENTION SERVICES approaches to HIV prevention such as pre-exposure prophylaxis (PrEP), microbicides, and vaccine are under way, therefore these types of interventions would not be part of this evaluation (OGAC, 2009i). How does the distribution of funding for prevention activities compare to the epidemiology of the local epidemic? What is PEPFAR’s progress toward the development and implementation of a prevention strategy and balanced funding portfolio? How is prevention funding, for countries and OGAC, aligned with elements of the prevention strategy? As a backdrop to the evaluation of outcomes and impact, the evaluation may also determine the percentage of funds spent by PEPFAR I and II (by year) on prevention overall, by subcategories, by region, and within countries. The subcategory expenditures can be examined by the committee to determine whether they match with the estimated populations at risk in the country to provide an assessment of the country’s ability or willingness to adhere to the guidance of OGAC in “know your epidemic/know your results” and to fund correspondingly appropriate activities. This new guidance from OGAC has resulted in headquarter-suggested realignment of funds in the country when there are significant mismatches (Moloney-Kitts, 2009). In addition, the committee will attempt to determine the funding for each region and country by the prevalence by year and by sub-population. Expenditure studies should also compare the level of investment in prevention activities and the types of programs supported by country and by implementing partners, as well as whether these factors are associated with achievement of prevention objectives. Given the sensitivity surrounding programmatic financial data, the committee anticipates there will be difficulty regarding availability for a complete assessment. The committee will seek whatever data and information are available from OGAC to assess the progress on the development and implementation of the described prevention strategy and its programmatic and budgetary elements, as well as the interaction between this legislative guidance and PEPFAR’s stated goal of mapping each country’s epidemic and aligning prevention responses to the identified needs (OGAC, 2009g). Within each country, have HIV prevention programs reached the performance targets for prevention for both PEPFAR I and II? Is coverage of each service or a combination of services expanding over time? Progress toward the targets previously outlined will be assessed primarily using aggregated indicators that are reported to OGAC. These are available for the majority of the targets. However, for some areas, such as behavior change communication (BCC), additional PEPFAR indicators not reported to OGAC will also be required in order to address the question more completely. The committee will also attempt to assess PEPFAR’s progress toward its stated goals in each of the specific prevention areas. For many of these goals for which no reported indicators exists at the aggregated headquarters level at OGAC, the committee will focus on qualitative primary data collection during site visits and will seek out additional quantitative data where available. Has the number of people who have been counseled and tested and received their results increased over time? 

76 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Have counseling and testing services been offered in an equitable way to women (pregnant and non-pregnant) and men, to young people as well as adults, and to locally- identified vulnerable populations? To address PEPFAR’s efforts toward expanding its counseling and testing services, the committee will analyze the reported PEPFAR indicators regarding the number of people who receive counseling and testing services and their results. While this indicator is disaggregated by sex, it may be insufficient to identify if programs are reaching all of the at-risk populations in need or if counseling and testing programs are successfully linking with intervention facilities. Where this is the case, the committee will pursue other sources of quantitative and qualitative data. What are the effects of the behavioral interventions (education and awareness, condom education and distribution, peer education, sex and HIV prevention education, group and individual behavioral interventions, etc.) that are being implemented? BCC programs occur in multiple settings and may be targeted to general population participants or specific at-risk groups. The committee will use indicators aggregated and reported to OGAC regarding number and types of individuals reached with BCC programs, as well as the disaggregation of these indicators where available. These indicators cover the range and frequency of behavior change programs, but do not sufficiently address the effectiveness of these programs with respect to whether the participants actually changed their behavior following the activity. To try to look longitudinally at prevalence as a reflection of the outcome of behavior change efforts, the committee will examine the utility of data from the antenatal population. Potential data sources include large international NGOs that have programs in many PEPFAR countries and therefore have very large cohorts across multiple countries. Some data may also be obtained through consultation with modelers. However, in general the committee will have limited ability to address this effectiveness issue. What has been the coverage and outcome or impact of biomedical prevention interventions, which include blood safety, PMTCT, PEP, and male circumcision? How has PMTCT been maximized with jointly operated activities between PEPFAR and the Global Fund? How has coverage expanded as result of joint efforts? Implementation and tracking of biomedical prevention programs varies significantly and targets a broad range of individuals at all ages. Data for the evaluation of biomedical prevention interventions will be obtained from aggregated OGAC level indicators and other sources at the country and local levels, as there are not reported PEPFAR indicators for blood safety, PEP, and male circumcision. Challenges associated with evaluating these interventions also include emerging knowledge and best practices that will have to be incorporated in the future and may alter the trajectory of affected programs. This information may be supplemented by qualitative data collected during site visits. Are successful linkages being developed between PEPFAR prevention services and other services where appropriate in order to better meet the needs of targeted individuals and to increase coverage and efficiencies when available? For example, are women identified 

77 PREVENTION SERVICES as HIV-positive and HIV-exposed infants in PMTCT programs successfully referred to and enrolled in treatment programs? Prevention services have the potential to serve as entry points to identify those in need and increase access for treatment and other services. Optimal implementation of PMTCT programs, for example, can not only reduce perinatal transmission to 2 percent with the use of combination ART, regimens but also serve as a linkage to provide HIV-positive women with ART (Cunningham et al., 2002; Dorenbaum et al., 2002; Kuhn et al., 2008). As there are currently no centrally reported indicators for “linkages or referrals,” the committee will rely primarily on recommended measures such as “the number of health facilities providing antenatal care that provide both HIV and PMTCT services on site” as well as information collected during country site visits. The committee may also attempt to triangulate, where available, the information collected with indicators on PMTCT services coverage and ART coverage for women in specific regions, in order to gain insight into PEPFAR’s progress toward its goal of 85 percent prophylaxis and treatment of pregnant women found to be HIV- positive. Has PEPFAR implemented effective prevention programs for injecting and other drug users (behavioral, biomedical, and structural interventions and approaches) in regions/countries where drug use drives the epidemic? What has been their effect? What has been the coverage and are interventions evidence-based? The committee will use the reported PEPFAR output indicator of “number of [IDUs] on opioid substitution therapy” and the recommended indicator of “percent of [IDUs] on opioid substitution therapy” where available. However, as more information will likely be required to perform a sufficient evaluation, the committee will also pursue alternative data sources. Issues of stigma and confidentiality pose challenges to program implementation and data collection as the population of IDUs is often hidden or difficult to access. In order to increase access to services and adoption of testing and preventive behaviors, what efforts has PEPFAR made to reduce stigma through community programs or campaigns or policy initiatives? What has been their effect? Structural and policy approaches to reduce stigma and vulnerability are important components of comprehensive HIV combination prevention, but their outcomes, effects, and impact are difficult to evaluate. The committee will use PEPFAR indicators that are disaggregated by sex and testing location (to avoid obscuring the potential male dominance in results outside of maternal and child health settings) where available and supplement these with qualitative data collected during site visits. However, given the lack of data availability, the potential for analysis of interventions to reduce stigma and vulnerability may be limited. 



SECTION 4: ADULT AND PEDIATRIC TREATMENT Recognizing the need for delivering life-saving treatment to the millions of people living with HIV as a global health emergency, PEPFAR I strategically focused on the rapid scale-up of HIV treatment services and an increase in the coverage of HIV-infected individuals in need of ART (IOM, 2007). As of September 30, 2009, PEPFAR has contributed to the support of treatment for 2,485,300 people, of which 8 percent are children under 15 years of age, through bilateral programs in countries (OGAC, 2010). As shown in Table 3, in 2008 PEPFAR reached the legislative 5-year target of assuring treatment for 2 million people (PEPFAR, 2009c), which included treatment for 131,500 children under 15 years of age (OGAC, 2009c). Since this contribution “has been the major success of PEPFAR” scale-up efforts in the early years (OGAC, 2009i, p. 26), PEPFAR II (2009–2013) plans will focus on supporting a country-led response to the HIV/AIDS epidemic, especially assisting countries in identifying resources available, increasing country coverage, and prioritizing the unmet needs for ART for PLWHA. TABLE 3 Cumulative Number of People on Antiretroviral Treatment by PEPFAR Country PEPFAR Five- Year Target (2003-2008)a Country FY2005 FY2006 FY2007 FY2008 b Botswana 33,000 37,300 67,500 90,500 111,700 77,000 50,500 Côte d’Ivoire 11,100 27,600 46,000 Ethiopia 210,000 16,200 40,000 81,800 119,600 Guyana 2,000 800 1,600 2,100 2,300 Haiti 25,000 4,300 8,000 12,900 17,700 Kenya 250,000 44,700 97,800 166,400 229,700 Mozambique 110,000 16,200 34,200 78,200 118,000 Namibia 23,000 14,300 26,300 43,700 56,100 67,100c Nigeria 350,000 28,500 126,400 211,500 Rwanda 50,000 15,900 30,000 44,400 59,900 South Africa 500,000 93,000 210,300 329,000 549,700 Tanzania 150,000 14,700 44,300 96,700 144,100 Uganda 60,000 67,500 89,200 106,000 145,000 Vietnam 22,000 700 6,600 11,700 24,500 Zambia 120,000 36,000 71,500 122,700 167,500 Total 2,000,000 401,200 822,000 1,358,500 2,007,800 NOTES: Numbers reflect totals of downstream (direct) and upstream (indirect) results. a Based on PEPFAR’s Congressional Report to Congress of persons on antiretroviral therapy in the original 15 focus countries as of September 30, 2008. b Botswana results are attributed to the National HIV Program. Beginning FY2006, U.S. Government (USG) downstream contributions in Botswana are embedded in the upstream numbers, following a consensus reached between the USG and the Government of Botswana to report single upstream figures for each relevant indicator. c In Nigeria, it is currently unknown if the government's number of people on treatment accounts for people who are lost to follow up, therefore the total number of people on treatment had been reduced by 15 percent to account for the estimated attrition. SOURCE: PEPFAR (2009c). 79 

80 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS For treatment programs, the host government may use Global Fund, other donor, or country resources to support specific components of services in some PEPFAR treatment sites, while PEPFAR supports other essential components of the treatment services. Beyond these joint operations of treatment sites by PEPFAR and other donors, other support to PEPFAR specific sites may be national or regional in nature (PEPFAR and USAID, 2007). At the same time, in countries where multiple funders (e.g., PEPFAR and the Global Fund) are present, some PEPFAR delivery sites also receive support from other bilateral and multilateral funding through their investments in the government’s national HIV/AIDS programs. From 2005–2009, the total number of individuals directly supported on ART that counted toward the 5-year legislative target included the estimated overlap of individuals receiving ART with support by both PEPFAR and the Global Fund. As of September 30, 2009, the overlap estimate was 1.3 million individuals (OGAC, 2010). This overlap estimate also is included in the treatment results reported by the Global Fund (2.5 million individuals receiving ART with support from the Global Fund). To estimate this overlap, PEPFAR conducts a review of the treatment and funding data with the Global Fund and WHO, on a country-by-country basis. In its review, PEPFAR and the Global Fund take into account the percentage or level of contribution to the national HIV/AIDS program in order to determine where there is likely to be overlap (GFTAM, 2009; OGAC, 2010; PEPFAR, 2010b). In FY2009, PEPFAR and the Global Fund directly supported approximately 3.6 million individuals on ART (number of unique individuals supported by PEPFAR and the Global Fund, excluding the estimated overlap). PEPFAR plans to continue to work with the Global Fund to refine attribution methodologies in relation to treatment results (PEPFAR, 2010). Globally, the efforts in expanding the availability of ARV drugs has resulted in a greater proportion of people living with HIV in need of treatment receiving ART, which is helping to lower HIV-related mortality in multiple countries and regions (UNAIDS and WHO, 2009). Particularly, there is increasing evidence of the effectiveness of ART in decreasing morbidity and mortality in PLWHA in resource-poor settings (Bussmann et al., 2008; Herbst et al., 2009; Jahn et al., 2008; Mermin et al., 2008). Access to ARV drugs in low- and middle-income countries increased 10-fold between 2003 and 2008 (UNAIDS and WHO, 2009). In 2008, ART coverage in low- and middle-income countries reached 42 percent of the 9.5 million people in need, while coverage for children was 38 percent among 730,000 children (WHO et al., 2009). Despite progress in scaling-up access to ART in these countries, the majority of PLWHA and in need of treatment are currently not receiving such services, specifically in sub-Saharan Africa, where there is the greatest need (WHO et al., 2009). As noted above, in 2008 nearly two-thirds of the children under 15 years of age living with HIV worldwide in need of ART are still not receiving treatment (WHO et al., 2009). Some of the challenges in achieving greater coverage of treatment services for children include availability of cluster of differentiation (CD4) testing and ART at primary care, antenatal, delivery, and postnatal facilities where most maternal-child health care takes place (WHO et al., 2009). Furthermore, the difficulty of early infant diagnosis of HIV (EID), mainly due to the lack of affordable and accessible diagnostic testing and monitoring and to the shortage of ART regimens for children, poses particular challenges for increasing ART coverage among HIV- exposed or HIV-infected infants (UNAIDS, 2008; WHO et al., 2009). Without treatment, HIV infection in children follows an aggressive course including a faster progression to AIDS and death than in adults (Newell et al., 2004; Violari et al., 2008). With approximately 6 percent of PLWHA being children (UNAIDS and WHO, 2009), the disease is disproportionately killing 

81 ADULT AND PEDIATRIC TREATMENT more children than adults even though current WHO guidelines indicate that all HIV-infected infants and children less than 2 years of age should be started on ART immediately upon diagnosis, irrespective of CD4 count or WHO clinical stage (WHO, 2010). In 2008, 14 percent of AIDS-related deaths (280,000) were in children under 15 years of age (UNAIDS and WHO, 2009). Improved access to pediatric treatment (which includes children 0–14 years) will depend on the ability to identify women and children routinely through maternal-child care service entry points. For example, scaling up optimal implementation of PMTCT programs not only can improve treatment for women by providing HIV-positive pregnant women with ART, but also has the potential to identify and increase access for children in need of HIV treatment services. WHO revised its ART recommendations in 2009 and is now recommending that ART be initiated at a higher CD4 threshold of 350 cells/mm3 (compared to previous levels of 250 cells/mm3) for all HIV-positive patients, including pregnant women, regardless of the symptoms (WHO, 2009a, 2009b, 2009c). The 2009 WHO ART treatment guidelines for adults and adolescents, however, adds another 50 percent to those in need of ART from the 2008 figure above—increasing this number to approximately 15 million people, which is almost half of the PLWHA worldwide (De Lay, 2010). In the past 6 years, PEPFAR has supported activities for building laboratory capacity (OGAC, 2006a, 2009a, 2009c), including laboratory equipment, training, and quality control, given that laboratory monitoring is used by clinicians to track key patient outcomes, including the effects of ART on the well-being of a patient (WHO, 2006b, 2007b). Both CD4 and viral load testing, when available, are now recommended under the new 2009 WHO ART guidelines as part of program M&E activities to ensure quality HIV treatment programs (WHO, 2009a, 2009b, 2009c). In many developed countries this is part of the standard patient treatment monitoring; however, there are challenges to implementing CD4 cell and viral load testing in resource-limited settings, including both test cost and lack of infrastructure. These challenges may result in countries making determinations about how high a priority current ART programs should put on making these tests available vis-à-vis expanding the number of patients on treatment (Phillips and van Oosterhout, 2010). There is some evidence that the use of viral load monitoring in resource-limited settings improves patient outcomes as much as it does in high- resource settings (Gupta et al., 2009). The effect of routine CD4 cell monitoring on mortality was modest in a randomized trial in Uganda and Zimbabwe, and thus it requires additional evaluation (Dart Trial Team, 2009). The expanding goals of ART are to drive and maintain HIV-1 RNA plasma levels below limits of detection, preserve and restore immune function, and prevent or delay the clinical progression of disease (Hammer, 2010). Minimizing drug toxicities and prevention of drug resistance are also important goals of ART, and so adherence to treatment is one of the crucial variables of effective ART programs. A continuous supply chain for drug access, tolerable and convenient treatment regimens, and counseling efforts can enhance adherence to ART (and ultimately provide a normal life expectancy including improving quality of life). It is expected that with the 2009 WHO ART guidelines, tolerability and convenience of the initial regimen will improve by withdrawing stavudine from the recommended list of nucleosides (WHO, 2009a, 2009b). Earlier initiation of treatment is another variable as it confers the potential benefits of reducing the mortality associated with treatment of late stage disease, diminishing transmission including transmission from mother-to-child in the peripartum and during breast-feeding, diminishing drug toxicity, and reducing treatment failure with its attendant development of drug 

82 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS resistance (Emery et al., 2008; Granich et al., 2010; Johansson et al., 2010; Kuhn et al., 2008; Kuhn, 2009; Lawn et al., 2008). Furthermore, treatment also confers other potential population health benefits such as reducing secondary transmission of HIV and TB (Corbett et al., 2003; Girardi et al., 2000; Granich et al., 2009; Lima et al., 2008; Montaner et al., 2006; von Linstow et al., 2010; Wood et al., 2009). HIV drug resistance in treated patients is a measure of regimen efficacy and patient adherence; HIV drug resistance in treatment naïve patients is a measure of transmitted drug resistance. Consequently, monitoring for HIV drug resistance is important in order to assess program efficacy and future challenges with regard to different treatment regimens, maintaining a supply chain for drug access, providing care and counseling, and monitoring of treatment and its outcomes (WHO, 2006b). Under PEPFAR’s laboratory program, PEPFAR’s investments have provided support for the monitoring of drug resistance, which included “establishing HIV drug resistance testing capacity” and training of people from PEPFAR countries “on monitoring the emergence of drug resistance mutations in antiretroviral drug treated populations” (OGAC, 2009c, p. 128). Objective and Scope The IOM evaluation of treatment programs under PEPFAR will assess the effects of these programs on long-term outcomes such as retention of people on treatment and increasing coverage and quality of services, as well as PEPFAR’s contribution to improving the health of individuals and decreasing population level mortality due to HIV/AIDS. This will include consideration of some health systems components. PEPFAR treatment programs, for purposes of budgeting and performance targets, include activities that directly or indirectly support the provision of ART. This includes the training of clinicians and other providers, clinical examinations, clinical monitoring, related laboratory services, and community-adherence activities—as well as procurement and in-freight delivery of ARV drugs (OGAC, 2009e). Like ARV drug procurement, all procurement for post-exposure prophylaxis for rape victims will be included under treatment activities. However, distribution, supply chain logistics, pharmaceutical management, and related systems-strengthening inputs do not fall under PEPFAR’s treatment activities and will be assessed and evaluated under PEPFAR’s health systems strengthening activities (see section 8 on Key Systems-Level Goals and Activities). The evaluation of treatment programs will focus on PEPFAR’s progress in achieving direct support of more than 4 million people on treatment (OGAC, 2009i)—the PEPFAR II programmatic target—and an assessment of the overall performance of treatment programs, including efforts to integrate with other health services and to address gender-specific aspects of HIV/AIDS treatment (see section 7 on Gender-Related Vulnerability and Risks). The committee might also find it necessary to consider some of the emerging issues identified by PEPFAR’s TWG on Adult and Pediatric Treatment during the evaluation and assessment of treatment programs under PEPFAR. One of the main challenges that country programs face is continuing to scale up services for those in need while maintaining current numbers of people on treatment. Furthermore, according to PEPFAR’s reauthorization, countries need to provide care and treatment services to HIV-infected children in proportion to the pediatric HIV burden in each country. 22 Its stated treatment and care goals for children in relation to allocation of funds across all treatment service categories over time may be valuable metrics for program “sustainability.” 22 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(v). 

83 ADULT AND PEDIATRIC TREATMENT Although the 2009 WHO guidelines on ART for adults, adolescents, and pregnant women substantially expand the population for whom treatment is indicated, PEPFAR’s new Five-Year Strategy indicates that priority for access will focus on increasing access primarily within the “sickest,” pregnant women, and patients with HIV and TB co-infection (OGAC, 2009i) (see Box 3). How to address treatment needs for the many eligible patients who may remain untreated is an important challenge. The evaluation committee can learn from the findings of detailed scholarly evaluations conducted by local researchers and the scientific communities of the partner countries related to overall access to treatment—conducted as a consequence of specific in-country decisions made and intended for future country leadership to learn from their historic experiences. Further, many challenges remain in the pediatric population, including issues with early identification and diagnosis and low access of ART in children less than 2 years old. This is compounded especially by inadequate funding for pediatric treatment—in several countries, UNITAID 23 and the Clinton Foundation are the sole purchasers of ART formulations and 2nd regimes for children, EID commodities, cotrimoxazole (CTX), and other drugs for opportunistic infections (OIs) (OGAC, 2009c; UNITAID-CHAI, 2010). UNICEF and UNAIDS have determined that about $6 billion of the $25 billion, needed by 2010 to enable countries to reach universal access goals, 24 is required to attain the universal goals specific to women and children (UNICEF, 2009). In 2010–2011, UNITAID and the Clinton Foundation will deliver the last shipment of some of these commodities; therefore, countries will need to make up for the gap (OGAC, 2009c). Since the United States alone cannot sustain ART for the millions of PLWHA, PEPFAR’s focus under its new strategy is to support countries in discussing what resources are needed to respond to the HIV/AIDS epidemic, how to prioritize the large unmet need for treatment of adults, adolescents, and children, and how to identify resources for the gap—PEPFAR plans to support these efforts in a five-year plan through partnership frameworks with partner countries. 23 UNITAID is an international facility dedicated to purchasing drugs for HIV/AIDS, malaria, and tuberculosis, primarily for people in low-income countries. UNITAID leverages its funds, received through airline ticket taxes or regular multi-year budget contributions from member countries, to reduce the price of quality diagnostics and medicines as well as to accelerate the development and availability of these products in low- and middle-income countries (UNITAID, 2010). 24 At the United Nations General Assembly High-Level Meeting on AIDS in 2006, countries committed to work toward “universal access to comprehensive [HIV] prevention programmes, treatment, care and support by 2010” (United Nations General Assembly, 2006, p. 3). 

84 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS BOX 3 Areas of Emphasis in PEPFAR II’s New Five-Year Strategy for Treatment In partnership with country governments, PEPFAR is continuing the scale up of treatment to directly support more than 4 million people in its next phase. PEPFAR is working with countries to reach a threshold of 85 percent ARV prophylaxis or treatment of pregnant women found to be HIV-infected, in order to optimize maternal health and maximize HIV-free infant survival. In generalized epidemics, PEPFAR is working to reach a target of 65 percent early infant diagnosis, and support treatment for pediatric populations at a level commensurate with their representation in a larger country epidemic. Through country- and global-level efforts, PEPFAR is creating increased sustainability and capacity in treatment efforts and supporting countries in mobilizing and coordinating resources from multiple donors. PEPFAR is working with countries and international partners to expand identification and implementation of efficiencies in treatment, while ensuring continued expansion of measures to maintain adherence, quality, and retention in care. As part of the U.S. Global Health Initiative, PEPFAR is integrating its treatment programs with prevention and care portfolios, other health programs, and larger development efforts. SOURCE: OGAC (2009i). Program Impact Pathway The evaluation approach for those activities under PEPFAR specific to adult and pediatric treatment will follow the impact pathway framework (see Figures 10 and 11). This approach is intended to illustrate how PEPFAR’s investments and other laboratory commodities (inputs) for the delivery of ART services increase availability of ARVs and access to treatment interventions (outputs). Subsequently, the coverage of those in need of and eligible for treatment and the delivery of quality treatment services for HIV patients (outcomes) are expected to ultimately improve health of individuals and decrease population-level mortality (impacts). Time is required for the effects of the PEPFAR program on outcomes for a chronic disease like HIV/AIDS to become apparent and, more importantly for this evaluation, to accrue the systematic collection of valid and relevant data. Given the timely availability of quality data, the committee has developed examples of illustrative questions that it could address during the evaluation of PEPFAR. The committee developed a separate impact pathway for evaluating pediatric treatment considering the differences with adult treatment, including different points of entry for pediatric treatment services and socio-cultural barriers associated with the effective delivery of pediatric treatment by clinicians and non-clinicians, as well as the availability of drug formulations for children or appropriate use of adult preparations when substitutions are necessary (see Figure 11). 

FIGURE 10 Program impact pathway for evaluation of PEPFAR’s adult treatment program. NOTE: CD4 = cluster of differentiation 4. 85 

86 FIGURE 11 Program impact pathway for evaluation of PEPFAR’s pediatric treatment programs. NOTES: ARV = antiretroviral drugs, ART = antiretroviral therapy, CD4 = cluster of differentiation 4, MCH = maternal and child health, mo = months old, yrs = years old. 

87 ADULT AND PEDIATRIC TREATMENT Illustrative Questions Most of the limitations in evaluating the impact of treatment programs under PEPFAR and assessing their performance have to do with the fact that all the essential indicators gathered by OGAC from all partners are aggregated and limited. Age disaggregation is exceedingly important, especially to assess PEPFAR’s performance in treating infants under 2 years old since ART, about one-third of HIV-infected infants will die by age 1 year and 50 percent by age 2 years (Newell et al., 2004). Meaningful data disaggregation will occur at the country level and, if available, are probably accessible through multilateral organizations such as UNAIDS and UNICEF, but without attribution to specific donors—these data will have to be requested. DHS and Multiple Indicator Cluster Survey (MICS data) will also need to be accessed in order to provide country context. The following are examples of illustrative questions that the committee could address. These questions are intended to contribute to addressing part A, item ii and part B, items i and iv of the areas for consideration in the congressional mandate. 25 Is treatment funding (solo and joint PEPFAR and Global Fund) and provision of services being efficiently (equitably) distributed among countries, contractors, and implementers? What is the number and proportion of adults and children with advanced HIV infection (i.e., CD4 <200 for adults) currently receiving ART? What is the number and proportion of adults and children receiving treatment according to the updated WHO guidelines (i.e., all newborns, adults <350, all pregnant women)? To what extent does the current methodology adequately or appropriately determine PEPFAR and Global Fund overlap in numbers? What is the financial sustainability of existing and expanded coverage with treatment services, specifically ART? To what extent do PEPFAR activities or PEPFAR/Global Fund joint activities assist countries in determining and projecting costs for existing and expanded coverage with treatment services including ART? To what extent are CD4 measurements being utilized to initiate treatment and monitor treatment? PEPFAR’s NGO sub-partners, especially the Track 1.0 partners for treatment and care as well as the Presidential initiative for PMTCT that pre-dates PEPFAR I (but was subsequently subsumed within PEPFAR), have longitudinal reporting for as long as 10 years, and the ability to disaggregate by facility, as well as by many other parameters. Longitudinal, individual patient 25 (A)(ii) an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by United States funding, including multilateral and bilateral programs involving joint operations (B)(i) an assessment of progress toward prevention, treatment, and care targets (B)(iv) an evaluation of the impact of treatment and care programs on 5-year survival rates, drug adherence, and the emergence of drug resistance PREPUBLICATION COPY: UNCORRECTED PROOFS 

88 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS data are available only from very few of sub-partners, but do include large numbers of people. The most detailed information will therefore represent specific facilities in selected countries. What are the rates of drug resistance in treated and naïve populations? How are these data used for policymaking decisions about selection of treatment regimens? PEPFFAR supports the WHO’s strategy on HIV drug resistance surveillance and monitoring, and has provided support to build drug resistance testing capacity in PEPFAR countries (OGAC, 2009c). Globally, the WHO HIV drug resistance threshold survey method is being implemented in resource-limited countries to determine transmitted drug resistance levels as part of the surveillance of transmitted HIV drug resistance. The results of these surveys are published in scientific journals. The committee will seek to collect these published papers in order to assess the effects of treatment programs on the emergence of drug resistance. What percent of adults, children, and pregnant women who initiated ART are alive and on treatment at 1 year and annually thereafter? Although treatment adherence is considered essential to successful therapy as well as to extending regimens and reducing medication and hospitalization costs, none of the OGAC indicators measure it. However, in the provision of treatment at the facility, district, and sub- partner levels, adherence has been monitored because it is necessary for the delivery of therapy. Some of the sub-partners, interested in quality of care, have routinely monitored adherence (albeit with multiple and not always standardized or validated definitions). Monitoring is usually done via return visits, pill counts, clinical assessment, and self-report. The proportion of the population on second-line regimens is another measure, although without genotyping/phenotyping it is not always clear whether regimen failure is due to non-adherence. Learning what the individual countries require will guide the committee’s effort to answer questions on adherence. Currently, assessing the impact of ART programs on mortality at the population level will be limited due to the lack of vital statistical data or the delay in generating vital statistics in many of the PEPFAR countries. Many low- and middle-income countries have inadequate vital or civil registration systems and where the systems exist, they are usually not comprehensive and the cause of death is misreported or underreported (WHO et al., 2009). “Surveys conducted sometimes use verbal autopsies to retrospectively assess HIV-attributable mortality in a population; however, this approach often lacks baseline data for assessing how access to ART affects HIV-related mortality. Some countries have also counted burials of deceased people in the age group of 15–49 years” (WHO et al., 2009). Therefore, although an evaluation of the impact of treatment on 5-year survival of HIV-infected adults and children receiving ART has been requested in the congressional mandate, any survival data will be limited and mortality data greater than one- to two-year survival will not be available for most PEPFAR treatment programs. 

SECTION 5: CARE AND SUPPORT SERVICES Care and support services are an important component of PEPFAR and other donor programs for HIV/AIDS. PEPFAR defines care and support services as “the wide range of services other than ART offered to people living with HIV/AIDS (PLWHA) and other affected persons, such as family members” (OGAC, 2009a, p. 16). During FY2009, PEPFAR directly supported care and support for nearly 11 million people affected by HIV/AIDS, including 7 million HIV-infected individuals and approximately 4 million orphans and vulnerable children (OGAC, 2010). Care and support includes clinical, psychological, social, spiritual, and preventive services that may be provided in facility-, community-, or home-based settings. The Lantos–Hyde Act of 2008 charges PEPFAR II to support care for 12 million people infected with or affected by HIV/AIDS (including 5 million children orphaned or made otherwise vulnerable by HIV/AIDS), “with an emphasis on promoting a comprehensive, coordinated system of services to be integrated throughout the continuum of care.” 26 Efforts to integrate care and support services with broader health and development programs, such as voluntary family planning and reproductive health services, are a key component of PEPFAR II (OGAC, 2009g). As outlined in the new Five-Year Strategy (2009–2013), PEPFAR has adopted a “woman- and girl-centered approach” to delivering services, which “takes into account the realities of women’s and girls’ lives as shaped by gender norms, service availability, and larger structural factors,” and is working to ensure that other marginalized populations have equal access to services (OGAC, 2009h, p. 6, 2009i). Components of PEPFAR Care and Support Services The latest PEPFAR-issued guidance for care and support services, “Guidance for United States Government In-Country Staff and Implementing Partners for a Preventive Care Package for Adults,” was released in 2006 and describes a menu of preventive care services for adults (OGAC, 2006b). A similar menu of services for children (0–14 years) was also released in 2006 and is described in the Child and Adolescent Well-Being section of this report. These preventive care services (see Figure 12) are intended to promote health and quality of life for PLWHA, slow the progression of AIDS, and reduce HIV-related complications and mortality. The preventive care services for adults include five of the thirteen interventions considered essential by WHO for adults and adolescents living with HIV in resource-limited settings (five additional interventions are delivered through PEPFAR prevention and treatment services) (OGAC, 2006b; WHO, 2008). 26 Supra., note 6 at §101(a), 22 U.S.C. §7611(a)(4)(C). 89 

90 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS FIGURE 12 PEPFAR care and support services. NOTES: HPV = human papillomavirus; ITNs = insecticide-treated nets; PCP = Pneumocystis jiroveci pneumonia; STI = sexually transmitted infections; TB = tuberculosis. 

91 CARE AND SUPPORT SERVICES Although the type of interventions in a preventive care menu will differ by region or country according to the capacity of implementing partners, some recommended components include: Screening, Prophylaxis, and Treatment for Tuberculosis TB and HIV co-infection is very common and has devastating consequences for PLWHA. HIV infection increases the risk of TB 10-fold, and in sub-Saharan Africa and Asia, TB is the leading cause of death for PLWHA (WHO, 2008). WHO guidelines recommend counseling (education) and regular TB screening for all PLWHA, and preventive therapy (isoniazid) for HIV-infected patients with latent TB (WHO, 2008). In order to identify people who have HIV infection, PEPFAR efforts have focused on the scale-up of same-day HIV testing at TB clinics. However, even in countries where this scale-up has been effective, a large number of TB patients identified as HIV-positive are lost to follow up after referrals to HIV/AIDS care and treatment programs, and efforts to screen, diagnosis, and treat all HIV patients for TB have been less successful (OGAC, 2009c). Further, lack of laboratory capacity has hindered efforts to scale up isoniazid preventive therapy for PLWHA with latent TB infection, as many countries lack the ability to rule out active TB, a prerequisite for isoniazid prophylaxis (OGAC, 2009c). The new Five-Year Strategy commits PEPFAR to scale up efforts to screen, diagnose, and if necessary, treat all HIV patients for TB, while expanding linkages and referrals to ensure that all TB patients are tested for HIV and if positive, referred to treatment (OGAC, 2009g, 2009h). Prophylactic Drugs for Opportunistic Infections CTX is a broad-spectrum antimicrobial agent that prevents Pneumocystis jiroveci pneumonia (formerly Pneumocystis carinii pneumonia), toxoplasmosis, and malaria (WHO, 2006a). WHO recommends that all adults with HIV receive CTX prophylaxis indefinitely as a cost-effective method to significantly reduce morbidity and mortality, but country-level policies vary according to the burden of HIV and other diseases, as well as the capacity and infrastructure of health systems (WHO, 2008). WHO also recommends that all infants born to HIV-infected mothers receive CTX (WHO, 2006a). Cryptococcal disease is common and often treatable in PLWHA, but many countries lack the infrastructure and human capacity for diagnosis (OGAC, 2009c; WHO, 2008). PEPFAR has provided limited training and laboratory capacity building for diagnosis, but without early recognition, mortality from cryptococcal disease is high (OGAC, 2009c). Where cryptococcal disease is common and diagnostic capacity exists, WHO recommends consideration of antifungal prophylaxis (fluconazole or itraconazole) for severely immunocompromised PLWHA (WHO, 2008). Currently, there is limited availability of antifungal prophylaxis, but PEPFAR is working with its Supply Chain Management System and Pfizer, which runs a fluconazole donation program, to increase access to drugs for treatment and prevention (OGAC, 2009c). Improved Screening and Treatment of Opportunistic Infections (including Cervical Cancer) Screening and diagnosis of OIs may be limited by human and laboratory capacity in many countries. Cervical cancer is almost always caused by certain types of human papilloma virus. Cancer-causing strains of human papilloma virus are common in women with HIV and 

92 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS increase the risk of cervical cancer (OGAC, 2009c). WHO recommends that where available, women with HIV should be screened for cervical cancer annually (WHO, 2008). As part of a comprehensive approach to OIs, PEPFAR is currently supporting “pilot programs which provide screening and treatment to prevent cervical cancer in HIV-positive women” (OGAC, 2009a, p. 55). These pilot programs are using the “see and treat” approach, which includes visual inspection with acetic acid, visual inspection with Lugol’s iodine, and direct visual inspection (Kaur and Singh, 2010). Increased Access to Safe Drinking Water to Prevent Waterborne Illnesses and the Promotion of Basic Hygiene and Sanitation to Reduce Exposure to Pathogens In many developing countries, poor infrastructure and lack of safe management of human waste increase the risk of waterborne and enteric pathogens, many of which cause diarrhea (WHO, 2008). Diarrhea affects 90 percent of PLWHA, and interventions to improve water, sanitation, and hygiene, such as provision of safe water storage vessels and education regarding hand-washing, can greatly reduce diarrhea-related morbidity (OGAC, 2009c). The latest USG “Framework for Addressing Water Challenges in the Developing World,” which guides USAID and DoS efforts, encourages the incorporation of these interventions into all HIV/AIDS programs (USAID and DoS, 2009). PEPFAR’s preventive care package also includes water purification systems, which in combination with other health interventions, may keep PLWHA healthy and delay the need for treatment (OGAC, 2009a). Prevention of Malaria WHO recommends the integration of malaria and HIV services with a particular focus on prevention (WHO, 2008). As previously mentioned, CTX may reduce malaria-related morbidity and mortality in PLWHA. Insecticide-treated nets (ITNs), when used properly and regularly, are cost-effective and greatly reduce exposure to malaria infection. WHO also recommends intermittent preventive therapy (IPTp), which can reduce the risk of malaria and its consequences, for HIV-positive pregnant women who are not taking CTX (WHO, 2008). The President’s Malaria Initiative (PMI) is a USG interagency initiative to reduce malaria in 15 focus countries, 9 of which have significant PEPFAR programs (PMI, 2009). PMI is working to expand coverage of effective malaria prevention and treatment interventions, including ITNs, indoor residual spraying with insecticides, IPTp, and artemisinin-based combination therapy (PMI, 2009). PEPFAR routinely links to PMI, and their efforts overlap in ITN distribution and education programs as well as coordination of lab services (OGAC, 2009a). Food and Nutrition Support Services HIV infection may cause or intensify malnutrition by reducing appetites, increasing energy needs, and impairing nutrient absorption in PLWHA (OGAC, 2009c). Proper nutrition supports the immune system, preventing OIs. Nutritional and micronutrient supplementation may reduce HIV-related morbidity and mortality and improve outcomes for patients on ART (OGAC, 2009i). Through its Food by Prescription programs, PEPFAR targets clinically malnourished children and adults with HIV infection, pregnant and lactating women and their infants in PMTCT programs, and orphans and other vulnerable children (regardless of HIV status) for food 

93 CARE AND SUPPORT SERVICES and nutrition care and support, including nutrition assessment and counseling services, specialized food products, and micronutrient supplementation (OGAC, 2009c). Although PEPFAR does not support direct food distribution to families, the new Five-Year Strategy emphasizes linkages and referrals of those in need to the new USG Global Hunger and Food Security Initiative, Title II programs, and other initiatives such as the World Food Program (OGAC, 2009c, 2009g). Health, Dignity, and Prevention Programming for PLHWA and Their Families Health, Dignity, and Prevention programming includes efforts related to health promotion and education, reducing stigma for PLWHA, and preventing HIV transmission from infected to non-infected people. The components of the preventive care package recommended by PEPFAR address health promotion, and prevention efforts are discussed in this report’s prevention section. PEPFAR is working with country governments to develop “policies that address the drivers of the epidemic in country and provide equitable access to quality services for marginalized populations” and expanding “linkages to multiple primary and specialty health services” which “increases community-level access to quality care and reduces the stigma associated with HIV” (OGAC, 2009i, pp. 15, 27). Palliative Care, Including Management of Pain and Other Symptoms PEPFAR defines palliative care as an “holistic approach to providing services that includes a focus on pain and symptom management and on improving quality of life,” which is consistent with the WHO definition (OGAC, 2009c, p. 25). Palliative care, including pain management and end-of-life care, enables PLWHA to lead happier, more productive lives and reduces the burden of care on families. In many countries, restrictive policy environments prohibit effective pain management programs, and access to strong pain medications such as opioids is limited (OGAC, 2009i). Up to 80 percent of those with advanced HIV infection experience pain, and pain management programs can greatly improve quality of life for PLWHA (OGAC, 2009c). PEPFAR’s new strategy calls for continued efforts to “support policy changes that ensure pain management is included both in guidelines and actual clinical services for PLWHAs,” as well as increased efforts to “strengthen commodity systems, train providers, and expand access to opioids for pain management” (OGAC, 2009i, p. 19). Economic Strengthening and Support Activities Recognizing that a lack of economic assets increases vulnerability to HIV infection, PEPFAR supports economic strengthening and support activities that “supply, protect, or grow physical, natural, financial, human, and social assets” (OGAC, 2009h, p. 17). These activities may include microfinance and microcredit programs to expand access to financial services, vocational training to offer alternatives to transactional sex, and income-generating activities, such as communal gardens (that may also provide food). Objective and Scope 

94 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS With regard to care and support services, the committee is charged to evaluate the impact on health of care efforts supported by USG funding and assess progress toward care targets. Complicating this task, country-level guidelines for ART eligibility differ and make program- wide comparisons difficult. The committee is also charged to evaluate impact of care programs on 5-year survival rates among people not yet eligible for ARV treatment. Data on 5-year survival rates is very limited for any population; unless these data are being collected and can be accessed, it will not be possible to assess PEPFAR’s impact on 5-year survival rates among people not yet eligible for ART. Program Impact Pathway To assess whether PEPFAR has achieved targets regarding care and support services, the committee will examine PEPFAR’s care and support activities and use output and outcome indicators (e.g., number of people accessing care services, percent of HIV-positive patients provided CTX prophylaxis) where available. To evaluate the impact of care services on health, the committee will use a combination of output and outcome indicators to determine the extent of delivery of PEPFAR-funded services in-country. Figure 13 depicts the committee’s understanding of some of PEPFAR’s care and support activities and some potential outputs and outcomes to be measured during the evaluation. 

FIGURE 13 Program impact pathway for evaluation of PEPFAR’s care and support programs. NOTES: ART = antiretroviral therapy; HBC = home-based care workers; HCW = heath care workers; ITNs = insecticide-treated nets; OIs = opportunistic infections; TB = tuberculosis. 95 

96 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions Care is composed of multiple components, and the complexities of measurement are increased by what is included or has changed over time during PEPFAR (i.e., 14 of the essential PEPFAR indicators of care have changed significantly, have been dropped, or are new). 27 The lack of consistently measured data may result in data gaps that limit the possibility of examining longitudinal trends. Given the timely availability of quality data, the following are examples of illustrative questions that the committee could address. These questions are intended to contribute to addressing part A, item ii and part B, items i and iv of the areas for consideration in the congressional mandate. 28 What populations are accessing care and support services? The number and types of populations accessing care will be measured through quantitative analysis of PEPFAR indicators. Aggregated data reported to OGAC provide limited insight about the types of populations accessing care, so the committee will seek disaggregated data from other sources. To reflect PEPFAR’s new women- and girl-centered approach to delivering services and the emphasis on equity across marginalized populations, the committee will seek data disaggregated by sex and population type (IDUs, MSM, etc.). Measurement of the number of people receiving care services is complicated by the potential of “double counting” (people accessing more than one service may be counted more than once). What is the access to prophylaxis for, diagnosis of, and treatment of OIs? How has treatment for TB been maximized with jointly operated activities between PEPFAR and the Global Fund? How has coverage expanded as result of joint efforts? What has been the effect on grantee performance to improve access? There are several PEPFAR indicators that measure the number and percentage of HIV- positive patients receiving CTX, isoniazid preventive therapy, and TB screening and treatment (OGAC, 2009d). In addition to the aggregated data reported to OGAC, the committee will seek additional data sources to provide information regarding testing and treatment for malaria and sexually transmitted infections, treatment for pneumonia and diarrhea, vaccinations for human papilloma virus, and more. The committee will also seek information regarding human and laboratory capacity building to increase diagnostic capabilities for OIs. How is PEPFAR supporting access to and distribution of ITNs? How is this measured? There are no PEPFAR indicators regarding ITNs. The committee will seek information from PMI or implementing NGOs or contractors that receive PEPFAR funding to distribute ITNs. 27 Essential indicators are those for which OGAC requires PEPFAR Country Teams to track data to monitor PEPFAR’s progress (OGAC, 2009d). 28 (A)(ii) an evaluation of the impact on health of prevention, treatment, and care efforts that are supported by United States funding, including multilateral and bilateral programs involving joint operations (B)(i) an assessment of progress toward prevention, treatment, and care targets (B)(iv) an evaluation of the impact of treatment and care programs on 5-year survival rates, drug adherence, and the emergence of drug resistance 

97 CARE AND SUPPORT SERVICES How is PEPFAR supporting access to safe drinking water, basic hygiene, and sanitation? How is this measured? Is there any discernible effect of this access for improved health or a decrease in OIs diagnosed and needing treatment? There are no PEPFAR indicators regarding safe drinking water, basic hygiene, and sanitation. The committee will seek data and information regarding access to safe water, ideally multi-year data to examine trends, from countries, implementing agencies, partners, sub-partners, and international stakeholders. Some data may be available from implementing NGOs or contractors that receive PEPFAR funding to conduct activities regarding basic hygiene and sanitation. How is PEPFAR incorporating palliative care or pain management into care services? What is the proportion of need compared to availability of services, particularly analgesics and opioids? Are PEPFAR teams following WHO guidelines for provision of analgesics including opioids? PEPFAR indicators measure the percentage of ART sites that have pain management programs and the progress toward incorporating pain management into national HIV/AIDS strategies, but the committee will seek additional information from other sources. The committee will also seek information from the Palliative Care TWG at OGAC about proposed activities for and guidance to Country Teams to develop or change policies in the country related to drug procurement, drug security, and scope of practice for dispensing drugs to patients. What economic support and strengthening activities does PEPFAR support? How is this measured? There is one PEPFAR indicator that measures the number of eligible adults and children provided with economic support or strengthening services, but the committee will need to seek additional information regarding the types of economic support and strengthening activities and their effectiveness from other sources. Who is receiving therapeutic or supplementary food and through what activities? Is there a discernible effect on ART adherence or mortality? PEPFAR indicators measure the number of people receiving food or nutrition services and the percent of HIV-positive clinically malnourished patients receiving food. The committee will use aggregated data reported to OGAC and seek disaggregated data from organizations and initiatives to which PEPFAR links or refers HIV-positive patients for nutritional support. 



SECTION 6: CHILD AND ADOLESCENT WELL-BEING The HIV/AIDS pandemic has produced devastating effects on the lives of millions of children and adolescents 29 worldwide endangering their development, life course, and survival. When HIV affects parents and other adult caregivers, it destroys their families and deprives them of care and protection by weakening communities and social support networks, welfare systems, and economies. Moreover, millions of children and adolescents are currently directly affected due to infection with HIV. In 2008, children under 15 years of age were estimated to be nearly 16 percent of incident cases, and the number of children under 15 years of age living with HIV/AIDS worldwide was approximately two million, with an additional number of adolescents among the 31 million adults (age 15 and over) living with the disease (UNAIDS and WHO, 2009). In 2007, the number of children and adolescents aged 0–17 years old who have lost one or both parents as a result of the AIDS epidemic 30 was estimated to be approximately 15 million worldwide—in sub-Saharan Africa alone, the number was estimated to be approximately 12 million children (UNAIDS, 2008). Combining the needs of children and adolescents infected with HIV with the needs of orphans, as well as other children and adolescents made vulnerable due to HIV/AIDS, gives a full perspective on the burden of the epidemic in these populations. Therefore, understanding the needs of all children and adolescents affected and made vulnerable by HIV/AIDS is a vital step in the response to the HIV/AIDS pandemic (JLICA, 2009). In addition to the need for access to HIV/AIDS services, there are also other critical developmental and societal factors influencing the health and psychosocial well-being of children and adolescents affected by HIV/AIDS. For example, many children and adolescents with sick and dying parents have to become the breadwinners and primary caregivers of their households (Cluver et al., 2007). When a parent dies, some of the effects related to the grieving process, as well as the deprivation and life changes that occur because of this loss, might affect the health and well-being of the lost parent’s children. Parental loss might result in trauma, relocation, loss of a breadwinner, residence in poorer households, and living with less closely related caregivers, which can lead to other effects including poorer access to adequate nutrition, shelter, and health care, and lack of educational support. These are all mediating factors of psychosocial well-being (Cluver and Orkin, 2009; Nyamukapa et al., 2008). Children and adolescents, in settings where HIV is highly stigmatized, have to cope with higher levels of psychosocial stressors associated with the loss of a parent due to HIV/AIDS than both children orphaned by other causes and non-orphaned children (Cluver and Gardner, 2007; Cluver and Orkin, 2009). Children and adolescents living within communities that experience a high HIV 29 The committee uses “children and adolescents” as a general term without a specific age definition, recognizing that the ages used to categorize children and adolescents vary by data source and organization. In particular, the age categories vary for terms like adolescents, youth, and young people. For example, adolescents are defined by WHO as young men and women 10–19 years of age and young people refers to men and women 10–24 years of age (WHO, 1999, 2006c). United Nations defines youth as men and women 15–24 years of age and young people refers to men and women 10–24 years of age (WHO, 1999, 2006c). Within PEPFAR, defined age ranges vary by programmatic area. Throughout this section, the specific age ranges used by PEPFAR or by the cited data source are indicated whenever feasible. 30 In 2001, a consensus was reached among members of the UNAIDS Reference Group on Estimates Modelling and Projection, and international researchers on the definition of orphans due to HIV/AIDS. An “AIDS orphan” was defined as a child who has at least one parent who has died due to AIDS and a dual (or double) “AIDS orphan” as a child whose mother and father have both died, at least one due to AIDS (UNAIDS Reference Group on Estimates Modelling and Projections, 2002). 99 

100 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS burden are at a greater risk of these forms of vulnerabilities, and are at a greater risk of physical and sexual abuse, sexual exploitation, homelessness, and exposure to HIV (UNAIDS et al., 2002, 2004). In addition to younger children, the vulnerabilities of youth between the ages of 15–24 years have been recognized by the international community (UNGASS, 2001; WHO et al., 1997). This developmental period is an important transition period, and youth are vulnerable due to age-specific changes that are physical (their physical and cognitive abilities), psychological (how they think about themselves), and social (their relationships and roles, expectations, economic security, and citizenship). These changes have implications for how they understand information and what influences them, how they think about the future and make decisions in the present, and how they perceive risk and their sexual behavior (Dick, 2009). In 2007, an estimated 45 percent of incident cases in people aged 15 years and older were found among youth aged 15–24 years. Overall, in 2008, a total of approximately 5 million youth aged 15–24 years were living with HIV in low- and middle-income countries (UNICEF, 2009), and in sub-Saharan Africa, youth, and in particular young women, are disproportionately vulnerable and therefore at greater risk of HIV infection (Gouws et al., 2008; Napierala- Mavedzenge et al., 2010; UNAIDS and WHO, 2009; UNICEF, 2009). Beyond biological susceptibility to HIV, socio-cultural factors that contribute to the vulnerability of young women to sexually transmitted HIV infection include entrenched gender roles, unbalanced power relations, sexual violence (such as coerced sex), unsafe sex with older men, and a lack of skills and information about how to protect themselves and access services (UNAIDS, 2009a) (see section on Gender-Related Vulnerability and Risk). The International Convention on the Rights of the Child,31 guides the international community’s efforts to protect the rights of children under the age of 18 years to survival, development, and access to health services, including a focus on reversing the HIV epidemic in children and mitigating its negative effects on their health and well-being through the MDGs, the UNGASS on HIV/AIDS, and the UNGASS on Children (UNICEF, 2007). The progress of countries in achieving the standards and goals outlined in these documents is monitored by the Committee on the Rights of the Child (CRC), primarily through country reports that may provide valuable information for this evaluation on the environment in which PEPFAR’s efforts to improve child and adolescent well-being are operating within a specific country or region. Efforts by multilateral and bilateral stakeholders to support policy on orphans and vulnerable children and adolescents affected by HIV/AIDS have resulted in the development of the “Framework for the Protection, Care, and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS,” which PEPFAR has adopted (IOM, 2007). This framework document lays out five key strategies to improve the well-being of children: (1) strengthening the capacity of families, (2) mobilizing and supporting community-based responses, (3) ensuring access for orphans and vulnerable children to essential services, (4) ensuring that governments protect the most vulnerable children through improved policy and 31 The Convention on the Rights of the Child is the first legally binding international instrument to incorporate the full range of civil, cultural, economic, political, and social rights of children. The Convention gives UNICEF the responsibility of promoting the rights of children by supporting the Committee on the Rights of the Child (CRC). As the oversight body, the CRC monitors the progress of State Parties in setting and meeting the standards outlined in the Convention. UNICEF provides technical assistance to governments on the implementation of the Convention and the development of implementing reports, which are required to be submitted by State Parties to the CRC 2 years after acceding to the Convention and every 5 years thereafter. The CRC convenes three times a year to review the States’ reports (OHCHR, 2007; United Nations, 1990; United Nations Treaty Collection, 2010). 

101 CHILD AND ADOLESCENT WELL-BEING legislation, and (5) raising awareness at all levels through advocacy and social mobilization to create a supportive environment (UNAIDS et al., 2002, 2004; UNICEF, 2004). PEPFAR-Supported Interventions for Children and Adolescents PEPFAR currently supports services for children and adolescents through its three main programmatic areas—prevention, care, and treatment. Additionally, in keeping with the “Framework for the Protection, Care, and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS,” PEPFAR supports programs specifically identified for orphans and vulnerable children and adolescents (hereinafter referred to as OVC programs or programming 32 ). In FY2009, PEPFAR supported care for 3,620,140 million children and adolescents through its OVC programming, and provided pediatric treatment for 201,500 individuals under 15 years of age (about 8 percent of the total number of people receiving ART with direct PEPFAR support) (OGAC, 2010). In addition, 273,100 individuals were trained or retained on the care and support of orphans and vulnerable children and adolescents (OGAC, 2010). Other PEPFAR-supported capacity building during this fiscal year to address the needs of children included the training or retaining of 62,100 healthcare workers on PMTCT services (including 15,597 PMTCT service facilities) (OGAC, 2010). The Lantos–Hyde Act of 2008 underscores children and adolescent needs as part of the USG commitment of preventing 12,000,000 new HIV infections worldwide and increasing the number of individuals with HIV/AIDS receiving ART. In particular, it states that programs supported by PEPFAR need to “provide care and treatment services to children with HIV in proportion to their percentage within the HIV-infected population of a given partner country.”33 Additionally, PEPFAR II performance targets for the care and support of PLWHA include providing care and support for five million children and adolescents orphaned or made otherwise vulnerable by HIV/AIDS. 34 In order to achieve this target, the new Five-Year Strategy states that PEPFAR will continue to address child-development issues through child-focused programming that targets the full range of needs at different developmental stages (OGAC, 2006e, 2009i, p. 22) (see Box 4). Within the scope of care and support activities for PLWHA, the act directs that “at least 10 percent” 35 of PEPFAR funds supporting country activities for prevention, treatment, and care go to OVC programs in order to mitigate the impact of HIV/AIDS for millions of children and adolescents living in affected communities—an earmark that was preserved in the reauthorization legislation. In FY2009, these funds amounted to approximately $320 million,36 with an additional $44 million provided in funding for pediatric care and support activities (OGAC, 2009f). Finally, the Lantos–Hyde Act of 2008 also highlights a new emphasis on the transition to young adulthood. The reauthorization legislation states that PEPFAR’s annual report to Congress will include a description of the strategies, goals, programs, and interventions that “address the needs and vulnerabilities of youth populations” and ‘‘expand access among young men and women to evidence-based HIV/AIDS health care services and HIV prevention programs.” 37 To 32 For the purpose of brevity, the acronym OVC will be used to describe programs targeting eligible children and adolescents under PEPFAR’s programs for orphan and vulnerable children. 33 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(v). 34 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iii). 35 Supra., note 1 at §403(b), 22 U.S.C. §7673(b). 36 Approved funds in FY2009 for supporting OVC programs’ activities found in PEPFAR operational plans. 37 Supra., note 6 at §301(a)(2), 22 U.S.C. §2151-b2(f)(2)(D)(ix). 

102 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS reflect the new priorities in programming that address the age-specific needs and vulnerabilities of adolescents and young people in particular, PEPFAR II is committing through its new Five- Year Strategy to support countries in pursuing these objectives (see Box 4). BOX 4 Reauthorization Programming for Orphans and Vulnerable Children Years 1–2: Support countries to define, map, and plan a prioritized, multisectoral response to the needs of OVC populations and sub-populations within a country. Work with partner countries to identify gaps in capacity, including gaps in coordination among ministries overseeing education, food and nutrition, social welfare, and health. Establish training, mentoring, and technical assistance programs in partnership with governments in order to increase the number of professional staff in all agencies who can address cross-cutting OVC needs. Work with countries to increase support for family-based care by establishing and strengthening linkages between clinical and home- and community-based care. Scale up and ensure robust monitoring of existing high-impact OVC programs and support countries in developing, implementing, and evaluating innovative OVC pilot programming. Help countries ensure that policies for most-at-risk populations have adequate coverage and referrals for youth subpopulations. Support countries in developing a case management capability to assist the transition of young adults from OVC services into society and careers. Years 3–5: Work with countries to engage in periodic and targeted surveys and other evaluations to determine impact of OVC programming. Ensure that countries have programs through which OVC can access livelihood development opportunities, including vocational training and microenterprise development training, to support themselves and their families. SOURCE: OGAC (2009i). Prevention The Lantos–Hyde Act of 2008 emphasizes the need to intensify efforts to prevent HIV as a priority within the USG-supported response to the global HIV epidemic. 38 The majority of new incident cases in infants and young children occur from transmission in utero, during delivery, or post-partum as a result of breastfeeding (UNAIDS and WHO, 2009). In 2008, an estimated 1.4 million pregnant women living with HIV in low- and middle-income countries gave birth (WHO et al., 2009). PMTCT, also described previously in the section on Prevention, is an evidence- based prevention intervention that can reduce perinatal HIV transmission to approximately 2 percent if appropriately delivered under optimal circumstances (Cunningham et al., 2002; 38 Ibid., §4, 22 U.S.C. § 7603(3)(A). 

103 CHILD AND ADOLESCENT WELL-BEING Dorenbaum et al., 2002; Kuhn et al., 2008). UNAIDS estimated that 200,000 cumulative new HIV infections have been averted in the past 12 years through the provision of ARV drugs for prophylaxis to HIV-positive pregnant women (UNAIDS and WHO, 2009). Similar to other bilateral and multilateral stakeholders, such as the Global Fund, PEPFAR II strategic plans include expanding access to PMTCT services “as a mechanism to both prevent transmission of HIV to children and support expanded access to care and related services for pregnant women” (OGAC, 2009i, p. 9). PEPFAR II targets, aligned to follow the targets for PMTCT of the Declaration of Commitment on HIV/AIDS, commits to providing “at least 80 percent of the target population with access to counseling, testing, and treatment to prevent the transmission of HIV from mother-to-child.” 39 In FY2009, PEPFAR reported that 509,800 HIV-positive pregnant women received ARV prophylaxis, and estimated approximately 96,862 infant HIV infections averted (OGAC, 2010). In addition, PEPFAR supports other prevention interventions that are “medically accurate, age-appropriate, and targeted to the needs based upon behavior” (OGAC, 2009i, p. 12; PEPFAR, 2008b). PEPFAR’s activities aimed at prevention of sexually transmitted HIV target at-risk youth (OGAC, 2009e), and PEPFAR is supporting prevention programming for young people (10–24 years old) (OGAC, 2009a). Country activities that support “efforts to expand HIV counseling and testing, which are entry points to care and treatment,” also serve orphans and vulnerable children and adolescents (OGAC, 2009a, p. 52). Treatment PEPFAR also serves the needs of children and adolescents through its treatment programs including appropriate pediatric formulations for HIV-infected infants and eligible children (ages 0–14) (see section on Adult and Pediatric Treatment). In addition to improving clinical outcomes and increasing survival in HIV-infected children and adolescents on ART, overall population improvements in access to ART may also have benefits for children and adolescents. Recent studies suggest that improved availability and coverage of ART may reduce the number of children and adolescents who will be orphaned due to HIV in the next couple of years (Mermin et al., 2008; Stover et al., 2008; UNAIDS, 2008; UNAIDS and WHO, 2009). In the FY2009 report to Congress, PEPFAR estimated that the number of orphans averted through PEPFAR-supported treatment programs was approximately 1.6 million through September 30, 2008. However, the availability of data and methods for estimating the effects of ART in protecting children and adolescents from orphanhood continues to be a challenge as more information is gathered on ART coverage and the current needs of child and adolescent populations infected or affected by HIV/AIDS (UNAIDS Reference Group on Estimates Modelling and Projections, 2002, 2007). Care and Support PEPFAR commits to supporting comprehensive care interventions other than ART to meet the new targets of providing care and support for 12 million people infected with or affected by HIV/AIDS, including 5 million children and adolescents orphaned or made otherwise vulnerable by HIV/AIDS, 40 (see section on Care and Support Services). In addition to PEPFAR- 39 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iv). 40 Ibid., §301(a)(2), 22 U.S.C. §2151b-2(b)(1)(A)(iii). 

104 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS supported pediatric care and support activities described here, the specific OVC programs described later are also among the HIV/AIDS care interventions that PEPFAR supports (OGAC, 2006e, p. 3). PEPFAR’s “Guidance for United States Government In-Country Staff and Implementing Partners for a Preventive Care Package for Children Aged 0–14 Years Old Born to HIV-Infected Mothers - #1,” which was released in April 2006, describes PEPFAR-supported activities for children born to HIV-infected mothers (HIV-exposed children), including children in whom an HIV diagnosis has been confirmed (OGAC, 2006c). Although prioritization and selection of the activities in the preventive care package for children (0–14 years old) is country-specific, PEPFAR recommends the following components: diagnosis of HIV infection in infants and young children, childhood immunizations, prevention of serious infections, and provision of nutritional care. PEPFAR’s activities include efforts to increase early identification of HIV exposure and infection status in children. In the FY2009 annual report to Congress, PEPFAR indicated its support in “expanding polymerase chain reaction (PCR) testing to identify the presence of HIV” including “country-level policy change to allow PCR-based dried blood spot testing,” which is thought to reduce the cost and burden of infant diagnosis (OGAC, 2009a, p. 49). PEPFAR also supports clinical pediatric care for children through various sites—facilities and community- or home-based settings—including the prevention and treatment of OIs, TB, and other diseases like malaria and diarrhea through the provision of pharmaceuticals, ITNs, safe water interventions, and related laboratory services (OGAC, 2009a, 2009e). Moreover, PEPFAR supports palliative care interventions for children and adolescents, including pain and symptom relief using age-appropriate interventions and methods of administration for pediatric palliative care (ages 0–14) (OGAC, 2006d). This includes psychological, social, and spiritual support services intended to alleviate the burden of families caring for family members living with HIV/AIDS, particularly for children and adolescents who often are forced to drop out of school to become the breadwinners of the household and care for the ill parent (OGAC, 2009a). Other critical factors in the care of children are nutrition and growth; therefore, the PEPFAR-supported preventative pediatric care package recommends services that address nutritional needs, such as therapeutic or supplementary feeding, replacement feeding, and provision of micronutrient supplements (OGAC, 2006c). Through the new Five-Year Strategy, PEPFAR II emphasizes the need for governments “to expand coverage, and access to quality basic care packages” for people diagnosed with HIV, including HIV-infected children and adolescents, “particularly through integration of care with other health and development programming” (OGAC, 2009i, p. 18). This is especially significant within the current efforts to guarantee the health and survival of children through referrals and follow-up of HIV-infected children for immunizations, as in the case of many PEPFAR-supported care and PMTCT programs linking services to maternal-child health programs (OGAC, 2009c). Programs for Orphans and Vulnerable Children In its guidance document, PEPFAR defines orphans and vulnerable children as children who are “either orphaned or made more vulnerable because of HIV/AIDS.” Further, it defines an orphan as a child who “has lost one or both parents to HIV/AIDS” (OGAC, 2006e, p. 2). A vulnerable child is defined as a child being “more vulnerable because of any or all of the following factors that result from HIV/AIDS: is HIV-positive; lives without adequate adult 

105 CHILD AND ADOLESCENT WELL-BEING support (e.g., in a household with chronically ill parents, a household that has experienced a recent death from chronic illness, a household headed by a grandparent, and/or a household headed by a child); lives outside of family care (e.g., in residential care or on the streets); or is marginalized, stigmatized, or discriminated against” (OGAC, 2006e, p. 2). Because children and adolescents can differ greatly in their needs and individual vulnerabilities, in the past 6 years, PEPFAR has supported “child-centered, family-focused, community-based, and government- supported” OVC programming for these groups in need of further care to facilitate age- appropriate development (OGAC, 2006e, 2009i, p. 22). Therefore, PEPFAR’s OVC programs target different age groups of orphans and vulnerable children and adolescents from age 0 to 17 years (see Table 4). TABLE 4 PEPFAR Age Categories for Programs for Orphans and Vulnerable Children Age Stage <2 years Infancy 2–4 Early Childhood/Toddler 5–11 Middle Childhood 12–17 Late Childhood/Adolescence SOURCE: OGAC (2006e). In addition to these age groups, the expanded program goals under the new Five-Year Strategy highlight the importance of refocusing OVC program efforts to also address the needs of several neglected subset populations such as young adults, and in particular the vulnerability of girls (see the Gender section of this report) (OGAC, 2009g). According to this strategic plan, PEPFAR will help countries to develop initiatives that target “adolescent and young adults as they transition from OVC programs into society and careers” (OGAC, 2009i, p. 23). PEPFAR guidance provides an operational definition and guiding principles for OVC programming decisions; nonetheless “each community will need to prioritize those children most vulnerable and in need of further care” (OGAC, 2006e, p. 2). PEPFAR countries use the Child Status Index tool (MEASURE Evaluation, 2009), developed by Duke University and MEASURE Evaluation with USG support, to assist in-country planning of OVC programming based on these core elements and 12 measurable factors that approximate a standard for child health and well-being (OGAC, 2009a). PEPFAR’s OVC programming guidance identifies important elements of a child’s and adolescent’s life in seven core areas that are based on the principles of the “Framework for the Protection, Care, and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS” (OGAC, 2006e). These factors, described in more detail below, include food and nutritional support, shelter and care, protection, health care, psychosocial support, education and vocational training, and economic opportunity and/or strengthening. PEPFAR also supports OVC programs “that link OVC services with HIV-affected families [through] linkages with PMTCT, palliative care, treatment” (OGAC, 2009e, p. 137). Further, PEPFAR-supported activities under OVC programs include services that directly support orphans and vulnerable children and adolescents as well as their caregivers, families, and community members (OGAC, 2009e). In addition to activities at the level of patient care and the level of the caregiver, PEPFAR also supports activities at the systems level. As emphasized in the new strategic plan, system-wide OVC program activities are aimed at building local, regional, and national capacity to enhance the structures and networks that support healthy child development. This includes 

106 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS efforts by PEPFAR to assist countries in coordinating among ministries overseeing education, social welfare, and health in order to develop policy and program responses that lead to comprehensive and effective care for orphans and vulnerable children and adolescents (OGAC, 2009i). Food and nutritional support OVC programs include nutritional assessments and counseling, therapeutic or supplementary feeding and micronutrient supplementation for HIV-infected children based on national and appropriate international guidelines, and replacement feeding and support for children born to HIV-infected mothers (OGAC, 2006e). Orphans and vulnerable children are one of the priority groups identified by PEPFAR to receive food and nutritional support. In FY2008, PEPFAR reported that 814,800 orphans and vulnerable children received support for food and nutritional supplementation in the 15 focus countries (OGAC, 2009a). PEPFAR-supported efforts to provide food and nutritional support use linkages with non-HIV funding mechanisms, such as USAID Title II (Food for Peace program41 ) and the United Nations World Food Program as well as programs in partner countries (OGAC, 2006g). In FY2010 PEPFAR’s COP Guidance introduced new budget codes for food and nutrition commodities, policy, tools, and service delivery in order to capture information on country activities under this cross-cutting issue (OGAC, 2009a, 2009e). Currently, the only indicator for OVC programs required by OGAC captures the number of eligible individuals under 18 years old who received food and nutrition services (OGAC, 2009d). Shelter and care Because of the growing number of orphans and vulnerable children and adolescents globally, it is necessary to enhance the capacity of families and communities to care for these children. PEPFAR funds can support shelter and care interventions such as: identifying potential caregivers prior to a guardian’s death, family tracing and fostering, providing access to temporary shelter for children in transition, supporting child- or adolescent-headed households in taking care of their homes, supporting access to programs that incentivize adoption or the provision of foster care, and strengthening community-based and family-based care models for children (OGAC, 2006e). Protection PEPFAR OVC programs addressing this core area may focus on interventions such as: health care and social services facilitating basic birth registration and identification, the provision of community-based assistance to orphans and vulnerable children for inheritance claims, the removal of children from abusive situations into safe temporary or permanent placements, and the strengthening of local community structures that are responsible for monitoring and protecting orphans and vulnerable children (OGAC, 2006e). Health care Core services for orphans and vulnerable children address (1) the general health needs of this group, (2) health care for HIV-positive children, and (3) prevention of HIV. OVC programs are required to disaggregate health requirements and interventions by age, and they 41 The Food for Peace program (Public Law 480, also renamed Food for Peace Act of 2008) is the principal mechanism through which the USG implements its international food assistance. Title II of the Food for Peace Act, which authorizes the vast majority of U.S. international food assistance, is managed by the USAID Office of Food for Peace. USAID Peace's implementing partners include private voluntary organizations registered with USAID, local and international nongovernmental organizations, and the United Nations World Food Program (USAID, 2009a). 

107 CHILD AND ADOLESCENT WELL-BEING should facilitate access to primary health care for orphans and vulnerable children (OGAC, 2006e). “General health” interventions include: referrals to child health care, the provision of support for survivors of abuse, the training of caregivers to monitor children’s health, and capacity building of public and private health providers (OGAC, 2006e). PEPFAR programs provide health care to HIV-positive children, including HIV-exposed children, through direct access to health providers, or through referrals to prevention and treatment services. Psychosocial support Children and adolescents affected by HIV/AIDS suffer anxiety, fear, grief, and trauma with the illness or death of a parent. PEPFAR programs are intended to address the psychosocial and life skills needs of orphans and vulnerable children, including gender- sensitive life skills and experiential learning opportunities; improved links between children affected by HIV/AIDS and their communities; rehabilitation for children who are living outside of family care; and referral to counseling where available and appropriate, particularly for HIV- positive youth (OGAC, 2006e). Education and vocational training Partnerships with the education sector on national and local levels provide an important opportunity to ensure that children and adolescents affected by HIV/AIDS have access to education. Linkages with other programs, such as the USG basic education program or the African Education Initiative (AEI) implemented through USAID, can help expand educational opportunities. For children and adolescents, the USG is providing $400 million through AEI to train 500,000 teachers and provide scholarships for 300,000 children and adolescents, particularly young girls (OGAC, 2009a). Other activities within this core area that are supported by PEPFAR funding include activities that encourage access for orphans and vulnerable children into early childhood development programs, vocational training, activities to integrate orphaned and vulnerable children into community social life, and anti-stigma education. PEPFAR’s efforts in this core area also extend to interagency activities, such as the Interagency Education Steering Committee and other strategic planning for the expansion of education wraparound programs that target HIV-infected children and adolescents, as well as those made vulnerable by HIV/AIDS (OGAC, 2009a). Economic opportunity and strengthening PEPFAR programs fund economic strengthening services so that caregivers can meet their responsibilities to ill family members or receive orphaned children into the household. For instance, PEPFAR supports activities that promote the entrepreneurism of caregivers of orphans and vulnerable children through microfinance programs, small-business development, and programs for community-based asset building (OGAC, 2008a). Programs also provide orphans and vulnerable children and adolescents with training and other skills to improve their future economic opportunities (OGAC, 2006e). Objective and Scope The IOM evaluation will assess PEPFAR’s progress toward meeting its programmatic targets and strategic goals for children and adolescents, including efforts to increase the number of HIV-infected children receiving treatment and the number of orphans and vulnerable children and adolescents receiving care and support services. Since PEPFAR provides services for children and adolescents through prevention, treatment, and care strategies for children and adolescents, as well as through specific OVC programming, activities in all of these areas, to the 

108 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS extent that they serve children and adolescents, will be a part of the assessment of PEPFAR’s effects on the well-being of children and adolescents. The evaluation committee, in its review of PEPFAR-supported programs aimed at the age-specific needs of children and adolescents infected and affected by HIV/AIDS, will consider the appropriateness of programmatic guidance and activities in providing effective care and support to improve the health and psychosocial well-being of these populations. This includes assessing the effect of services provided to children and adolescents in each of the programmatic areas, as well as efforts in supporting family-centered programs and community-led initiatives that respond to the needs of orphans and other vulnerable children and adolescents in PEPFAR countries. The evaluation committee will consider how PEPFAR’s efforts to guide countries to implement international standards of care for children and adolescents, including those orphaned or made vulnerable by HIV/AIDS, have resulted in measurable effects on the well-being of children and adolescents in PEPFAR countries. This will include a review of PEPFAR-supported initiatives such as the development and implementation of national plans for orphans and vulnerable children and other policies related to improving survival and healthy development of children and adolescents. Program Impact Pathway An impact pathway (Figure 14) summarizes how the IOM evaluation committee proposes to examine the strength of evidence establishing plausible causal links between PEPFAR programs for children and adolescents and their intended impacts. This impact pathway reflects that significant impacts of PEPFAR services in one setting (such as health) may emerge in other service settings (such as education or child welfare). The impact pathway framework will help the committee understand the changes at each stage, in order to describe the relationship between the processes of interventions and their effects. Given the timely availability of data, the evaluation committee will examine whether PEPFAR-funded activities have had an effect on the well-being of children and adolescents through an assessment of mediating output and outcome indicators or intermediate measures of child and adolescent well-being as defined by the committee. 

FIGURE 14 Program impact pathway for evaluation of PEPFAR’s services for children and adolescents. 109 

110 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions The limitations in evaluating the effects of PEPFAR’s activities on the well-being of children and adolescents include differences in definitions with respect to the age categories of children, adolescent, youth, and young people (UNAIDS Reference Group on Estimates Modelling and Projections, 2002). While PEPFAR defines age-specific categories in some areas, such as pediatric treatment and OVC programming (Table 4), PEPFAR, country-level, and international data-collecting systems do not all use consistent age categories. However, international efforts to harmonize indicators for M&E of interventions targeting these populations, including orphans and vulnerable children under the age of 18 years, will provide some data for the evaluation of PEPFAR activities. Another potential limitation is the limited availability of age-disaggregated data for many key outcome indicators. The evaluation committee will review program-level data and country- level data to seek sources of age-disaggregated data. However, although countries are increasingly collecting age- and sex-disaggregated data systematically from programs, in particular from those strategically targeting children and adolescents, in many places there still are not enough data to assess and evaluate these programs (UNICEF, 2009). This will be an important issue not only for this evaluation but also during the implementation of PEPFAR II, in particular within the context of implementation plans for partnership frameworks, which under the reauthorization legislation need to contain age- and sex- disaggregated data for “an identification of the intended beneficiaries […] including information on orphans and vulnerable children, to the maximum extent practicable.” 42 In their latest review of PEPFAR-supported care and support activities for children and adolescents, PEPFAR’s Pediatric TWG indicated that data on current progress in care and support are not disaggregated by age, HIV exposure, or infection status (OGAC, 2009c). Moreover, they indicate that international indicators, such as UNAIDS globally harmonized indicators, provide only data on scale-up and coverage of particular priority program elements and limited data on receipt of care and support services by HIV-infected children (OGAC, 2009c). Finally, OVC programs may be offered within different settings in which eligible children and adolescents may receive multiple services. This means that there is a risk of a single child being counted several times by different implementing partners, which can make it difficult to determine if targets expressed as “number of children receiving services” are being met (OGAC, 2009d). The lack of unique identifiers for each participant in PEPFAR activities constitutes a major methodological challenge for program evaluations in this context. The following are examples of questions that the committee will consider in the evaluation. These questions are intended to contribute to addressing part B, items v, vi, and vii of the areas for consideration in the congressional mandate. 43 How are services supported by PEPFAR tailored to the cultural context for the children and adolescents presenting with needs? Specifically, how are these activities tailored or 42 Supra., note 6 at §301(d)(2), 22 U.S.C. §2151-b2(e)(2)(E)(iv). 43 (B)(v) an evaluation of the impact of prevention programs on HIV incidence in relevant population groups (B)(vi) an evaluation of the impact on child health and welfare of interventions authorized under this Act on behalf of orphans and vulnerable children (B)(vii) an evaluation of the impact of programs and activities authorized in this Act on child mortality 

111 CHILD AND ADOLESCENT WELL-BEING targeted to specific age cohorts among populations of children and adolescents orphaned and made vulnerable by HIV/AIDS? The committee will try to assess the cultural sensitivity and age-appropriateness of PEPFAR programs as they attempt to address culturally entrenched beliefs and practices that increase the risk-taking behavior of children and adolescents orphaned or made vulnerable by HIV/AIDS. The committee will rely on qualitative data to ensure that interventions for children are contextually relevant and that PEPFAR program interventions are responsive to variances in high and low HIV prevalence areas. What criteria are used by PEPFAR countries to establish priorities and determine the balance among prevention, treatment, and care interventions for HIV-infected infants, children, and adolescents including those orphaned or made vulnerable by HIV/AIDS? How effective are multi-sector strategic planning and implementation for child and adolescent intervention efforts? Is PEPFAR engaging social, behavioral, and prevention scientists in guiding programs, strategies, and analysis targeting infants, children, and adolescents? According to PEPFAR’s stated goals, treatment as well as care and support programming targeting HIV-infected children need to be consistent with the pediatric HIV burden in a community but also with the country’s continuing sustainability of PEPFAR programs. In order to address these goals, the committee will need to understand how funding priorities are determined for these three main program areas with regard to children and adolescents. The committee will seek to understand the different funding allocation decisions within these programmatic areas, as well as changes in funding over time, through a review of PEPFAR’s operational plans and a sample of the COPs. Key informant interviews during country visits will be a complementary source of information. What is the responsiveness and adequacy of psychosocial care services to meet the needs of eligible HIV-positive children and adolescents orphaned or made vulnerable by HIV/AIDS? PEPFAR recommends countries collect data on the number of orphans and vulnerable children provided with psychological, social, or spiritual support. The committee will rely on qualitative data to assess the “responsiveness and adequacy” of these services as programs do not provide relevant quality data to address these specific issues. To address PEPFAR’s efforts in this core OVC program area, the committee will try to evaluate the responsiveness, screening, and referral of children and adolescents to appropriate psychosocial support providers at different stages of need. Efforts will also be made to evaluate the engagement of expert guidance by country programs in determining best practices for addressing the psychosocial needs of children and adolescents orphaned or made vulnerable by HIV/AIDS. To what extent have PEPFAR-supported OVC programs had an effect on the educational enrollment of orphans and vulnerable children and adolescents? 

112 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Are PEPFAR efforts supporting policy or programs aimed at mitigating the consequences of children and adolescents, particularly young women, having increasing socioeconomic responsibilities in the household due to the illness of a parent or both parents living with HIV/AIDS? To what extent are programs averting orphanhood and over what period of time? What economic support programs are in place to generate income for adult non-parental caregivers to support orphans and vulnerable children and adolescents in their households? The committee will review available data to assess other cross-cutting socioeconomic interventions such as vocational and technical training, small-business development, and household economic-strengthening workshops. The committee will need to access data disaggregated by age and sex in order to assess the effects of PEPFAR’s economic strengthening programs on the socioeconomic needs of these groups at different developmental stages, particularly in young women. What activities does PEPFAR support for in-service training and pre-service training for child and adolescent service providers and facilities? The long term benefits of PEPFAR on the well-being of children and adolescents, including those orphaned or made vulnerable by HIV, will largely depend on the capacity of country programs to assume responsibility for lessening the negative effects of HIV/AIDS in the communities where they live. Thus, the committee will assess qualitative and quantitative data to evaluate whether PEPFAR programs are investing in building the capacity of district and national authorities and communities to respond to the needs of these vulnerable groups. What is the current state of child protection legislation in the community? Are activities supported by PEPFAR contributing to changes in implementation and enforcement? Some of the PEPFAR-supported activities include training workshops on children’s rights, as well as will writing and succession planning for children, their guardians, and for PEPFAR program staff. PEPFAR programs also link children, adolescents, and their guardians to appropriate legal services. The committee will seek to assess the effects of PEPFAR activities on current or proposed legislative reforms that incorporate child and adolescent protection measures and support protection programs mainly through primary collection of qualitative data during country visits and a review of publicly available documents. To what extent have PEPFAR policy initiatives contributed to changes in national health priorities for children and adolescents? Are PEPFAR efforts resulting in the development and adoption of national strategic plans for orphans and vulnerable children? What has been the progress of the country programs to implement the plans? 

113 CHILD AND ADOLESCENT WELL-BEING A primary measure will be whether PEPFAR programs for children and adolescents orphaned or made vulnerable by HIV/AIDS are an integral part of national HIV/AIDS strategies and plans. The committee will review a range of policy documents (see Box 2a-d) and conduct interviews to assess whether these strategies and/or planning activities are designed to engage different sectors or agencies within the national government dedicated to child and adolescent well-being. The committee will seek any available data from OGAC and governments such as partnership framework implementation plans. 



SECTION 7: GENDER-RELATED VULNERABILITY AND RISK The global HIV/AIDS epidemic cannot be fully addressed without considering the ways in which societal gender norms, which can be described as “how societies define acceptable and customary roles, responsibilities, and behavior of women, girls, men, and boys,” contribute to women’s and men’s vulnerability in the epidemic (AIDSTAR-One, 2009, p. 1). For example, gender inequalities can limit the ability of women to exert control over their sexual choices, while societal norms may encourage men to engage in riskier sexual behavior. Norms can inhibit access for both men and women to obtain testing and treatment due to, for example, fear of reprisal by a partner or the nature of service provision. In addition, socially defined responsibilities may lead to an increased burden, typically for women, associated with caring for individuals affected by HIV and AIDS (UNIFEM, 2006). A focus on gender inequality and how to reduce it thus has been recognized to be an essential ingredient of HIV prevention, treatment, and care. Current priorities among stakeholders in the global HIV/AIDS community in reducing gender-related vulnerabilities are focused on a variety of country-level strategies. These include the passage and implementation of legislation to protect women, girls, and other vulnerable groups (such as MSM) from gender-based violence, bias, discrimination, and stigma; programs to address male norms; and efforts to empower women through increasing their income generation opportunities (AIDSTAR-One, 2009; UNAIDS, 2010a; UNIFEM, 2006). This section provides a brief background on gender-related programming in the context of PEPFAR and outlines the committee’s approach to assessing PEPFAR’s efforts toward addressing these complex issues. However, it is important to note that unlike other aspects of addressing HIV and AIDS, issues associated with gender-related risk and vulnerability cut across all of PEPFAR’s implementation, and do not currently comprise their own unique programmatic or funding category. In addition, while ongoing efforts to highlight the differential needs of men and women have been underway since the beginning of the program, addressing issues associated with gender norms, violence, and stigma are still in the early stages in many PEPFAR countries. PEPFAR’s Efforts on Gender to Date The original legislation authorizing the creation of PEPFAR and the first Five-Year Strategy acknowledged the importance of addressing the unique vulnerability of women and girls, most notably through prevention efforts, but did not outline any specific targets for addressing gender-related issues (OGAC, 2004). Recognizing the centrality of gender inequality in the AIDS epidemic, PEPFAR subsequently convened a gender TWG in 2005 and submitted a report to Congress on gender-based violence and other gender programming activities in 2006, which emphasized the need to address challenges beyond those associated with access to care, including gender norms and linking services to reduce HIV risk among victims of gender-based violence (OGAC, 2006f). The Lantos–Hyde Act of 2008 expanded the program’s mandate with respect to women and called for the development of a plan to address the particular vulnerability of women and girls and improve research in areas related to gender programming. Working from the reauthorization legislation, the gender TWG also emphasized the following five strategic areas to guide program development and implementation (OGAC, 2009c): 115 

116 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS 1. Increasing gender equitable access to prevention activities and services 2. Reducing violence against women, coercion, and the exploitation of women and girls by sex trafficking, rape and sexual abuse and providing post-rape prophylaxis 3. Addressing male norms and behaviors 4. Increasing women’s legal rights and protections 5. Increasing women’s access to income and productive resources The recently released new PEPFAR Five-Year Strategy reaffirmed the program’s commitment to gender equity in its prevention, treatment, and care services and to scaling up programs to address gender-based violence (OGAC, 2009g). It also commits the program to work with countries in establishing initial targets and reporting mechanisms to track outcomes for activities targeted at gender needs (OGAC, 2009h). Objective and Scope The IOM evaluation committee will examine the progress made toward incorporating the ideals laid out by the reauthorization legislation and Five-Year Strategy into program guidance, planning, and budgeting processes. The committee will also assess the extent to which PEPFAR has increased its gender programming to include activities aimed at reducing gender-based violence, addressing male norms, empowering women through behavior change and income generating activities, and improving equity in access to prevention, treatment, and care services. Program Impact Pathway The following impact pathway (Figure 15) illustrates the committee’s understanding of how activities consistent with PEPFAR’s stated overarching gender goals could lead to outputs, outcomes, and health impacts in the countries in which PEPFAR operates. As described in the previous section, efforts to address gender-related issues are still being developed in many countries and are not incorporated in a systematic manner into country program planning. Thus, the activities listed do not represent PEPFAR-wide initiatives, but rather serve as examples of programs that might be undertaken, or are already underway as pilots in some PEPFAR countries. Due to these realities, the committee’s assessment will focus primarily on the process of developing gender-related activities and, where possible, the outputs achieved from their implementation. However, the ability to access data on outputs is expected to vary substantially by country and may not be feasible in many settings. The committee may also pursue data from external sources (including from multilateral partners such as UNAIDS), existing studies, and qualitative analysis during country visits. 

FIGURE 15 Program impact pathway for evaluation of PEPFAR’s gender efforts. 117 NOTES: PHE = public health evaluations; PMTCT = prevention of mother-to-child transmission. 

118 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Illustrative Questions The following are illustrative examples of the types of questions related to gender activities that the committee may assess given the timely availability of quality data. These questions are intended to contribute to addressing part B, item vii of the areas for consideration in the congressional mandate. 44 There are currently no PEPFAR reported, required, or recommended indicators that are specific to measuring efforts toward changes in gender norms, vulnerability related to gender, or empowerment. However, a number of prevention, treatment, and care indicators are required to be disaggregated by sex and/or age, which can provide information related to access rates and equity in service provision. In addition, due to potential variability in the definition and understanding of gender-related activities across programs, there will likely be substantial variation both within and between countries. As this represents an evolving area of programming, case studies may provide useful information at this stage. Are women and girls accessing prevention, treatment, and care programs in equal numbers to men and boys? Sex-disaggregated PEPFAR indicators (such as the number of people accessing prevention, treatment, and care programs) will be used where available. The committee may also assess whether these services are being delivered in a comparable and appropriate manner to women and men, using qualitative data collected during country visits. Has PEPFAR made special efforts to reduce the sexual risk of adolescent girls and young women (e.g., through early marriage, transactional, or cross-generational sex, etc.)? Has PEPFAR made efforts to address the sexual risk of older women (e.g., through wife inheritance by relatives of late brothers, cleansing rites, etc.)? Have these efforts been successful? As with assessing male norms, the committee may use structured interviews with key informants from the OGAC, program, and project levels to determine the incorporation of efforts to reduce women’s sexual risk into program planning and implementation. When available at the country level, the committee will pursue access to a limited number of recommended PEPFAR indicators, including those related to cross-generational sex. In addition, there are a number of global level indicators collected by organizations and survey efforts (including UNAIDS, national demographic and health surveys, and behavioral surveillance surveys) that provide information regarding this topic (Measure DHS, 2010). The committee will assess the feasibility of incorporating information from these types of sources to provide contextual data on the countries in which PEPFAR operates. How many programs integrate or are linked to programs that seek to economically empower women? What has been their impact in improving women’s income, savings, or access to credit? 44 (B)(iii) an assessment of efforts to address gender-specific aspects of HIV/AIDS, including gender related constraints to accessing services and addressing underlying social and economic vulnerabilities of women and men 

119 GENDER-RELATED VULNERABILITY AND RISK Currently, activities that promote economic empowerment and income generation are included under the umbrella of care services provided by PEPFAR, and indicators such as “number of people provided with a minimum of one care service” disaggregated by sex may provide some information as to the existence of these programs. However, in order to gain a more comprehensive picture of the success and coverage of these types of activities, the committee will seek other sources of data at the project level where available. How many programs has PEPFAR supported to address male norms and behaviors? What criteria are used to determine whether such programs should be a part of a COP? How effective have these programs been in fostering more gender equitable attitudes among men? The availability of activity type data at the OGAC level will determine the committee’s ability to assess the efforts of PEPFAR toward addressing male norms. Interviews with OGAC, country, and project level staff may be used to determine how these types of activities are addressed in the program planning and reporting process. However, there are no PEPFAR indicators for measuring impact of programs to change male behavior or norms. Thus, the evaluation committee will use project level data where it exists. How many country programs include programs to reduce gender-based violence? How and to what extent have they succeeded in reducing incidence of violence? There are no PEPFAR indicators specific to measuring gender-based violence, so the committee will use country data and project level data where it is available. Data may also be gathered from key informants who run these programs. In addition, the committee will attempt to follow-up at the OGAC level on the “Report on Gender-Based Violence and HIV/AIDS” that was submitted to Congress in 2006, to assess the potential of updating the information presented. Has PEPFAR made efforts to influence national policy and legal environments that affect the vulnerability of women and girls? How have they been successful? What impact have these efforts had on decreasing this vulnerability? Efforts toward positively influencing policy and legal reform efforts will be assessed through interviews with country team and OGAC staff. In addition, for the countries for which they have been completed, the newly formed Partnership Frameworks may be reviewed for the presence of gender-related strategies as they are intended to reflect national political and legal priorities. Given the often protracted nature of policy implementation and enforcement, an evaluation of the potential impact of these efforts may not be feasible. 



SECTION 8: KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Reauthorization Legislation Shifts Priority to Sustainability The Lantos–Hyde Act of 2008 specifically required PEPFAR’s strategy to “include a longer-term estimate of the projected resource needs to progress toward greater sustainability and country ownership of HIV/AIDS programs […] during the 10-year period beginning on October 1, 2013.” 45 The new PEPFAR Five-Year Strategy stated that management of the response to the disease and its effects must increasingly be led by countries, with support from bilateral and multilateral partners. Countries also need to increasingly own the process of monitoring, evaluating, and responding to the unique characteristics of the epidemic in their country (OGAC, 2009g). This section therefore outlines how the evaluation committee will explore the role of PEPFAR programs in facilitating improvements to health systems in partner countries and the readiness of PEPFAR and partner countries to increasingly share responsibility for managing the response to the epidemic and move to greater sustainability and country ownership. Definitions Neither the authorizing legislation nor the PEPFAR strategy defines sustainability. For the purposes of this evaluation, one of several definitions proposed by the OECD Development Assistance Committee may be used; it defines sustainability as “the continuation of benefits from a development intervention after major development assistance has been completed” (DAC Network on Development Evaluation, 2002). While continuation of benefits into the future is the ultimate goal, a number of intermediate outputs or outcomes can be posited to improve sustainability: Affordability: The extent to which countries can bear the cost of programs. Efficiency/cost-effectiveness: “A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results” (DAC Network on Development Evaluation, 2002, p. 21). Country capacity: The ability of government, the private sector, and civil society to “plan, manage, implement, and account for results of policies and programs” (High-Level Forum, 2005, p. 5). 46 Country “ownership”: A situation in which “partner countries exercise effective leadership over their development policies and strategies, and coordinate development actions”(High- Level Forum, 2005, p. 3). Coordination and harmonization 47 with donors and governments: The extent to which donors “implement, where feasible, common arrangements at country level for planning, funding (e.g., joint financial arrangements), disbursement, monitoring, evaluating and reporting to government on donor activities and aid flows.” (High-Level Forum, 2005, p. 6). 45 Supra., note 6 §101(a), 22 U.S.C. §7611(a)(29). 46 The Paris Declaration does not specify whose capacity within countries this defines, but it is inferred to be the government’s capacity. Thus, this proposed definition is somewhat broader. 47 As the extent to which PEPFAR has contributed to harmonization is being evaluated by the U.S. Government Accountability Office, it will not be explicitly addressed in this proposed evaluation. 121 

122 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Partnership Frameworks to Promote Sustainable Approaches The Lantos–Hyde Act of 2008 permitted the USG to establish framework documents (Partnership Frameworks) with countries to promote a more sustainable approach of the USG’s global efforts against HIV/AIDS, malaria, and TB “that is characterized by strengthened country capacity, ownership, and leadership” (OGAC, 2009b, p. 3). Further, these 5-year joint strategic frameworks between the USG and partner governments are meant to intensify focus on cooperation through “technical assistance and support for service delivery, policy reform, and coordinated financial commitments” (OGAC, 2009b, p. 3). At the end of the 5-year time frame, the expectation is that “country governments will be better positioned to assume primary responsibility for the national response to HIV/AIDS in terms of management, strategic direction, performance monitoring, decision making, coordination, and where possible, financial support and service delivery” (OGAC, 2009b, p. 3). The axiom of “do no harm” has been adopted by OGAC for continued support of existing implementing partner service delivery systems to continue to provide quality services while this transition to county ownership occurs over time. There are additional expectations of transparency, accountability, and engagement of multiple stakeholders in the country. As for policy reform and financial commitments, the Partnership Frameworks are supposed to emphasize policy areas identified by the government and civil society that require additional or focused attention and overall accountability for resources and appropriate budgeting in HIV programs. They also provide a capacity building opportunity for the USG to assist countries in managing multiple funding sources. Some countries, based on their resources, are expected to increase their financial contributions over time. The Partnership Frameworks also provide an opportunity for the USG to provide technical assistance to countries for improved monitoring and tracking of overall health spending (including HIV/AIDS) from different sources, including financial monitoring and reporting systems (OGAC, 2009b). Although the reauthorization legislation did not define sustainability per se, PEPFAR did define how to promote sustainability in the Partnership Frameworks and Partnership Framework Implementation Plans guidance issued in September 2009—based on the principles of the Three Ones, the Paris Declaration, and the Monterey Accord of 2002 (activities that facilitate financing for development): “For purposes of the Partnership Frameworks, promoting sustainability means supporting the partner government in growing its capacity to lead, manage, and ultimately finance its health system with indigenous resources (including its civil society sector), rather than external resources, to the greatest extent possible” (OGAC, 2009b, p. 4). The Partnership Frameworks are distinct from the annual work plans for USG-supported intervention, the COPs. However, COPs are expected to reflect the Partnership Framework principles and the transition strategy outlined in the Partnership Framework Implementation Plans. The implementation plans have minimal required elements, including “an analysis of how the existing portfolio of USG- supported, NGO-implemented programs will transition to the partner government, remain NGO- based, or be terminated within the 5-year timeframe” (OGAC, 2009b, p. 6) and “a description of the approach to supporting increased country ownership, baseline data, specific strategies for achieving the 5-year goals and objectives, and a monitoring and evaluation plan” (OGAC, 2009b, p. 8). It also seems reasonable that cost efficiencies for the future national response can be identified by both PEPFAR and the partner government during the implementation of the Partnership Frameworks and by the end of the 5-year performance period, when countries are 

123 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES expected to assume primary responsibility for the national HIV/AIDS response, even if the partner government does not assume full responsibility of financing its health system. The goals and objectives would be measureable goals for the USG and all partners in the Partnership Framework. As such, the Partnership Frameworks are expected to identify indicators to assess progress toward meeting the goals, objectives, and programmatic and financial commitments—with an eye toward international efforts to harmonize indicators. PEPFAR- specific reporting systems are expected to be transitioned to nationally- and country-owned systems. This step would be in full support of the Third One—one agreed HIV/AIDS country- level M&E system (OGAC, 2009b). Lastly, the guidance also states that country governments should be developing the “capacity to support all relevant components … of a multi-sector health system” (OGAC, 2009b, p. 4). These components are the six areas of the WHO six-building block framework for effective health systems (see Figure 16) that has been endorsed by PEPFAR. Program and Health System Interaction and Integration The most widely accepted definition of a health system was proposed by the WHO—“all organizations, people, and actions whose primary intent is to promote, restore, or maintain health” (WHO, 2007a, p. 2). A health system therefore includes both public and private (for- profit and not-for-profit) providers, which may be either community- or facility-based. Identifying both well-planned synergies and unintended antagonisms with national health systems that may have resulted from the implementation of a program like PEPFAR requires a systems model, as interventions in one part of the health system may have impacts in others. The WHO Positive Synergies working group took such an approach, resulting in several recommendations that are relevant to PEPFAR: prioritize health system strengthening, agree on and track health system strengthening indicators, align resource allocation between global health initiatives and country health systems, generate more reliable data for the costs and benefits of strengthening health systems, and commit to increased national and global health financing that is more predictable to support sustainable and equitable growth of health systems (Samb et al., 2009). While global HIV prevalence stabilized in 2000 (Bongaarts et al., 2008; UNAIDS, 2008), the absolute burden of HIV treatment demand will continue to grow each year for the foreseeable future. (UNAIDS, 2008; UNAIDS and WHO, 2009). With newly-infected people still outpacing both AIDS-related deaths and numbers of people being put on ART, it is increasingly clear that treatment alone will not be a sufficient response to control the epidemic (Bertozzi et al., 2009). Achieving sustainable HIV programs not only requires health system strengthening, but also successful scale-up of effective HIV prevention strategies to avert continued growth in the HIV treatment burden in health systems that are already overburdened. As previously mentioned in the Mapping of PEPFAR Funding section, scaling-up does not necessarily mean just increased spending, so understanding unit costs of HIV-program delivery may help elucidate variation in health system capacity, efficiencies, and quality. Also program and data management and health information systems are needed to more completely assess population impact rather than just process measures. 

124 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS Capacity Building As has been noted, the capacity to scale up programs in low-income countries is limited by resource constraints, including lack of trained health workers and fragile health systems. PEPFAR II therefore added an additional target of training 140,000 new health workers to support the capacity of countries to improve “the overall quality of their services … and build capacity to plan, manage and sustainably finance their health systems” (OGAC, 2009h). Investing in healthcare workers and health systems increases the likelihood of more people receiving prevention and treatment services for HIV and of achieving a broader health impact (PEPFAR Reauthorization Action Team, 2010). Joint Activities Between PEPFAR and the Global Fund As seen in the statement of task in Appendix A, Congress mandates the committee to evaluate the impact on health of prevention, treatment, and care efforts that are supported by U.S. funding, including multilateral and bilateral programs involving joint operations. Further discussions among IOM staff, OGAC, and congressional staff 48 clarified “multilateral and bilateral programs involving joint operations” to mean programs operated in conjunction with bilateral funding through PEPFAR and the Global Fund (Bressler, 2009; Marsh, 2009). In the preliminary research of the planning committee, this includes financing of ART and procurement and supply management of ARVs and other commodities for its Voluntary Pooled Procurement (discussed in the Adult and Pediatric Treatment section of this report), as well as prevention and care activities (PMTCT and TB treatment, respectively discussed in the Prevention and Care and Support Services sections). It also includes time-limited, outcome-oriented technical assistance through centrally-funded grants from OGAC to Global Fund recipients with active grants (which are not necessarily PEPFAR recipients). The USG Global Fund Technical Support Advisory Panel advises OGAC headquarters on these technical assistance activities. Begun in 2005 under the Grants Management Solutions project, this technical support is intended to (1) improve the functioning of Global Fund grants, (2) strengthen local capacity, and (3) alleviate specific bottlenecks to address under-performing Global Fund grants. The main areas identified with inadequate or poor performance were organizational development (including governance and leadership), financial management, procurement and supply management, and M&E (Coleman, 2007; PEPFAR, 2009b). These four areas correspond to four of the six blocks in the WHO building-block model for effective health systems (see Figure 16). There are stated goals for improvement for each of the four areas of technical assistance. In July 2008, a USG Global Fund Technical Support Evaluation was conducted, which showed achievements through 2008 in each of the areas from the $12 million available in FY2005 technical support through the $31 million available in FY2007 (Coleman, 2010). The committee will examine the 2008 evaluation, including its methods and findings and will also request available country/recipient progress reports as well as copies of the grant requests from recipient countries. 48 Supra., note 10. 

125 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Evaluation Strategy As part of an evaluation of the impact of PEPFAR, it is necessary to determine partner country readiness to make this transition to sustainability. To assess partner country readiness, the evaluation committee aims to examine the partnership frameworks with partner countries. In addition, the committee will assess PEPFAR efforts and synergy with other global stakeholders for capacity building and technical assistance, including activities similar to those identified by the International Health Partnership, such as donor funding harmonization and collaboration between international actors and developing countries to develop and implement national health plans. The committee will also establish working definitions, define data sources, and identify evaluation methodologies for the key parameters of sustainability that include country ownership and local capacity building, including health system strengthening and healthcare workforce expansion. However, it has been noted that “little [global] consensus has emerged to provide uniform guidance” to indicators that can track these activities, although the development of these indicators is an area of intense international interest and activity. PEPFAR defines indicator and reporting requirements in health systems strengthening “to reflect a more narrow scope of interest tied to PEPFAR’s focus on HIV” (OGAC, 2009d, p. 199)—resulting in selection of two indicators to be reported centrally to OGAC that reflect laboratory and health workforce strengthening. Evaluating data for other parameters will require metrics not currently available from routine data sources or the PEPFAR indicator database (OGAC, 2009d), including extra- health sector factors such as good governance (Dybul, 2009), to permit the committee to assess performance of current or past PEPFAR activities. The committee will also examine Partnership Frameworks and Implementation Plans to assess what the countries and OGAC are responsible for, how it is being measured or tracked, and how the processes evolve for PEPFAR-related responsibilities and activities to be transitioned to country leadership for sustainable programs and positive impacts on individual and population health. Health System Frameworks PEPFAR’s increased focus on health systems is shared with other global health initiatives, including The Global Alliance for Vaccines and Immunisation (2010) and the Global Fund, as well as the World Bank and the WHO. The committee will benefit from and incorporate deliberations among these agencies that will address both how to define health systems and how to measure progress toward strengthened health systems (Frenk, 2010; Shakarishvili, 2009). Adopting a common conceptual framework is a key requirement for the evaluation of health systems strengthening and is important to framing such measures. The most widely adopted framework, proposed by the WHO in 2007, is based on operational “building blocks” of the health system (Figure 16)—services, workforce, information, commodities and technologies, financing, and leadership/governance (WHO, 2007a). These building blocks serve several functions including describing what a health system should have the capacity to do in each of these blocks to lead to improved health, system responsiveness, improved efficiency, and social and financial risk protection. It also lends itself well to a descriptive set of activities that may be undertaken through PEPFAR programs. The importance of “systems thinking” has also been emphasized, because health systems are complex, dynamic, and non-linear systems whose 

126 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS function is dependent on the interplay of all of its components, including “the spaces in between” (Atun and Menabde, 2008; de Savigny and Taghreed, 2009). FIGURE 16 Representation of WHO’s six building blocks for effective health systems. SOURCE: Adapted from WHO, 2007a. 

127 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Program Impact Pathway The committee will utilize an impact pathway (Figure 17) to assess PEPFAR inputs, output, and outcomes where measurable, within the context of the WHO building blocks framework for health systems strengthening. The pathway can also help assess the same block elements for technical assistance and other activities related to country readiness for assuming increased to total responsibility for their HIV/AIDS response. Although a useful evaluation tool, the linear nature of an impact pathway is a simplified representation of the reality of PEPFAR programs and their impact, and the committee found this to be particularly challenging in the area of systems-level activities. Activities at the systems level are intended to result in an outcome or impact on the health system, but these are also critical inputs to all other programmatic areas. Thus, the functioning of the health system can be both a starting and end point for a pathway. The committee grappled with illustrating the complexities, dynamism, and non-linear nature of health systems with this linear pathway—analogous to forcing a “square peg in a round hole” —but ultimately recognized the utility of the impact pathway to evaluate the process by helping to frame the areas of inquiry and the measures that may be undertaken to assess health system strength. The findings will be interpreted in light of the more complex realities when the evaluation committee draws conclusions and makes recommendations. As previously mentioned, PEPFAR II (as well as the GHI) has adopted the WHO six building block framework to assess its capacity building and programmatic impact on health systems, categorizing contribution to (1) core HIV activities, (2) secondary benefits or intentional spillover effects of PEPFAR activities on other programs, and (3) targeted leveraging including partnerships. The six building block elements are denoted in the figure with blocks in all uppercase letters. The committee’s working definition of an intervention that strengthens health systems is one that improves the activities and processes within the six health system building blocks, and manages interactions within the “building blocks” over time to achieve more equitable and sustained improvements in population health outcomes. These effects may be either short-term (e.g., better trained workforce delivering higher quality care) or longer-term (e.g., higher-quality care results in a healthier population that is economically more productive), thus growing resources that feed back into the causal chain as an input to the health system. Additional outcomes of an effective health system (e.g., social and financial risk protection for beneficiaries accessing and utilizing services or increased block or system efficiency) can result in increased responsiveness and potentially expanded service coverage when the activities within and among the blocks dynamically interact (see Figure 17). As previously mentioned, interventions in one part of the health system may have impacts in others. Since any intervention may affect the entire system, evaluation is, by definition, context-specific and requires a mix of evaluation strategies, both quantitative and qualitative. Although improved health is an important impact of health systems strengthening, additional measures regarding health system efficiency and equitable distribution of benefits may likewise be important to track as they are developed and adopted by international working groups. 

128 FIGURE 17 Program impact pathway for evaluation of PEPFAR’s health systems strengthening activities at the country level. NOTES: Upper case headers indicate the WHO’s six building blocks. All of the elements listed in the pathway can be categorized under one of the six building blocks, but due to space limitations, select elements were chosen to illustrate blocks within the pathway. ARV = antiretroviral drugs; ART = antiretroviral therapy; CD4 = cluster of differentiation 4; EQA/PT = external quality assurance/proficiency testing; M&E = monitoring and evaluation. 

129 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES Illustrative Questions The main questions that the committee will consider in the evaluation include the potential positive and negative impact of PEPFAR activities on country-level health system functioning, with regards to both HIV and non-HIV programs. Examples of the kinds of questions the committee may ask that specifically link the WHO six building blocks to our results chain framework include the following. These questions are intended to contribute to addressing part B, item ii of the areas for consideration in the congressional mandate. 49 Because health systems are a fundamental aspect of all program activities, these questions are also intended to contribute to addressing all of the areas for consideration described in the Statement of Task (see Appendix A). Finance: To what extent has PEPFAR funding and technical assistance for sound public finance systems resulted in more efficient and equitable care in HIV and non-HIV health systems? To what extent do the joint funds from the Global Fund support the six building blocks? To what extent have the technical support funds from PEPFAR to the Global Fund improved the performance of grant recipients in general and how is this measured? For the four main areas of technical assistance which are among the six WHO building blocks and how are they measured? Commodities and Procurement: How have PEPFAR technical assistance and training affected HIV laboratory diagnostic capacity, pharmacy infrastructure, and supply chain management for reagents and drugs? Information Systems: What is the evidence that PEPFAR-supported health information systems are resulting in higher functioning, quality-driven health systems performance? How will new and existing information officers be trained to meet country needs to strengthen higher functioning and increased quality? Service Delivery: What elements of more efficient, equitable, and effective service delivery should be expected to result in improved population health over time with better integration of HIV- and non-HIV health care and use of continuous quality improvement methods? Leadership and Governance: To what extent will partnership frameworks, jointly funded Global Fund activities, and other expressions of country ownership lead to improved and accountable governance as well as transfer of oversight, management, guidance, and financing for HIV-related services in health systems? Health Workforce: How will the pre-service education target of 140,000 new workers affect health system equity of access (e.g., rural versus urban) and health system strengthening such as increased skill capacity and retention of workforce? Are there other types of workers, (e.g., social service workers and program managers) who also need to be trained? What proportion of newly trained new health or social workers is 49 (B)(ii) an assessment of the effects on health systems, including on the financing and management of health systems and the quality of service delivery and staffing 

130 STRATEGIC APPROACH TO THE EVALUATION OF U.S. GLOBAL HIV/AIDS PROGRAMS retained one year later? How does PEPFAR support the country’s existing health resources development plan or help develop those where absent? The committee will also assess progress toward the goals of increased country ownership to transition the HIV/AIDS response to national governments for long-term sustainability. Although there are high-volume discussions within the global development assistance community in this area, there appear to be few to no meaningful metrics to measure increased ownership or country readiness, no accepted logic models to describe and illustrate the transition process, and no indicators for when a country has sufficiently strengthened governance/ leadership or built and operated a financially-sound public finance system to implement and oversee a national health plan. Illustrative questions for this area of the evaluation include the following. Country ownership and sustainability cut across all PEPFAR activities and as such these questions are intended to contribute to addressing all of the areas for consideration in the congressional mandate, as described in the Statement of Task (see Appendix A). How is country ownership defined by the country? By PEPFAR? By relevant global stakeholders in aid development? Are there differences? Have they been reconciled? Do the Partnership Frameworks represent an “agreement” on the definition between the country and PEPFAR? To what extent have PEPFAR capacity-building, technical assistance, and financing activities to countries contributed to country readiness for transitioning the knowledge management, decision making responsibility, financing, and accountability/oversight of the PEPFAR-funded HIV/AIDS response to the national government? How is it being measured? Are plans in place for future activities needed to improve or increase country readiness if PEPFAR were absent? How will achievements or milestones by country and/or PEPFAR be measured in the Partnership Frameworks? Will these measures adequately reflect country readiness for program, policy, and financial transitions? Is the country transitional process explained adequately? Will it be standard for all countries or adapted for each country? Will the transition be parceled over time? Which pieces might be transitioned over time, how, and why? What are the achievements and lessons learned from the Technical Support Grants to the Global Fund from PEPFAR? How are they measuring whether bottlenecks have been opened or bypassed? What has been the impact on the functioning of programs? How do Global Fund and PEPFAR measure their relative and unique contributions to programs they jointly fund or operate? How are access, equity, and quality measured for these programs? A more extensive listing of illustrative questions for this complex issue that the committee may attempt to address, given the timely availability of quality data, can be found in Appendix H. These may help guide assessment of whether and how PEPFAR has helped 

131 KEY SYSTEMS-LEVEL GOALS AND ACTIVITIES countries to mount a stronger response to their HIV epidemics, plan for transition to country ownership including effective leadership and oversight of multilateral donor activities, and focus on affordable care that meets population health needs. Sustainability and its associated elements must be considered for all of PEPFAR’s support. PEPFAR activities will be evaluated relative to their contribution of inputs and support of processes, and the resulting outputs and outcomes. These should ultimately result in the desired outcome of stronger health systems that can more adequately respond not only to HIV but to other serious causes of morbidity and mortality, as well as any emerging challenges to the health and safety of their respective populations.