Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 149
4 Measurement and Evaluation The preceding chapters have outlined the scope and context of the global epidemic of cardiovascular disease (CVD), the extent of the resulting health and economic burden, and the challenge that lies ahead. This provides a compelling rationale for aggressively reducing risk factors that lead to CVD globally. Measurement is the basis for determining the scale of the global CVD epidemic and for understanding how best to intervene, and it will be critical to the success of efforts to reduce disease burden. While there is a need for CVD-specific measurement tools, existing global health efforts provide a robust foundation to draw upon and to avoid duplication as the global CVD community continues to develop and expand its evaluation of program and policy initiatives. Over the past several decades, advances in the field of global health have led to a wealth of measurement knowledge, tools, and techniques that have been developed for evaluating policy and program outcomes and impact on health status at all levels. Indeed, many of these national and global measurement initiatives are currently at risk of overlooking measurement of CVD and related chronic diseases, which will in fact be crucial in order to obtain a truly complete picture of national health needs. This chapter1 first describes the functions and principles of measurement, monitoring, and evaluation. The chapter then addresses several critical cross-cutting considerations that affect measurement and evaluation. This is followed by a discussion of the potential for measurement ap- 1 This chapter is based in part on a paper written for the committee by Jeff Luck and Riti Shimkhada.
OCR for page 150
proaches that can provide timely feedback and guide decision making at multiple levels to achieve reductions in cardiovascular disease, including a discussion of emerging technologies to improve measurement. Finally the chapter touches on the use of measurement at the global level to inform actions to reduce the burden of CVD. FUNCTIONS AND PRINCIPLES OF MEASUREMENT Measurement serves a number of critical roles in the effort to address any health problem. The use of measurement to inform the cycle of decision making in addressing a public health problem is outlined in Figure 4.1. This cycle applies to decision making at any level of stakeholder, from global to local, and at any scale of intervention, from a demonstration project to a global action plan. First, it is used to assess the magnitude of the problem at the level of the population and subpopulation and informs the mitigation of risk factors. When coupled with an assessment of capacity, these can inform priorities and the setting of realistic intervention goals. This in turn guides implementation of interventions, including policies, programs, and clinical interventions at the level of the population, the provider, and the individual. Measurement then can be used to assess the processes, outcomes, and impact of the implemented interventions. This feeds back into the cycle to encourage adaptations that help ensure sustainable progress. Thus, measurement is not simply an endpoint to determine the value of an intervention; it is also the foundation for an ongoing cycle of planning, prioritizing, and operationalizing interventions. Ultimately, measurement strategies have the potential to lead to changes in health outcomes by changing the decisions and behavior of policy makers, providers, and individuals. This derives from the fundamental purpose of measurement: to create awareness that increases understanding and motivates change. In this way, as illustrated in Figure 4.2, measurement can be viewed as a critical component of any effort to result in an impact on health outcomes, serving to guide those efforts and to accelerate the pace of change to achieve the targeted outcomes. To serve as an instrument of change, measurement needs to be ongoing and cyclical. Transparent information can increase knowledge and change intentions throughout the process of implementing an intervention approach, just as it can lead to overall changes in baseline status and new policies or programs in response to achieving a new baseline. A number of underlying principles drive measurement as a fundamental part of efforts to decrease CVD. First, in order to be effective measurement needs to be relevant to the context in which it is implemented (Majumdar and Soumerai, 2009). Contextual elements are typically local—occurring at the level of countries, regions, cities, and villages. Context includes local
OCR for page 151
FIGURE 4.1 Measurement-based decision-making cycle.
OCR for page 152
FIGURE 4.2 Role of measurement in achieving health impact.
OCR for page 153
elements of economics, financing, existing policies, existing capacity, population demographics, and social and cultural factors. These in turn exist in a larger global context. A second principle is that measurement is most effective when it is transparent and when there are feedback mechanisms to ensure that the resulting data is widely available and widely used. Indeed, measurement alone is not sufficient—the data must actually be used by policy makers, providers, and individuals. Third, to truly document and maximize impact, measurement is needed at all levels, from individuals to providers to policy makers. Measurement is also needed across all kinds of interventions approaches, from clinical interventions and individual risk reduction to changes in the infrastructure to deliver interventions to policy changes and other population-based strategies. A fourth principle is that measurement needs to focus on the intermediate outcome of behavior change, for it is changes in the behavior of those at risk, of care providers, and of policy makers that will lead to lessening of the CVD burden. In order for measurement to be effective it must also be accurate, feasible, affordable, actionable, responsive, and timely (Majumdar and Soumerai, 2009). Finally, measurement outcomes should be able to be communicated clearly. Although there may be necessary complexity in the design of measurement systems, this complexity should be converted into relatively simple reporting of the data. The number and variety of determinants that contribute to cardiovascular disease means that no single set of measures or data collection system will suffice for all goals or settings. Instead, this complexity necessitates the use of an array of measures and a variety of collection strategies, along with careful planning to set priorities for measurement and to balance feasibility with the need for comprehensive data that can be integrated and compared across countries, programs, and levels of measurement. As a final principle, it is critical in the planning and implementation of measurement strategies to make the following determinations: who is expected to use the data; what is to be measured; what metrics or indicators should be used; who will be collecting the data; what tools will be used to collect the data; who will analyze the data; how the data will be reported and disseminated; and how much it will cost to implement the measurement strategy and to analyze and disseminate the data.
OCR for page 154
CROSS-CUTTING CONSIDERATIONS IN MEASUREMENT There are several critical cross-cutting considerations that affect measurement and evaluation that are important to discuss as the basis for interpreting the potential use of the methodologies described later in this chapter. These include standardization of indicators, data ownership and capacity for data analysis, and costs of measurement. Standardizing Indicators for CVD Surveillance, Intervention Research, and Program Evaluation To monitor the epidemic of CVD and ensure that there are effective intervention approaches that can be disseminated widely, it is critical to be able to gather data and make comparisons across countries, across sectors and systems, and across intervention and program evaluations. Therefore, while measurement efforts need to be tailored to the context, program, or intervention approach, some measurement strategies would benefit from standardization and global coordination of surveillance systems and evaluation systems. The question of which indicators to use and how to prioritize them must be agreed upon by the relevant stakeholders in the international community. A number of key categories of metrics are crucial to measuring CVD and its breadth of determinants and would need to be considered. These include demographics; risk and risk mitigation including behaviors (e.g., smoking rates, physical activity, diet and nutrition) and biomedical measures (e.g., weight and height, blood pressure, cholesterol); disease outcomes (e.g., cardiovascular events); cause-specific mortality; health provider and quality improvement measures; health systems performance; economic measures; intersectoral policy measures (e.g., cigarette costs and sales data, agricultural trends, urbanization); and measures of global action. Some of these measures need to be disease specific, while others need to be harmonized and coordinated with measurement strategies for related chronic diseases and for other areas of health and development. While there may already be consensus within a few of these indicator categories, far more are currently still being debated, and setting priorities within and across categories to balance comprehensive measurement with feasibility will not be simple. Although it was beyond the scope of this committee to do so, a minimum set of indicators with clear definitions with guidance on prioritization needs to be developed to allow for uniform and comparable data across countries and systems. Developing an indicator framework of this kind could be achieved through a consensus process involving key stakeholders such as researchers, practitioners, economists, funders, and representatives from national health and public health authorities from developing countries. This process would need to realistically con-
OCR for page 155
sider how to balance the need for comprehensive data collection with the practicalities of timeliness and resources. In addition, a critical component for any indicator framework is what the implementation and maintenance of each measurement system would cost. The World Health Organization (WHO) has convened an epidemiology reference group, drawing on headquarters and regional offices, to develop guidance for chronic disease surveillance systems and to agree on core indicators that will be used to monitor the major chronic diseases and their risk factors (Alwan, 2009, personal communication). If this effort takes into account the considerations described here, it could be a first step in achieving an implementable indicator framework. This need for standardization and coordination has been recognized by the global HIV/AIDS community and is addressed in large part by the United Nations’ Joint Programme on HIV/AIDS (UNAIDS’s) Monitoring and Evaluation Reference Group (MERG) (UNAIDS, 2009a). Created in 1998 by the UNAIDS Secretariat, the MERG provides technical guidance for HIV monitoring and evaluation and is a key driver in the harmonization of HIV indicators at the global level (Global HIV M&E Information, no date). Working through a coordinated effort with individuals at the Global Fund and the U.S. President’s Emergency Plan for AIDS Relief, the MERG identified, collected, and defined high-quality indicators, making them freely accessible online (UNAIDS, 2009a, 2009b). In addition, while the indicator registry identifies which measures have been harmonized and endorsed by other stakeholders, it leaves the decision on determining the indicators that are most important to collect to the implementer, be it a national government or program manager (UNAIDS, 2009b). This use of online resources to lower the cost of use for developing countries as well as the leadership and coordination from a body with the capacity to also provide relevant technical support could provide a useful model for WHO during indicator standardization efforts for chronic diseases. Once developed, coordinated support will be needed for the implementation of these globally comparable indicators. Technical assistance and training in surveillance, research, and evaluation will be needed to provide options for measurement tools that incorporate the uniform data from globally comparable indicators, but also to allow for national or local/ program-level choices on which tools to use and which indicators to collect (beyond the minimum set) based on local and project- or program-specific priorities, resources, and needs. Data Ownership and Capacity for Data Analysis The collection and reporting of data, regardless of how detailed, accurate, or comprehensive, is a potential waste of time and resources unless the information is appropriately processed, analyzed, and communicated
OCR for page 156
to relevant stakeholders. Currently there is a growing need to develop and maintain data analysis capacity at the local level, in an effort to help communities feel ownership of reported outcomes (Stansfield, 2009). Limits in local capacity to conduct both analysis and operations research have left some national governments hesitant to take on new measurement initiatives as they could overwhelm already fragile health information systems (Bennett et al., 2006). Thus, these absorptive capacity concerns must be kept in mind when determining how rapidly and to what degree to scale up measurement and evaluation initiatives. Addressing these capacity needs will require a paradigm shift at the international, national, and local levels about the importance of developing locally relevant measurement solutions. Targeted funding from donors may be required not only for the development of sustainable health information systems but also to assist organizations with training of local individuals in data collection and analysis where there are shortages in this expertise, as well as with the retention of trained individuals. In order to be effective, these efforts need to be coupled with an assessment of the existing monitoring and evaluation capacity of local actors. Tools that could be used to improve measurement capacity include workshops and training sessions to instruct health authorities or program coordinators on how to set up and maintain data collection systems, implement core indicators, design evaluations, adapt preferred guidance documents to their unique situation, and analyze data. Centers of excellence in this area that are established within a developing country need not be disease specific and must have the potential to build capacity at both the national or regional level that would benefit multiple health sectors. Expanding local analytic capacity could also potentially help to reduce the prevalence of unused data “piles” that amass in developing countries. The failure of both donors and national governments to invest in sustainable health information systems inhibits countries’ abilities to routinely process these data (Stansfield, 2009). It is important to note that building the capacity to collect and analyze data is not sufficient. There is also a need to strengthen the motivation and capacity for policy makers to interpret and act on the data. To achieve this, data collection strategies could be developed in consultation with policy makers and include mechanisms for timely reporting to inform policies and programs. In addition, the proliferation of multilateral organizations, international and local nongovernmental organizations, and the expanding private sector all place their own, often redundant measurement and evaluation demands on local actors, which adds an additional burden to local and national measurement efforts and contributes to the accumulation of unused data. Following the completion of their individual evaluation processes, the information collected is typically analyzed and disseminated within
OCR for page 157
the organization itself, completely extracting it from the communities to which it refers. This practice has had two notable negative consequences: first, it limits the amount of community involvement in the measurement and evaluation process, missing an important opportunity to develop local analytic skills, and second, it propagates a culture of non-evidence-based policy making by failing to connect policy or program interventions with impact assessment results (Stansfield, 2009). Costs of Measurement The cost of measurement can pose an important limitation on feasibility. Along with capacity for data collection and analysis, costs must be taken into consideration when prioritizing, planning, and implementing any of the specific measurement approaches that will be described later in this chapter. Methods to collect population data, such as systematic surveillance and health information systems, can be very expensive and have required subsidization from external funders in many countries. Although there is limited publicly available information and analysis of the costs to implement population measurement strategies, some estimates for country spending on health data suggest that comprehensive measurement can be affordable for developing countries. For example, the Health Metrics Network (HMN) estimates a national health information system comprising six essential subsystems (health service statistics, public health surveillance, census, household surveys, vital events, and health resource tracking) would cost $0.53 per capita in a low income country (Stansfield et al., 2006). The health information system in Belize was implemented at an initial cost of approximately $2 per capita (Bundale, 2009). The Millennium Development Goals Africa Steering Group (2008) estimates that to support censuses, household surveys, and civil registration and vital statistics systems across Africa would cost $250 million annually (less than $1 per capita). In Tanzania, 11 information systems that generate health and poverty indicators were able to generate all but one of the indicators recommended by four major poverty reduction and reform programs, at an aggregate cost of $0.53 per capita in 2002/2003 (Rommelmann et al., 2005). Program evaluation also requires an investment of a proportion of the project budget, but there is little publicly available information on the amount spent on measuring, monitoring, and evaluating health programs, and there is limited evidence to assess the costs, cost-effectiveness, benefit-cost ratio, or financial return on investment for different measurement strategies to evaluate these programs. Indeed, although measurement activities usually receive some funding as part of the implementation of a program, no empirical basis supports specific budget targets for measurement or monitoring and evaluation. The most explicit guidance regarding
OCR for page 158
the proportion of program activities that should be devoted to monitoring and evaluation comes from the Global Fund. The Global Fund’s 2009 Monitoring and Evaluation Toolkit: HIV, Tuberculosis, and Health Systems Straightening, which was co-sponsored by a number of major multilateral global health organizations, states that over the past years “global and national efforts have been made to increase financial resources for monitoring and evaluation to the widely recommended 5–10 percent of the overall program budget.” It endorses this amount and offers a framework on how to allocate these funds (The Global Fund to Fight AIDS, Tuberculosis and Malaria, 2009, p. 32). APPLYING MEASUREMENT METHODS FOR GLOBAL CVD The following sections describe methods and tools that can serve to improve measurement for global CVD by providing information for feedback and decision making from multiple sources (such as surveillance, intervention research and program evaluation, clinical practice data, and policy analysis) and at multiple levels (including national, subnational, health systems, communities, households, and individuals). Although a distinction in levels and sources is made in the discussion that follows, it is also ideal for measurement approaches to work across different levels—for example, by using nested measures with relevance to each other. For comparable use of data across sources and levels, there also needs to be agreement on what is to be measured and how it is disseminated. To address global CVD, the methods described here draw from successful CVD measurement strategies and programs from the developed and the developing world where available, as well as from significant advances in measurement in other areas of global health, especially HIV/AIDS. Measurement to Inform Policy For policy makers at all levels, measurement provides information that can motivate changes in priorities and policies, influence public opinion, help select and manage intervention approaches, and set priorities for the allocation of resources. The discipline of policy analysis strives to provide objective data and analyses to support rational policy decisions (i.e., “evidence-based policy”). There is an important distinction between evidence for policy and evidence on policy. Evidence for policy supports a rationale for prioritizing and implementing policies and programs and often comes from population-level evidence as well as from system- and program-level evidence. A well-supported rationale, however, involves uncertainty as to the actual benefit or relevance, especially when being translated from other contexts.
OCR for page 159
Evidence on policy attempts to address that uncertainty by actively assessing the impact of public health and prevention policies using measurement of population endpoints, such as smoking prevalence or clinically recognized myocardial infarctions (MIs). Such research is often described as health policy and systems research in the global health literature. For example, a policy change that is phased in allows experimental data to be gathered comparing population outcomes with and without the implemented policy; this can be especially valuable in informing future policies. Policy makers also often make implementation decisions based on evaluations that assess the effectiveness and implementation of particular clinical, organizational, or public health strategies. This evaluation approach is described in more detail later in this chapter. A recent review of the literature indicates that health policy analysis in developing countries is quite limited, especially with regard to CVD (Gilson and Raphaely, 2008). Although spending on health policy and systems research is growing, it remains low and results remain limited in rigor and generalizability compared to the needs of policy makers and providers in developing countries (Anonymous, 2008; Bennett et al., 2008). The literature on implementation science—which addresses how interventions demonstrated to be effective can be implemented in a wider range of settings—is also limited for developing countries (Madon et al., 2007). However, there is an emerging movement to use more evidence-based policy at all levels in low and middle income countries, and it is crucial to be sure that this movement does not continue to develop without being applied to policies related to chronic diseases. The strength and mix of national, regional, and local policy measurement will depend on country-specific factors, such as the governance system, the size of the relevant population, and other local attributes. Working to fill the evidence-based policy gap in low and middle income countries, the Evidence-Informed Policy Network (EVIPNet) aims to synthesize research results into products useful to developing-country health policy makers. EVIPNet teams have now been established in Africa (van Kammen et al., 2006), Asia, and the Americas (Corkum et al., 2008). However, these efforts remain limited in scope and applicability for cardiovascular disease as none of the policy briefs currently being developed by the nine-country coalition of EVIPNet Africa relate to policy decisions on CVD risk factors (EVIPNet, 2008). The Future Health Systems: Innovations for Equity consortium is another example of an active approach to making a research–policy linkage, by working with six developing countries to develop 5-year research plans whose results will address priorities identified by policy makers (Syed et al., 2008). While significant progress can be made by engaging national governments around measurement, the use of evidence-based policy should not be
OCR for page 174
Evaluation provides an online basic measurement and evaluation Fundamentals Self-Guided Mini Course, originally developed for USAID, which includes discussions on how to identify indicators, plan and conduct intervention evaluations, and analyze the results (Frankel and Gage, 2007). The Global Fund to Fight HIV/AIDS, Tuberculosis and Malaria has also developed a Monitoring and Evaluation Toolkit that addresses not only HIV and TB but also efforts to strengthen health systems (The Global Fund to Fight AIDS, Tuberculosis and Malaria, 2009). Given the potential role of health systems strengthening programs in addressing the global burden of CVD, adaptation of this toolkit could provide an opportunity to harmonize relevant chronic and infectious disease health systems indicators. The Global Fund Guide for Operational Research offers the addition of process measures for long-term adaptation and sustainability of ongoing programs. This kind of operational research is particularly critical for programs to address CVD, which requires ongoing intervention. Indeed, new research may also be needed to develop program evaluation strategies that can address the long-term needs of measurement follow-up and impact evaluation, as the reality is that many of the benefits of CVD risk-factor interventions will not accrue until years or decades after individual programs have been completed. Impact measures for programs to prevent and manage CVD would follow principles similar to those for intervention research, including both CVD-related outcomes and economic measures, as well as process measures to monitor program implementation. Although measurement strategies need to be tailored to specific interventions or programs, some standardization would provide an opportunity for a better continuum from intervention trials through implementations of interventions at scale, with a set of streamlined indicators that would be useful to assess whether the original effectiveness is being maintained. The incorporation of some agreed-upon standardized metrics would also allow for comparisons across programs and over time and greater long-term feasibility of program evaluation. As intervention programs increase in scale, so do their data collection and reporting needs, and their risk of developing duplicate systems that operate alongside national health information systems. A review of how reporting mechanisms for major global HIV/AIDS programs interact with national data collection efforts showed that this can lead to inefficient use of resources on parallel reporting structures, a failure to develop one coherent national picture of impact, and an increased burden on program implementers (Oomman et al., 2008). Avoiding this duplication by identifying CVD indicators that can meet the needs of both the program and the health information system as well as encouraging the integration of reporting with the national systems where appropriate will be important considerations as global CVD programs expand in developing countries.
OCR for page 175
Measurement of Individual Health Status Measurement of individual behavioral or biological risk factors can be useful to motivate changes in a person’s behavior if the data are meaningful enough for the individual to be able to act upon the results. This requires that the data be presented in clear terms alongside health counseling or education initiatives to establish clinically relevant behavior-change goals for individuals. Several decades of behavioral research in developed-country settings indicate that an individual’s knowledge of his or her health status and/or risk for disease is a necessary (albeit insufficient) precursor to behavior change. This theoretical principle is supported by research of several cardiovascular risk behaviors. For example, regular self-weighing has been associated in several studies with weight loss and weight maintenance (Butryn et al., 2007; Linde et al., 2005; VanWormer et al., 2009). Also, a recent review found that the use of pedometers to track the number of steps a person takes, particularly if a goal for steps was set, was consistently associated with increased physical activity (Bravata et al., 2007). In addition, limited data indicate that simply the knowledge of cholesterol levels can influence fat intake (Aubin et al., 1998). In addition to providing feedback to individuals and providers to motivate and guide individual behavior change, individual measures can also be aggregated to improve provider performance, to inform measurement of health systems, or to reflect populations at a broader level when it is statistically appropriate to do so and appropriate methods are used. However, an important consideration in individual-level measurement is whether standards and norms are replicable in different populations. This is true for single measures, such as body mass index, and especially for methods used to score aggregate risk. Emerging Technologies to Support Measurement Methods The emergence of electronic health (e-health) and mobile health (m-health) initiatives in both developed and developing countries have opened the door to an enormous new set of potential efforts to help make both health care delivery and measurement more effective and efficient. These technologies cut across all levels of measurement and interact with each to varying degrees. While there is a need for much more research on the training, infrastructure, and cost barriers to introducing new technology and mobile data collection devices, they present a rapidly growing field of research and investment on which global health initiatives have already begun to capitalize (United Nations Foundation, 2010). Thus, it is in the
OCR for page 176
interests of the global CVD community to actively pursue involvement in ongoing efforts to improve these nascent systems. A recent review of e-health initiatives in developing countries showed that technology is already being used in resource-poor settings with some success for a wide variety of projects, ranging from electronic health records to laboratory and pharmacy management systems to data collection and evaluation tools (Blaya et al., 2010). E-health and m-health technology are also emerging to support measurement through new tools to conduct population-based surveys and surveillance, to link data to geographic information, and to present that data to policy makers in more coherent and compelling manners (Gapminder Foundation, 2010; IDRC, 2009; Tegang et al., 2009). In particular, the potential application of new tools to track patient status over the long term and to integrate information with health systems is uniquely suited to chronic disease management. For example, the use of electronic medical records systems in health care settings is one potential mechanism for improved data collection and analysis. These systems are already in use in a number of developing countries for monitoring patients on antiretroviral therapy (Braitstein et al., 2009; Forster et al., 2008; Kalogriopoulos et al., 2009), and a variety of both proprietary and open source software tools are available. However, it is important that these be adapted to local needs in order to prevent inefficiencies caused by a failure to ensure the collection of all necessary data or by the use of multiple systems to cover duplicate reporting needs (Forster et al., 2008; Kalogriopoulos et al., 2009; OpenMRS, 2010). Some organizations, such as AMPATH in Kenya, have already begun to adapt their antiretroviral therapy focused electronic medical records systems to include measures for diabetes and cardiovascular disease (Braitstein et al., 2009). In addition to assisting with the management of patient-level data, electronic medical records systems, if designed appropriately, also have the potential to incorporate measures that can be aggregated to inform health systems priorities. The use of mobile health approaches to improve patient outcomes is discussed further in Chapter 5. GLOBAL USES OF MEASUREMENT The use of measurement data compiled and analyzed at the global level is crucial to the success of current and future initiatives as it can serve to raise awareness and to prioritize and coordinate efforts among global stakeholders. As described in Chapters 1 and 2, analyses of the global burden of CVD have been critical in illuminating the scope and magnitude of the CVD epidemic and advancing the advocacy message of the CVD community. Burden-of-disease analyses are an important method of data aggregation and modeling, which can lay out mortality and morbidity estimates,
OCR for page 177
showing changes in the epidemic across countries and linking this information to economic data (Abegunde et al., 2007; Lopez et al., 2006). In addition to these aggregated analyses, individual country efforts need to be tracked in a coordinated manner in order to inform global efforts, learn from emerging best practices, prevent duplication, and identify where additional resources and focus should be directed. A number of broad measures of progress would benefit from leadership at the global level, including the evaluation and dissemination of the impact and implementation of global efforts, behavioral and biomedical surveillance and its integration into national surveillance systems at the population level, infrastructure, training, health education, and tracking and evaluating the effectiveness of funding and expenditures. In addition, all new policies by major global players should be backed by a financial assessment of the implementation cost and should describe means by which pledges and commitments will be reported. A variety of stakeholders are currently responsible for either coordination or measurement at the international level. First and foremost an extensive list of measures was proposed in the 2008 WHO Noncommunicable Disease Action Plan to track global progress and characterize the different actions underway in member states. WHO is scheduled to release a preliminary progress report on a select number of these metrics (WHO, 2008). In addition, globally coordinated research efforts, such as the newly created Global Alliance for Chronic Disease (Daar et al., 2009), will need to establish indicators for tracking the distribution of funds, demonstrating the impact of their efforts, and identifying successful coordination strategies. Given the overlapping interest of many of these multilateral organizations, the development of harmonized indicators is an essential next step, as described previously in this chapter. An epidemiology reference group has also been working with WHO staff from headquarters and regional offices to develop guidance for chronic disease surveillance systems and to agree on core indicators that will be used to monitor the major chronic diseases and their risk factors (Ala Alwan, World Health Organization, 2009, personal communication). Finally, the creation of a routine global reporting mechanism that convenes to compare and disseminate results is also needed. Mechanisms for developing this are discussed further in Chapter 8. CONCLUSION Measurement is crucial to the success of efforts at every stage of the process to avert the rise of CVD in developing countries. Stakeholders of all kinds, from national governments to development agencies and other donors, who have committed to taking action to address the burden of chronic diseases will need to carefully assess the needs of the population
OCR for page 178
they are targeting, the state of current efforts, the available capacity and infrastructure, and the political will to support the available opportunities for action. This assessment will inform priorities and should lead to specific and realistic goals for intervention strategies that are adapted to local baseline capacity and burden of disease and designed to improve that baseline over time. These goals will determine choices about the implementation of both evidence-based policies and programs and also capacity-building efforts. Ongoing evaluation of implemented strategies will allow policy makers and other stakeholders to determine if implemented actions are having the intended effect and meeting the defined goals, and to reassess needs, capacity, and priorities over time. Over the past two decades great progress has been made toward identifying risk-factor prevalence and CVD incidence, prevalence, severity, and mortality, as described in Chapter 3. At the global level, this has fulfilled the first step in the cycle of measurement for CVD. However, many low and middle income countries still lack sufficient local data to inform their decisions about how to prioritize actions to target CVD. In addition, while basic epidemiologic knowledge has been expanding, other core functions of measurement, such as policy analysis, health services research, intervention research, and program impact evaluation, have not been keeping pace. As a result, although there exists greater awareness about which risk factors require the most attention, less is known about what intervention approaches will be most effective and feasible in the resource-constrained settings of low and middle income countries. This lack of knowledge about program and policy effectiveness within local realities not only constrains program implementers, but also prevents national governments, nongovernmental organizations, and multilateral organizations from effectively making and implementing decisions to address the cardiovascular disease epidemic. For some CVD measurement needs, there are well-established models for evaluation and data collection in developing countries, such as models for national surveillance, behavioral surveys, electronic medical records, and tools for program evaluation. For other purposes, new tools need to be developed. In either case, it is important, when feasible, to build upon current approaches used in monitoring and evaluation both locally and globally in order to take advantage of existing infrastructure, to build capacity in measurement and monitoring, and to avoid the inefficiencies of duplicate systems. Finally, for comparable use of data across programs and countries, there also needs to be international agreement on what is to be measured and how the information is disseminated.
OCR for page 179
REFERENCES Abegunde, D. O., C. D. Mathers, T. Adam, M. Ortegon, and K. Strong. 2007. The burden and costs of chronic diseases in low-income and middle-income countries. Lancet 370(9603):1929-1938. AbouZahr, C., J. Cleland, F. Coullare, S. B. Macfarlane, F. C. Notzon, P. Setel, S. Szreter, R. N. Anderson, A. A. Bawah, A. P. Betran, F. Binka, K. Bundhamcharoen, R. Castro, T. Evans, X. C. Figueroa, C. K. George, L. Gollogly, R. Gonzalez, D. R. Grzebien, K. Hill, Z. Huang, T. H. Hull, M. Inoue, R. Jakob, P. Jha, Y. Jiang, R. Laurenti, X. Li, D. Lievesley, A. D. Lopez, D. M. Fat, M. Merialdi, L. Mikkelsen, J. K. Nien, C. Rao, K. Rao, O. Sankoh, K. Shibuya, N. Soleman, S. Stout, V. Tangcharoensathien, P. J. van der Maas, F. Wu, G. Yang, S. Zhang. 2007. The way forward. Lancet 370(9601):1791-1799. Anonymous. 2008. The state of health research worldwide. Lancet 372(9649):1519. Arah, O. A., N. S. Klazinga, D. M. J. Delnoij, A. H. A. Ten Asbroek, and T. Custers. 2003. Conceptual frameworks for health systems performance: A quest for effectiveness, quality, and improvement. International Journal for Quality in Health Care 15(5):377-398. Asma, S. 2009. Global tobacco surveillance system. Presentation at the Public Information Gathering Session for the Institute of Medicine Committee on Preventing the Global Epidemic of Cardiovascular Disease, Washington, DC. Aubin, M., G. Godin, L. Vezina, J. Maziade, and R. Desharnais. 1998. Hypercholesterolemia screening. Does knowledge of blood cholesterol level affect dietary fat intake? Canadian Family Physician 44:1289-1297. Baiden, F., A. Hodgson, and F. N. Binka. 2006. Demographic surveillance sites and emerging challenges in international health. Bulletin of the World Health Organization 84(3): 163. Bennett, S., J. T. Boerma, and R. Brugha. 2006. Scaling up HIV/AIDS evaluation. Lancet 367(9504):79-82. Bennett, S., T. Adam, C. Zarowsky, V. Tangcharoensathien, K. Ranson, T. Evans, and A. Mills. 2008. From Mexico to Mali: Progress in health policy and systems research. Lancet 372(9649):1571-1578. Blaya, J., H. S. Fraser, and S. Holt. 2010. E-health technologies show promise in developing countries. Health Affairs 29(2):245-251. Bonow, R. O., F. A. Masoudi, J. S. Rumsfeld, E. Delong, N. A. Estes, 3rd, D. C. Goff, Jr., K. Grady, L. A. Green, A. R. Loth, E. D. Peterson, I. L. Pina, M. J. Radford, and D. M. Shahian. 2008. ACC/AHA classification of care metrics: Performance measures and quality metrics: A report of the American College of Cardiology/American Heart Association Task Force on Performance Measures. Journal of the American College of Cardiology 52(24):2113-2117. Boutayeb, A. 2006. The double burden of communicable and non-communicable diseases in developing countries. Transactions of the Royal Society of Tropical Medicine and Hygiene 100(3):191-199. Braitstein, P., R. M. Einterz, J. E. Sidle, S. Kimaiyo, W. Tierney, P. Braitstein, R. M. Einterz, J. E. Sidle, S. Kimaiyo, and W. Tierney. 2009. “Talkin’ about a revolution”: How electronic health records can facilitate the scale-up of HIV care and treatment and catalyze primary care in resource-constrained settings. Journal of Acquired Immune Deficiency Syndromes: JAIDS 52(Suppl 1):S54-S57. Bundale, B. 2009. Company’s software benefits Belize. New Brunswick Business Journal, 8 June. Butryn, M. L., S. Phelan, J. O. Hill, and R. R. Wing. 2007. Consistent self-monitoring of weight: A key component of successful weight loss maintenance. Obesity (Silver Spring) 15(12):3091-3096.
OCR for page 180
Centers for Disease Control and Prevention. 2010. CDC’s behavioral risk factor surveillance system. http://www.cdc.gov/brfss/ (accessed October 8, 2009). Corkum, S., L. G. Cuervo, and A. Porras. 2008. EVIPNet Americas: Informing policies with evidence. Lancet 372(9644):1130-1131. Daar, A. S., E. G. Nabel, S. K. Pramming, W. Anderson, A. Beaudet, D. Liu, V. M. Katoch, L. K. Borysiewicz, R. I. Glass, J. Bell. 2009. The global alliance for chronic diseases. Science 324(5935):1642. Diaz, T., J. M. Garcia-Calleja, P. D. Ghys, and K. Sabin. 2009. Advances and future directions in HIV surveillance in low- and middle-income countries. Current Opinion in HIV AIDS 4(4):253-259. Engelgau, M. M. 2009. Measuring success: Using tools we already have. Presentation at the Public Information Gathering Session of the Committee on Preventing the Global Epidemic of Cardiovascular Disease, Washington, DC. Epstein, A. J. 2006. Do cardiac surgery report cards reduce mortality? Assessing the evidence. Medical Care Research and Review 63(4):403-426. EUROASPIRE Study Group. 1997. EUROASPIRE: A European Society of Cardiology survey of secondary prevention of coronary heart disease: Principal results. European Heart Journal 18(10):1569-1582. EVIPNet. 2008. Evidence informed policy network: For better decision making. Geneva: World Health Organization. Ezzati, M., S. Vander Hoorn, C. M. Lawes, R. Leach, W. P. James, A. D. Lopez, A. Rodgers, and C. J. Murray. 2005. Rethinking the “Diseases of affluence” paradigm: Global patterns of nutritional risks in relation to economic development. PLoS Med 2(5):e133. Family Health International. 2010. An inventory of program evaluation tools and guidelines. Arlington, VA: Family Health International. Flay, B. R., A. Biglan, R. F. Boruch, F. G. Castro, D. Gottfredson, S. Kellam, E. K. Moscicki, S. Schinke, J. C. Valentine, and P. Ji. 2005. Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science 6(3):151-175. Forster, M., C. Bailey, M. W. Brinkhof, C. Graber, A. Boulle, M. Spohr, E. Balestre, M. May, O. Keiser, A. Jahn, and M. Egger. 2008. Electronic medical record systems, data quality and loss to follow-up: Survey of antiretroviral therapy programmes in resource-limited settings. Bulletin of the World Health Organization 86(12):939-947. Frankel, N., and A. Gage. 2007. M&E fundamentals: A self-guided mini course. Chapel Hill, NC: MEASURE Evaluation. Gapminder Foundation. 2010. For a fact-based world view. http://www.gapminder.org/ (accessed March 9, 2010). Gaziano, T. A., G. Galea, and K. S. Reddy. 2007. Scaling up interventions for chronic disease prevention: The evidence. Lancet 370(9603):1939-1946. Gilson, L., and N. Raphaely. 2008. The terrain of health policy analysis in low and middle income countries: A review of published literature 1994-2007. Health Policy Plan 23(5):294-307. The Global Fund to Fight AIDS, Tuberculosis and Malaria. 2009. Monitoring and evaluation toolkit: HIV, tuberculosis, and malaria and health systems strengthening. Geneva: The Global Fund. Global HIV M&E Information. MERG background. http://www.globalhivmeinfo.org/AgencySites/Pages/MERG%20Background.aspx (accessed February 2, 2010). IDRC (International Development Research Centre). 2008. Fixing health systems, 2nd edition—executive summary. Ottawa, Canada: International Development Research Centre. IDRC. 2009. Assessing the use of PDAs for household surveys in Tanzania. http://www.idrc.ca/en/ev-88026-201-1-DO_TOPIC.html (accessed March 9, 2010).
OCR for page 181
IOM (Institute of Medicine). 2001. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press. IOM. 2009a. State of the USA health indicators: Letter report. Washington, DC: The National Academies Press. IOM. 2009b. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: The National Academies Press. Joshi, R., S. Jan, Y. Wu, and S. MacMahon. 2008. Global inequalities in access to cardiovascular health care: Our greatest challenge. Journal of the American College of Cardiology 52(23):1817-1825. Joshi, R., A. P. Kengne, and B. Neal. 2009. Methodological trends in studies based on verbal autopsies before and after published guidelines. Bulletin of the World Health Organization 87(9):678-682. Kalogriopoulos, N. A., J. Baran, A. J. Nimunkar, and J. G. Webster. 2009. Electronic medical record systems for developing countries: Review. Conference Proceedings Engineering in Medicine and Biology Society 1:1730-1733. Kellam, S. G., and D. J. Langevin. 2003. A framework for understanding “evidence” in prevention research and programs. Prevention Science 4(3):137-153. Khosravi, A., C. Rao, M. Naghavi, R. Taylor, N. Jafari, and A. D. Lopez. 2008. Impact of misclassification on measures of cardiovascular disease mortality in the Islamic Republic of Iran: A cross-sectional study. Bulletin of the World Health Organization 86(9):688-696. Kivimaki, M., M. J. Shipley, J. E. Ferrie, A. Singh-Manoux, G. D. Batty, T. Chandola, M. G. Marmot, and G. D. Smith. 2008. Best-practice interventions to reduce socioeconomic inequalities of coronary heart disease mortality in UK: A prospective occupational cohort study. Lancet 372(9650):1648-1654. Knaul, F. M., H. Arreola-Ornelas, O. Mendez-Carniado, C. Bryson-Cahn, J. Barofsky, R. Maguire, M. Miranda, and S. Sesma. 2006. Evidence is good for your health system: Policy reform to remedy catastrophic and impoverishing health spending in Mexico. Lancet 368(9549):1828-1841. Krishnan, A., B. Nongkynrih, S. K. Kapoor, and C. Pandav. 2009. A role for INDEPTH Asian sites in translating research to action for non-communicable disease prevention and control: A case study from Ballabgarh, India. Global Health Action 2. Kruk, M. E., and L. P. Freedman. 2008. Assessing health system performance in developing countries: A review of the literature. Health Policy 85(3):263-276. Lee, J. 2003. Global health improvement and WHO: Shaping the future. Lancet 362(9401): 2083-2088. Linde, J. A., R. W. Jeffery, S. A. French, N. P. Pronk, and R. G. Boyle. 2005. Self-weighing in weight gain prevention and weight loss trials. Annals of Behavioral Medicine 30(3): 210-216. Lopez, A. D., C. D. Mathers, M. Eszati, D. T. Jamison, and C. J. L. Murray. 2006. Global burden of disease and risk factors. Washington, DC: World Bank. Madon, T., K. J. Hofman, L. Kupfer, and R. I. Glass. 2007. Public health. Implementation science. Science 318(5857):1728-1729. Majumdar, S. R., and S. B. Soumerai. 2009. The unhealthy state of health policy research. Health Affairs (Millwood). Mathers, C. D., D. Ma Fat, M. Inoue, C. Rao, and A. D. Lopez. 2005. Counting the dead and what they died from: An assessment of the global status of cause of death data. Bulletin of the World Health Organization 83:171-177c. MEASURE DHS. 2010. Demographic and health surveys: Measure DHS: Surveys and methodology. http://www.who.int/healthmetrics/library/countries/HMN_8Board_4hi_khm.pdf (accessed March 9, 2010).
OCR for page 182
MEASURE Evaluation. 2007. Sample vital registration with verbal autopsy: An overview. Chapel Hill, NC: Carolina Population Center. http://www.cpc.unc.edu/measure/tools/monitoring-evaluation-systems/savvy (accessed March 8, 2010). Mendis, S., D. Abegunde, S. Yusuf, S. Ebrahim, G. Shaper, H. Ghannem, and B. Shengelia. 2005. WHO study on prevention of recurrences of myocardial infarction and stroke (WHO-premise). Bulletin of the World Health Organization 83(11):820-828. Millennium Development Goals Africa Steering Group. 2008. Achieving the millennium development goals in Africa. New York: United Nations. Murray, C. J., and J. Frenk. 2000. A framework for assessing the performance of health systems. Bulletin of the World Health Organization 78(6):717-731. Ng, N., H. Van Minh, S. Juvekar, A. Razzaque, T. Huu Bich, U. Kanungsukkasem, A. Ashraf, S. Masud Ahmed, and K. Soonthornthada. 2009. Using the INDEPTH HDSS to build capacity for chronic non-communicable disease risk factor surveillance in low and middle-income countries. Global Health Action 2. Nsubuga, P., M. E. White, S. B. Thacker, M. A. Anderson, S. B. Blount, C. V. Broome, T. M. Chiller, V. Espitia, R. Imtiaz, D. Sosin, D. F. Stroup, R. V. Tauxe, M. Vijayaraghavan, and M. Trostle. 2006. Public health surveillance: A tool for targeting and monitoring intervention. In Disease control priorities in developing countries. 2nd ed. New York: Oxford University Press. Pp. 997-1018. Oomman, N., M. Bernstein, and S. Rosenzweig. 2008. Seizing the opportunity on AIDS and health systems. Washington, DC: Center for Global Development. OpenMRS. 2010. OpenMRS. http://openmrs.org/ (accessed February 1, 2010). PAHO (Pan American Health Organization). 2007. Basic country health profiles for the Americas: Summaries. http://www.paho.org/English/DD/AIS/cp_index.htm (accessed October 20, 2009). PAHO. 2008. Pan American version of STEPS. Washington, DC: Pan American Health Organization. PAHO Health Analysis and Statistics Unit. 2007. Regional core health data initiative: Indicators glossary. Washington, DC: Pan American Health Organization. Pangea Global AIDS Foundation. 2009. Report from the Expert Consultation on Implementation Science Research: A requirement for effective HIV/AIDS prevention treatment and scale-up. Cape Town, South Africa: Pangea Global AIDS Foundation. Petersen, L. A., L. D. Woodard, T. Urech, C. Daw, and S. Sookanan. 2006. Does pay-for-performance improve the quality of health care? Annals of Internal Medicine 145(4): 265-272. Rommelmann, V., P. W. Setel, Y. Hemed, G. Angeles, H. Mponezya, D. Whiting, and T. Boerma. 2005. Cost and results of information systems for health and poverty indicators in the United Republic of Tanzania. Bulletin of the World Health Organization 83(8):569-577. Shengelia, B., C. Murray, and O. Adams. 2003. Beyond access and utilization: Defining and measuring health system coverage. In Health systems performance assessment: Debates, methods and empiricism. Edited by C. Murray and D. Evans. Geneva: World Health Organization. Pp. 221-234. Sikanyiti, P., and S. Nalishebo. 2009. Sample vital registration with verbal autopsy: Background, scope, and methodology. Presentation at the 2nd Global HIV/AIDS Surveillance Meeting, Bangkok, Thailand. Spertus, J. A., M. J. Radford, N. R. Every, E. F. Ellerbeck, E. D. Peterson, and H. M. Krumholz. 2003. Challenges and opportunities in quantifying the quality of care for acute myocardial infarction: Summary from the Acute Myocardial Infarction Working Group of the American Heart Association/American College of Cardiology First Scientific Forum on Quality of Care and Outcomes Research in Cardiovascular Disease and Stroke. Circulation 107(12):1681-1691.
OCR for page 183
Stansfield, S. K. 2009. Cardiovascular disease in developing countries: Meeting the challenge. Presentation at the Public Information Gathering Session for the Institute of Medicine Committee on Preventing the Global Epidemic of Cardiovascular Disease, Washington, DC. Stansfield, S. K., J. Walsh, N. Prata, and T. Evans. 2006. Information to improve decision making for health. In Disease control priorities in developing countries. 2nd ed. New York: Oxford University Press. Pp. 1017-1030. Syed, S. B., A. A. Hyder, G. Bloom, S. Sundaram, A. Bhuiya, Z. Zhenzhong, B. Kanjilal, O. Oladepo, G. Pariyo, and D. H. Peters. 2008. Exploring evidence-policy linkages in health research plans: A case study from six countries. Health Research Policy and Systems 6:4. Tegang, S., G. Emukule, S. Wambugu, I. Kabore, and P. Mwarogo. 2009. A comparison of paper-based questionnaires with PDA for behavioral surveys in Africa: Findings from a behavioral monitoring survey in Kenya. Health Informatics in Developing Countries 3(1):22-25. Ten-year evaluation of the regional core health data initiative. 2004. Epidemiological Bulletin 25(3):1-7. Tryhorn, C. 2009. Developing countries drive explosion in mobile phone use. The Guardian, March 2, 2009. Tunstall-Pedoe, H., K. Kuulasmaa, H. Tolonen, M. Davidson, S. Mendis, and WHO MONICA Project. 2003. MONICA monograph and multimedia sourcebook: World’s largest study of heart disease, stroke, risk factors, and population trends 1979-2002. Geneva: World Health Organization. UNAIDS (United Nations Joint Programme on HIV/AIDS). 2009a. New HIV indicator registry improves access to high-quality indicators. http://www.unaids.org/en/KnowledgeCentre/Resources/FeatureStories/archive/2009/20090313_Propertyright_UNDP.asp (accessed February 2, 2010). UNAIDS. 2009b. UNAIDS: Indicator registry. http://www.indicatorregistry.org (accessed February 2, 2010). United Nations Foundation. 2010. United Nations foundation: Mhealth alliance. http://www.unfoundation.org/global-issues/technology/mhealth-alliance.html (accessed March 9, 2010). University of Washington. 2010. Institute for Health Metrics and Evaluation (IHME). http://www.healthmetricsandevaluation.org/ (accessed March 9, 2010). van Kammen, J., D. de Savigny, and N. Sewankambo. 2006. Using knowledge brokering to promote evidence-based policy-making: The need for support structures. Bulletin of the World Health Organization 84(8):608-612. VanWormer, J. J., A. M. Martinez, B. C. Martinson, A. L. Crain, G. A. Benson, D. L. Cosentino, and N. P. Pronk. 2009. Self-weighing promotes weight loss for obese adults. American Journal of Preventive Medicine 36(1):70-73. Veasnakiry, L. 2007. Cambodia briefing note. Geneva: WHO Health Metrics Network, World Health Organization. Warren, C. W., N. R. Jones, A. Peruga, J. Chauvin, J. P. Baptiste, V. Costa de Silva, F. el Awa, A. Tsouros, K. Rahman, B. Fishburn, D. W. Bettcher, and S. Asma. 2008. Global youth tobacco surveillance, 2000-2007. MMWR Surveillance Summaries 57(1):1-28. WHO (World Health Organization). 2008. 2008-2013 action plan for the global strategy for the prevention and control of noncommunicable diseases. Geneva: World Health Organization. WHO. 2010a. STEPS country reports. http://www.who.int/chp/steps/reports/en/index.html (accessed October 8, 2009). WHO. 2010b. STEPS resources. http://www.who.int/chp/steps/resources/en/index.html (accessed October 8, 2009).
OCR for page 184
WHO Health Metrics Network. 2007. Assessment of the Ethiopian national health information system: Final report. Geneva: World Health Organization. WHO Health Metrics Network. 2008a. Assessing the national health information system: An assessment tool. Geneva: World Health Organization. WHO Health Metrics Network. 2008b. Framework and standards for country health information systems. Geneva: World Health Organization. WHO Health Metrics Network. 2010. How can countries benefit from HMN? http://www.who.int/healthmetrics/about/howcancountriesbenefitfromhmn/en/index.html (accessed March 9, 2010). WHO Noncommunicable Diseases and Mental Health Cluster Surveillance Team. 2001. STEPS instruments for NCD risk factors (core and expanded version 1.4): The WHO STEPwise approach to surveillance of noncommunicable diseases (STEPS). Geneva: World Health Organization.