Summary

Over the centuries, from Edward Jenner to Bill Gates, as our scientific understanding of diseases has increased, so has the focus on prioritizing new vaccines to help achieve better health. Despite the expanding interest in and support toward improving global health, constraints inherent to vaccine development and delivery present decision makers with difficult choices. Given the lack of effective tools and models to assist the decision-making process, renewed attention is needed to improve the approaches available for priority setting and for guiding investment decisions.

Prioritizing vaccines—“arranging in the order of relative importance”—is a time-and resource-intensive process requiring diverse considerations. Examples of such considerations include the emergence and reemergence of disease threats, limits in the progress of research related to the disease in question, technological feasibility, economic and other resource constraints, possibilities for enhancing vaccine administration methods, and other broader objectives. Decision makers involved in setting priorities come from different constituencies with different perspectives. Therefore, it becomes vitally important to develop not only a practical approach that provides a common language to assist decision making but also a flexible tool that embraces a wide spectrum of inputs and perspectives in efforts to advance vaccine development.

This report, Ranking Vaccines: A Prioritization Framework, describes a decision-support model and the blueprint of accompanying software being developed to help prioritize vaccines. The consensus study that produced this report is being carried out in two phases. Phase I work, described in this report, provides the conceptual underpinning of Strategic Multi-Attribute Ranking Tool for Vaccines, or SMART Vaccines. SMART Vaccines Beta, developed by the committee in Phase I, is not available for public use. SMART Vaccines 1.0 is expected to be released at the end of Phase II, when it will be fully operational and capable of guiding discus-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary Over the centuries, from Edward Jenner to Bill Gates, as our scientific understanding of diseases has increased, so has the focus on prioritizing new vaccines to help achieve better health. Despite the expanding inter- est in and support toward improving global health, constraints inherent to vaccine development and delivery present decision makers with difficult choices. Given the lack of effective tools and models to assist the decision- making process, renewed attention is needed to improve the approaches available for priority setting and for guiding investment decisions. Prioritizing vaccines—“arranging in the order of relative impor- tance”—is a time- and resource-intensive process requiring diverse con- siderations. Examples of such considerations include the emergence and reemergence of disease threats, limits in the progress of research related to the disease in question, technological feasibility, economic and other resource constraints, possibilities for enhancing vaccine administration methods, and other broader objectives. Decision makers involved in setting priorities come from different constituencies with different perspectives. Therefore, it becomes vitally important to develop not only a practical approach that provides a common language to assist decision making but also a flexible tool that embraces a wide spectrum of inputs and perspec- tives in efforts to advance vaccine development. This report, Ranking Vaccines: A Prioritization Framework, describes a decision-support model and the blueprint of accompanying software being developed to help prioritize vaccines. The consensus study that produced this report is being carried out in two phases. Phase I work, described in this report, provides the conceptual underpinning of Strategic Multi-Attribute Ranking Tool for Vaccines, or SMART Vaccines. SMART Vaccines Beta, developed by the committee in Phase I, is not available for public use. SMART Vaccines 1.0 is expected to be released at the end of Phase II, when it will be fully operational and capable of guiding discus- 1

OCR for page 1
2 RANKING VACCINES: A Prioritization Framework sions about prioritizing the development and introduction of potential new vaccines. In the committee’s view, a “new vaccine” (or “vaccine candidate”) can refer not only to a completely novel vaccine but also to an existing vac- cine given improvements to some of its features, including innovations in its production or delivery methods. The audience and potential users of SMART Vaccines include those institutions funding and carrying out basic biomedical research, private firms involved in vaccine production, philanthropic foundations with a strong interest in vaccination and global health programs, international health organizations, and high-level decision makers, such as ministers for health, commerce, and finance or senior administrators. The committee’s charge Phase I of the study was supported by the National Vaccine Program Office of the U.S. Department of Health and Human Services. The Phase I state- ment of task is presented in Box S-1. Phase II of the study is oriented toward expanding and enhancing the capabilities of the model and transforming SMART Vaccines Beta to SMART Vaccines 1.0. This report describes the committee’s approach toward demonstrat- ing a proof of concept using three hypothetical vaccine candidates that have not yet been developed. The committee included a broad range of attri- butes that represent the various perspectives relating to vaccine develop- ment and impact. Some of the data for these attributes are readily available (such as population characteristics), while other data are estimated by the user (e.g., qualitative attributes of the vaccines) or through expert opinion (e.g., disease burden or cold-chain requirements). Because the data inputs in this report were not intended to be precise, readers should not take any output of SMART Vaccines Beta as the “exact” or “recommended” priority value relating to any particular vaccine; instead the outputs should be seen only as illustrative examples of how the model and beta software currently operate. Previous Institute of Medicine reports Previous Institute of Medicine (IOM) studies from 1985–1986 and 2000 that focused on vaccine prioritization provided specific lists of vaccine ranks. The two-volume IOM study New Vaccine Development, released in 1985–1986, prioritized vaccines both for the United States and from an

OCR for page 1
3 Summary BOX S–1 Committee on Identifying and Prioritizing New Preventive Vaccines for Development Institute of Medicine Phase I Statement of Task Task 1: Review domestic and global research and development pri- oritization activities relevant to identifying new preventive vaccine targets. Task 2: Develop an analytical framework and model for prioritizing vaccines of domestic and global importance. Engage stakeholders to inform the process of the model development and implementa- tion. Task 3: Test and validate the model using two to three predeter- mined vaccines, including at least one vaccine candidate of domes- tic importance and one of global importance. Task 4: Prepare a report containing the analytical framework and model for evaluating and prioritizing vaccine targets along with rec- ommendations as to how to use the model for reviewing the catalog of preventive vaccines every 2 to 3 years. international perspective, based on infant mortality equivalents—a proxy measure of health burden. The 2000 report Vaccines for the 21st Century focused entirely on the U.S. population and, unlike the 1985–1986 report, used an efficiency measure for ranking vaccines: incremental cost per incremental quality- adjusted life years saved ($/QALY), a measure derived from a classic wel- fare economics model. The cost-effectiveness model of the 2000 report represented important progress toward vaccine prioritization, but it did not provide guidance for answering some challenging questions often encountered in decision making. For instance, the model provided no guid- ance on how to choose between two diseases with equal QALYs when one was a low-impact disease affecting the majority of the population and the other a disease with few cases but with very high mortality and potential large-scale social disruption.

OCR for page 1
4 RANKING VACCINES: A Prioritization Framework While both of the earlier reports noted that vaccine prioritization can include aspects of social value beyond net costs (or savings) and health burden reduction, these variables were considered to be beyond the scope of the cost-effectiveness or infant-death-equivalents-prevented frame- work. SMART Vaccines significantly expands the single criterion frame- work of the earlier prioritization efforts to include a number of additional criteria that influence decision making in vaccine development. An overview of SMART Vaccines The committee’s principal contributions have been broadening the set of criteria for valuing preventive vaccines and demonstrating how the selec- tion of criteria and data can influence the prioritization process. Users are offered a choice of up to 29 attributes drawn from broad categories which include health burden considerations, economic considerations, demo- graphic considerations, public concerns, scientific and business consider- ations, programmatic considerations, and policy considerations. Table S-1 presents the general list of attributes influencing the rank of vaccine candi- dates in SMART Vaccines. Because decision makers may represent different constituencies, their criteria for prioritizing various vaccine candidates are likely to differ as well. Further, each of these selected criteria can be valued and weighed differently in the prioritization process. Thus, not only does SMART Vac- cines broaden the scope of the valuation criteria, but it also allows users to select and weigh criteria according to their values or those of the commu- nities they represent. From the technical standpoint, SMART Vaccines Beta expands the utility function for evaluating vaccines compared to the models published in the earlier reports. But the fact that different users may make differ- ent choices when using SMART Vaccines adds further value: It provides a framework to compare, discuss, and perhaps reconcile differing priori- ties. Thus, rather than pre-specifying which criteria are used and how they should be weighed, the committee has opted to allow the users to select their own. Model and software development The modeling strategy of the committee was based on multi-attribute util- ity theory. The multi-attribute utility approach has a well-grounded theo- retical basis, but employing the theory for SMART Vaccines presented var- ious challenges. The report discusses how the committee sought to tackle

OCR for page 1
5 Summary TABLE S-1 Choices of Attributes in SMART Vaccines Beta Health • Premature Deaths Averted per Year Considerations • Incident Cases Prevented per Year • QALYs Gained or DALYs Averted Economic • One-Time Costs Considerations • Annual Net Direct Costs (Savings) of Vaccine Use • Annual Net Workforce Productivity Gained • Cost-Effectiveness Demographic • Benefits Infants and Children Considerations • Benefits Women • Benefits Socioeconomically Disadvantaged • Benefits Military Personnel • Benefits Other Priority Population Public Concerns • Availability of Alternative Public Health Measures • Potential Complications Due to Vaccines • Disease Raises Fear and Stigma in the Public • Serious Pandemic Potential Scientific • Likelihood of Financial Profitability and Business for the Manufacturer Considerations • Likelihood of Successful Licensure in 10 Years • Demonstrates New Production Platforms • Existing or Adaptable Manufacturing Techniques • Potential Litigation Barriers Beyond Usual • Interests from NGOs and Philanthropic Organizations Programmatic • Potential to Improve Delivery Methods Considerations • Fits into Existing Immunization Schedules • Reduces Challenges Relating to Cold-Chain Requirements Intangible Values • Eradication or Elimination of the Disease • Vaccine Raises Public Health Awareness Policy • Special Interest for National Security, Considerations Preparedness, and Response • Advances Nation’s Foreign Policy Goals these challenges throughout the model and software development and evaluation process. Early prototypes were modeled after the one presented in the 2000 report. The committee then began the development of a user-friendly soft- ware interface to enable data input with the aim of incorporating sensi- tivity testing, advanced dynamic modeling, and improved visualization of results in the future. As mentioned earlier, this software will be avail- able for public use at the end of Phase II. This report provides illustrative

OCR for page 1
6 RANKING VACCINES: A Prioritization Framework screenshots of SMART Vaccines Beta, which is currently under develop- ment. The committee also engaged consultants to serve as concept evalua- tors to help improve the design and features of SMART Vaccines from the perspective of potential users. SMART Vaccines uses two submodels—a computational submodel and a value submodel—to combine the levels of various attributes into a single measure of priority “score” for each vaccine under consideration. The weights used for criteria in the model must satisfy a number of condi- tions in order for the model to work properly. Normally, satisfying these conditions would require users to make many explicit quantitative value comparisons. To minimize these demands on the user in the current ver- sion of the model, the committee adopted the rank order centroid method to approximate additive multi-attribute utility weights. The only require- ment that this method places on users is that they rank order the impor- tance of attributes selected for their prioritization model. The rank order information is used to derive numerical weights which are then used in a scoring function. This approach is known to produce weights that are robust and predictive of the users’ eventual decisions. SMART Vaccines Beta permits only an ordinal ranking of the vaccine attributes with no tie scores. The committee selected three diseases for evaluation: influenza, tuberculosis, and group B streptococcus. These diseases were compared between two countries, the United States and South Africa. Representa- tive test results are discussed in this report with the acknowledgement that sensitivity testing and further validation will be required in Phase II of this study. To demonstrate the extent to which the selection and ranking of attributes affects the priority scores among vaccines generated by the model, the committee conducted a “value experiment” in which commit- tee members and staff selected attributes and provided ranking scores for six hypothetical vaccines: an influenza vaccine with a 1-year immunity; an influenza vaccine with a 5-year immunity; a tuberculosis vaccine with a 3-year immunity; a tuberculosis vaccine with lifetime immunity; an influ- enza vaccine with a 1-year immunity but with 50 percent increased cover- age; and a tuberculosis vaccine with a 3-year immunity but in a setting with a 100-fold increase in disease prevalence. The results of this experiment, as described in this report, show how each user’s selection and weighting of attributes shifted the final rankings among these six hypothetical vaccines. The purpose of this experiment was to emphasize both the importance of the attribute-weighting process in the final rankings and the sensitivity of the ranks to preferences inherent in the decision-making process.

OCR for page 1
7 Summary Data requirements SMART Vaccines Beta requires substantial data inputs from users. In some cases, depending on the country for which the model is employed, the data required to drive the model may be sparse or unavailable. The usefulness of SMART Vaccines will rely upon concerted data collection and future soft- ware enhancements. The model requires refined age- and sex-specific population data; these can generally be imported from the World Health Organization and other existing data sources. SMART Vaccines Beta also requires quantita- tive inputs concerning age- and sex-specific disease burdens to the popu- lation of interest, typical patterns of vaccination and health care use (and their costs) for relevant illnesses with and without the availability of a pre- ventive vaccine, and health complications that might arise from the use of a new vaccine. These data are not widely available at this time and will likely have to be provided at least in part by processes led or guided by expert opinion. “Expert opinion” in this context refers to input from some- one who is able to provide knowledgeable, informed estimates about the data needed within the country or region of interest. Economic data are also needed on typical wage rates for workers in each age group in order to compute worker productivity gains achieved by reducing or eliminating disease burden—both in workers directly and, indirectly, in children they may care for—through vaccination. The model’s computational engine uses all data and other user- supplied entries to calculate a series of attributes, including cost- effectiveness, premature deaths averted, incident cases prevented, annual health care costs saved, and net annual gains in worker productivity. These quantities are computed through detailed modeling of the disease and its prevention through vaccination in the population over time. SMART Vaccines Beta also allows users to specify qualitative attri- butes for each potential new vaccine, features that are not captured within the computed attributes, and add additional new attributes per their choice. These include, by general category, attributes focusing on the abil- ity of existing health infrastructures to deliver the new vaccine; whether the vaccine has the capability of disease eradication; whether the vaccine targets major population health risks (such as pandemic diseases or bio- terrorism attacks); and the likelihood of successful development, which in turn hinges on the likelihood of scientific progress and regulatory approval. Potential users of SMART Vaccines will have the option to include or not include any of these attributes in generating their final priority ranking; obviously, if an attribute is not used in generating the priority ranking, that obviates the need to provide related data for the candidate vaccines.

OCR for page 1
8 RANKING VACCINES: A Prioritization Framework Ways to use (and not to use) SMART Vaccines By design, SMART Vaccines offers users considerable flexibility in speci- fying attributes and their rank order to determine the final prioritization score. Among other things, this means that SMART Vaccines does not produce one unique list of priorities among vaccine candidates, unlike the techniques in the predecessor IOM reports in 1985–1986 and 2000. The rankings are sensitive to the choice and the order of attributes and to the trade-offs the user is willing to accept in determining priorities. SMART Vaccines does not “make decisions.” It is intended to be used exclusively as a decision-support tool and only that. The committee expects that a major use of SMART Vaccines will be to facilitate discussions about attributes and values among diverse users, helping them to converge upon mutually beneficial priorities and collaborations. The committee envisions that various organizations could use SMART Vaccines independently to guide their efforts in vaccine develop- ment and implementation. This might begin at the basic science level in organizations conducting and funding research to break through bottle- necks in vaccine development. Other potential users, such as manufactur- ers, might be involved directly in the development and eventual production of vaccines and thus may wish to emphasize an entirely different set of vac- cine attributes (e.g., profitability, development and regulatory risks) com- pared to a basic research organization. Still some users (or user consor- tia) might use SMART Vaccines to enhance market stability (say, through pre-purchase agreements) and hence the likelihood of successful vaccine development. SMART Vaccines can help diverse users understand how and why their rankings differ. Variations in rankings due to differing data inputs can be discussed among users to discover common data sources. When the model produces different results as a consequence of differing values, it can motivate discussions relating to individual or inter-institutional pri- orities among users. SMART Vaccines may also help inform users of the value of strengthening vaccine delivery methods (e.g., by augmenting the cold-chain capacity) and alternative methods of disease control (e.g., clean water supply, mosquito netting, food safety measures, or health-related education). A further expected benefit of using SMART Vaccines is that it will enable users to identify data needs to ultimately improve their vac- cine prioritization process. Future data collection activities, surveillance activities, and resource allocation may be informed and planned by use of SMART Vaccines.

OCR for page 1
9 Summary Observations and next steps This report is intended to introduce potential users to the concept of SMART Vaccines and to encourage stakeholders to inform the develop- ment of SMART Vaccines 1.0 in Phase II of this study. The committee will next enhance SMART Vaccines Beta, test its use with three additional vac- cine candidates of domestic and global importance, and further improve the user interface as part of the development of SMART Vaccines 1.0. The value of SMART Vaccines will depend, in part, on data that need to be generated as vaccine candidates evolve and as disease epidemiology becomes better characterized in different parts of the world. In the future— beyond Phase II—an active community of users and an open-source envi- ronment would likely lead to future enhancement of the SMART Vaccines’ capabilities. Potential enhancements could include creation and sharing of databases for populations from different countries, the enhancement of validation tools and user interface, and the development of ways to address the risk and uncertainty surrounding the characterization of vaccines that have not yet been developed. This study is the first step in moving toward these goals.

OCR for page 1