Cover Image

PAPERBACK
$43.50



View/Hide Left Panel

3
Overview of the Analytic Approach

This chapter presents an overview of the combined cost-effectiveness/decision analysis approach taken by the committee. Although reasonably straightforward in principle, the necessary calculations* demand a substantial amount of quantitative information, expose areas of ignorance, and require value judgments as well as facts. The committee sought the most reliable available data from published sources and experts as a foundation for its calculations. Although concerned about the probabilistic and subjective aspects of its estimates, the committee recognized these elements as unavoidable in a priority setting exercise such as this one. When factual information was not available, the choice was whether needed estimates should be made explicit or left implicit. The committee chose to identify and quantify pertinent estimates rather than to leave them vague or unspecified.

The report strives to identify the sources and reasons for all assumptions and estimates. One purpose is to make it easier to adjust assumptions and assess their effects on the implied rank order. The explicit, quantitative approach also facilitates the performance of sensitivity analyses, in which selected estimates are varied systematically across their plausible ranges to examine their impact on the calculations.

Several assumptions, discussed below, underlie the analysis, and certain issues are best clarified at the outset of any discussion of the methods employed. Some of these are considered further in Chapter 8.

  1. Only specified vaccines and diseases are assessed. The preliminary selection of candidates for new or improved vaccine development (described in Appendix A) was based on expert views of the current state of knowledge about each disease pathogen and the corresponding host response. The supplement to this volume summarizes the committee’s conclusions about some disease problems for which vaccine prospects are

*  

See Appendix F for information on the computer software used for this analysis.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries 3 Overview of the Analytic Approach This chapter presents an overview of the combined cost-effectiveness/decision analysis approach taken by the committee. Although reasonably straightforward in principle, the necessary calculations* demand a substantial amount of quantitative information, expose areas of ignorance, and require value judgments as well as facts. The committee sought the most reliable available data from published sources and experts as a foundation for its calculations. Although concerned about the probabilistic and subjective aspects of its estimates, the committee recognized these elements as unavoidable in a priority setting exercise such as this one. When factual information was not available, the choice was whether needed estimates should be made explicit or left implicit. The committee chose to identify and quantify pertinent estimates rather than to leave them vague or unspecified. The report strives to identify the sources and reasons for all assumptions and estimates. One purpose is to make it easier to adjust assumptions and assess their effects on the implied rank order. The explicit, quantitative approach also facilitates the performance of sensitivity analyses, in which selected estimates are varied systematically across their plausible ranges to examine their impact on the calculations. Several assumptions, discussed below, underlie the analysis, and certain issues are best clarified at the outset of any discussion of the methods employed. Some of these are considered further in Chapter 8. Only specified vaccines and diseases are assessed. The preliminary selection of candidates for new or improved vaccine development (described in Appendix A) was based on expert views of the current state of knowledge about each disease pathogen and the corresponding host response. The supplement to this volume summarizes the committee’s conclusions about some disease problems for which vaccine prospects are *   See Appendix F for information on the computer software used for this analysis.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries unclear or for which more basic research is required before targeted vaccine development will be realistic (see Appendix I). If a candidate vaccine is omitted from the full analysis, it obviously will not appear in the rank order, and no conclusions should be drawn regarding its position relative to the assessed contenders. Differences between the assessments of vaccine candidates for diseases of importance in the United States (Institute of Medicine, 1985) and in the developing world (this volume) mean that values for (or relative rankings based on) benefits and costs should not be compared between analyses. In the first exercise an effort was made to calculate absolute likely benefits that would accrue from vaccine development and the actual net costs (including costs of treatment averted) associated with their use. Because real values were calculated, the incremental cost per unit of benefits could legitimately be used to differentiate between vaccine candidates in the interpretation of rankings on these criteria. In the present analysis, the committee treated certain factors affecting actual vaccine benefits and net costs differently than in the first report. Vaccine utilization is not incorporated into the calculation of benefits (for reasons discussed in Chapter 6), hence benefits are potential rather than expected. Utilization is also not included in cost calculations, and disease treatment costs are also not included (see Chapter 4). These differences in methods have important consequences. The values derived for the health benefits and the expenditures on vaccines to achieve them should be viewed as representing relative vaccine attributes and not their absolute magnitude. Thus, as noted above, it is not appropriate to compare health benefit or cost calculations between the assessments presented in this volume and its companion (Institute of Medicine, 1985). The committee’s method can produce a priority ranking of candidate vaccines, but it is silent on the question of how many of the vaccines are worthy of development. The committee expressly refrained from equating dollars with the value of any health benefits. The analysis has no bearing on basic scientific research. it does not compare the value of further investment in basic scientific research with the benefits or costs of vaccine development. The analysis views costs and benefits from a perspective of the developing world as a whole: it does not anticipate the source of funds for vaccine development, trials, or utilization, or the identity of those who will benefit from potential cost savings. Once the ranking of vaccine candidates has been completed, decision makers at the National Institute of Allergy and Infectious Diseases (NIAID) can determine the most effective distribution of the Institute’s funds among those candidates selected for accelerated development, in the light of their knowledge of other efforts within and outside the United States. The analysis recognizes only those primary economic impacts of new vaccines that were judged likely to differentiate between vaccine candidates, that is, the costs of vaccine development and the expenditures on vaccine for the immunization programs. while the cost

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries (price) of the vaccine for the immunization program (Cv in Figure 3.2) is included, the cost of vaccine administration is a primary economic impact that is not included in the analysis. As discussed in Chapter 6, it was assumed for the purposes of this analysis that all vaccines would be delivered through the World Health Organization’s Expanded Program on Immunization (WHO-EPI). All vaccine candidates could then reach the level of utilization achieved by that program, and differences in utilization need not be incorporated as a factor in the analysis. Similarly, the cost of vaccine administration is assumed not to be significantly different between vaccine candidates if all are delivered through the EPI (although there may be some differences in cost, e.g., between injected and oral vaccines). The first volume of the committee’s report (Institute of Medicine, 1985) presents a method that can be used to estimate the net costs associated with vaccine use, where the cost of treatment averted and differential vaccine utilization are incorporated into the calculations of costs and health benefits. This more detailed approach may be possible where these factors can be reliably estimated, for example, within a specific country. Secondary impacts of vaccines deal with changes in the costs of care for patients who avoid having the disease in question or who develop side effects requiring treatment. (These impacts are sometimes called “induced costs and savings.”) For the reasons discussed in Chapter 4, it was judged impractical to attempt to estimate global averages for treatment costs for the conditions resulting from the target diseases. The tertiary impacts, which are also not considered in this analysis, involve changes in the costs of care for other diseases that the patient may get because the vaccine has prevented death due to the target disease. This analysis covers vaccine priorities for the population of the developing world as a whole. It aggregates vaccine benefits and costs irrespective of the local, national, or regional groupings affected by particular diseases. Chapter 4 contains the working definition of the developing world adopted for this analysis. The analysis treats each potential vaccine as an independent investment decision. For example, the analysis, for reasons discussed in Chapters 4 and 8, does not attempt to incorporate quantitatively the synergism that exists between some diseases, resulting in mortality or more severe morbidity (e.g., measles and diarrhea). Thus, some vaccines may in practice avert a disease burden greater than that nominally attributable to the pathogens against which they protect. Additionally, the analysis does not take into account other interactions, such as the effect of an improved pertussis vaccine on the long-term acceptance of immunization in general or the benefits of an improved polio vaccine on the ease of delivery of other childhood vaccines (see Chapter 8). The method of estimating disease burdens used in this analysis treats diseases as noninteracting phenomena, although possible interactions are recognized as a factor for consideration in the final choice of priorities for accelerated development. If it becomes possible to better quantitate known or suspected interactions, for example, between measles and diarrhea or between viral and bacterial

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries respiratory infections, then future applications of this method might formally account for them in disease estimates. The purpose of the committee’s effort was to provide U.S. government decision makers with a tool to help guide their investments in accelerated development of vaccines for use in developing countries. The approach adopted in this analysis is not necessarily the correct approach for other purposes (see Chapter 8, Table 8.1). It is hoped, however, that the methods outlined in this report and its companion volume (Institute of Medicine, 1985) may be useful to other groups, such as regional organizations or specific countries, in their efforts to establish priorities. METHOD The basic strategy of the approach adopted by the committee is reductionist: each logical component of expected benefits and of expected expenditures is assessed separately; then the components are aggregated in a stepwise fashion for each disease-vaccine contender. The analysis distinguishes valued consequences, that is, benefits and costs, from the probabilistic events that contribute to the likelihood of their occurrence. All component estimates are identified so they may be examined, questioned, and altered, if necessary. The net expected health system expenditures and the net expected health benefits are “annualized” and discounted to their present values. Annualized means that all benefits and expenditures are expressed as steady state (constant) streams, beginning immediately and extending indefinitely into the future. The procedure of discounting converts any benefit and cost streams that are delayed for some years to their equivalent annualized values starting now. Fixed expenditures (e.g., for vaccine development) are “amortized” to produce a constant annual equivalent value. Discounting enhances the relative importance of effects realized after a short, as compared to a long, delay. The discount rate (r) used in the committee’s calculations reflects the preference for present over future consumption of resources. in the central analysis, the discount rate is set at 0.05. The effect of discounting is evaluated in the sensitivity analyses described in Chapter 9. Calculation Procedures A comprehensive approach to comparing costs associated with achieving the health benefits from a vaccine would entail calculating the present value of the annualized equivalent of the net expected health system costs. This would include the cost of development, the cost of the vaccination program, and the cost of adverse side effects, less the cost of medical treatment averted. it may be expressed as

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries (1) where r = discount rate; CDev = cost of vaccine development; PDev = probability of vaccine development; CVP = annual cost of the vaccination program (which includes the the cost [price] of the vaccine CV for the program and the cost of its administration program, Cp); TUse = time until steady-state vaccine use, that is, the time to licensure plus the time to adoption at the predicted use rate; CTr = annual cost of medical treatment averted; TLag = lag between administration of vaccine and realization of health benefits, that is, the delay of vaccination benefits; and CSE = annual medical costs from side effects. Because of the decision that for a global analysis the estimation of average treatment cost was impractical and because of the judgment (discussed in Chapters 5 and 9) that side effects were likely to be negligible, the above formulation simplifies. The expenditures on vaccines to achieve the anticipated benefits can be expressed as (2) where rCDev = amortized cost of vaccine development; and Net expected health benefits, expressed as the annualized equivalent of the present value, consist of clinical benefits adjusted for adverse side effects of the vaccine. The annualized equivalent may be expressed as (3)

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries where symbols are defined as in Equation 1 and B = expected annual steady-state benefits from vaccine, adjusted for efficacy (E) and differential utilization (U), if necessary; and SE = expected annual incidence of vaccine side effects. In this analysis, it is assumed that differential utilization will not be a factor in determining the potential health benefits achievable with vaccine candidates (all are assumed delivered through WHO-EPI) and, as discussed in Chapter 7, side effects are judged to be negligible. Thus, in this analysis the expression for relative benefits simplifies to (4) where B’ = expected annual steady-state benefits from vaccine, adjusted for efficacy but not adjusted for utilization (it thus represents relative potential benefits). Figure 3.1 summarizes the hierarchy of components that would make up a comprehensive analysis of the expected health benefits and expected costs for each prospective vaccine. Figure 3.2 shows the components assessed in the simplified analysis judged appropriate for the purposes of the exercise described in this report. Each element of benefit and of cost is based on estimates related to the target disease or to the subject vaccine. Implementation of the comprehensive approach shown in Figure 3.1 for vaccines for diseases of importance in the United States is described in the committee’s first report (Institute of Medicine, 1985). Chapter 7 of this report describes the implementation of the comparison scheme shown in Figure 3.2. The scheme integrates the components of benefit and cost separately, adjusts each for the probability that it will occur and for the expected delay until realization, and presents the conclusions of a central analysis. INTERPRETATION OF RESULTS The interpretation of results from implementation of the comprehensive approach to the analysis (Figure 3.1)—which develops annualized net expected costs and annualized expected health benefits for each candidate vaccine—is discussed in the committee’s first report (Chapter 3, Institute of Medicine, 1985). The description below illustrates procedures for the interpretation of the results of the simplified analysis shown in Figure 3.2.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries FIGURE 3.1 Summary of comprehensive analytic strategy.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries FIGURE 3.2 Summary of analytic strategy adopted for comparison of vaccine candidates for developing countries.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries TABLE 3.1 Sample Array of Hypothetical Benefits and Expenditures for Various Vaccines Vaccine Annualized Present Value of Potential Health Benefits Annualized Present Value of Potential Expenditures on Vaccines ($ millions) A 250 10 B 200 20 C 200 40 D 100 80 E 150 100 F 100 200 G 125 300 H 105 400 I 150 500 J 80 1,000 NOTE: Health benefits may be expressed in any unit as long as it is used consistently for all diseases or vaccines. After all assumptions and estimates have been made and all calculations performed, the analysis yields the annualized potential health benefits of and the annualized potential expenditures on vaccines for each candidate. These could be arrayed in a table like that shown in Table 3.1. The entries in the table illustrate the interpretation of conceivable results and do not represent actual results. Table 3.1 summarizes results for 10 vaccine candidates, A through J. Each project is associated with a potential expenditure and a potential benefit, both adjusted to their present values to make results with different time horizons comparable to one another, and converted to annualized equivalents. The figures take into account many (but not all) uncertainties that apply to the development, use, and consequences of each vaccine. However, some factors (e.g., utilization) are assumed not to differentiate between vaccine projects, and some costs or possible savings (e.g., treatment cost savings), which may differentiate between projects, are not included in the calculations. Hence, the values are not real or absolute. Within a category they should only be regarded as relative. That is, differences between benefits of vaccine candidates or differences between costs are meaningful, but comparisons of benefits and costs for a particular vaccine are not. Built into the interpretation of these results is an assumption that society is risk-neutral with respect to alternative vaccine investments. This means, for example, that the benefits from a vaccine investment that has a 50 percent chance of ultimately saving 2,000 lives per year are valued the same as the benefits from an alternative vaccine investment that has a 25 percent chance of ultimately saving

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries 4,000 lives per year. The expected number of lives saved per year is 1,000 in both cases. In Table 3.1 a potential expenditure necessary to achieve the potential health benefit is listed for each project. In reality the net costs associated with a vaccine may be negative (i.e., a savings might be realized) if the treatment costs averted by its use (not included in the expenditure estimates) outweigh the expenditures on vaccines and their administration. The committee proposes that the potential global health benefit of a vaccine take precedence in determining its initial ranking for accelerated development priority. The affordability of the benefit, represented by the relevant expenditures on vaccines, can also be entered into the decision process if desired, along with a variety of further nonquantifiable considerations (discussed in Chapter 8). Since the expenditures on vaccines do not represent net costs, the committee favors using the information provided by the ranking on this criterion as a secondary input into the decision-making process. The priority ranking of vaccine candidates for accelerated development on the basis of potential health benefits is a straightforward process. The spacing of the numerical values may permit grouping of vaccines into clear categories. Depending on the degree of confidence decision makers have in estimates incorporated into the calculations for vaccine candidates that achieve nearly equivalent health benefit values, it may be necessary to resort to other criteria, for example, affordability or availability of other control measures to inform choices. In either case, decision makers should examine the nonquantifiable considerations outlined in Chapter 8 to guide the final selection of priorities. The calculation procedures outlined in this report are an aid to—not a substitute for—the final process of informed decision making. The process for ranking vaccines in order of desirability depends on the type of constraint that limits the number of candidate vaccines that may be selected. One constraint for NIAID, for example, could be the total funds available to the agency for investment in new vaccines. In this case the ranking process would need to account for the anticipated investment required from NIAID for each candidate vaccine. This might influence the number of accelerated development projects pursued, the particular projects pursued, or both. These are issues best decided by NIAID policymakers; the proposed method merely informs the decision process. “Affordability” or willingness to pay to achieve benefit can be incorporated into the decision process in one of two ways. First, adjustments can be made to the health benefit values to reflect the effect of various levels of financial resource constraints; this may affect the rankings. This procedure is illustrated in Chapter 9. Second, costs may be considered as equal in importance to benefits. If decision makers wish to incorporate the costs of vaccine development and use into the ranking process as a decision criterion equally important to the potential health benefit, then rankings can sometimes be developed based on the concept of dominance of one investment over another. If vaccine x is better on one dimension

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries TABLE 3.2 Rankings of Various Hypothetical Vaccinesa Annualized Present Value of Potential Health Benefits Annualized Present Value of Potential Expenditures on Vaccines ($ millions) A 250 A 10 B, C 150 B 20 E, I 150 C 40 G 125 D 80 H 105 E 100 D, F 100 F 200 J 80 G 300   H 400   I 500   J 1,000 aGreater benefits and lower expenditures are preferred. (either benefits or affordability) than vaccine y, and if x is as good as or better than y on the second dimension, then the choice of vaccine x dominates y. The first step in this procedure is to rank vaccine candidates on the basis of health benefit (greater is preferable) and on expenditure (lower is preferable). For the vaccines A through J listed in Table 3.1, the rankings are shown in Table 3.2. GUIDELINES Applying the test for dominance noted above to these candidate vaccines produces the results shown in Table 3.3 and summarized below: vaccine A dominates all others vaccine B dominates all except A vaccine C dominates all except A and B vaccine D dominates F and J vaccine E dominates F, G, H, I, and J vaccine F dominates J vaccine G dominates H and J vaccine H dominates J vaccine I dominates J By the rule of dominance, the top three social investments (as judged by these criteria) are vaccines A, B, and C, and the least attractive of the listed vaccines is J. The need to proceed further depends on the number of alternatives to be selected for development: if only one, two, or three were desired, we could identify the priorities as vaccines A, A and B, or A and B and C, respectively. The procedures for further selections are outlined below.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries Step 1 If S vaccines are to be selected from among N vaccine contenders, select any candidate vaccine that dominates at least the number (N–S) of other vaccines. In the example, there are 10 vaccine candidates (N=10). To select four candidates for investment (S=4), begin by choosing any that dominate as many as six (10–4) other vaccines. From the bottom row of Table 3.2, it is apparent that vaccines A, B, and C satisfy this condition, and so would be selected. This would leave one more to be selected. Step 2 Eliminate any candidate vaccines that are dominated by the number of vaccines that will be selected for investment. If, for example, we want to invest only in four vaccines, we should eliminate F, G, H, I, and J because, from the far right column of Table 3.2, these are each dominated by at least four other vaccines. This leaves two remaining candidates for the fourth vaccine, D and E. In the system employed in this assessment the choice between vaccines D and E requires consideration of the other factors listed in Chapter 8. It may also require judgments about considerations omitted from the cost calculations, such as the potential savings from treatment averted for some vaccines. (In the more comprehensive form of the assessment where “real” costs can be related to “real benefits,” the incremental cost-effectiveness (C/E) ratio can be used to differentiate between contenders. This process requires value judgments about society’s willingness to forgo resource savings or to incur costs in TABLE 3.3 Vaccine Dominance   A B C D E F G H I J Total A —   0 B X —   1 C X X —   2 D X X X —   3 E X X X   —   3 F X X X X X —   5 G X X X   X   —   4 H X X X X X —   5 I X X X X   —   4 J X X X X X X X X X — 9 Total 9 8 7 2 5 1 2 1 1 0   NOTE: Column dominates row; see text for explanation.

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries order to realize health benefits. These procedures are described and illustrated in the first volume of the committee’s report [Institute of Medicine, 1985].) SUMMARY The approach described in this report is recommended by the committee for the selection of priorities for accelerated development of vaccines against diseases prevalent in developing countries because it separately identifies each logical component of potential benefits and potential expenditures associated with individual vaccine contenders. The analysis distinguishes quantifiable consequences from the probability they will occur and also incorporates information on when the consequences are likely to occur. In addition, the approach requires that an effort be made to state the sources and reasons for all assumptions and estimates. The committee suggests that the potential global health benefits of a vaccine take precedence in determining its initial ranking for accelerated development. The affordability of the benefits, as represented by the relevant expenditures on vaccines, should also be considered along with other nonquantifiable considerations (discussed in Chapter 8) in the final selection of priority projects. The committee does not attempt to place a monetary value on health benefits, to suggest how many vaccines are worthy of development, to compare investment in basic scientific research with investment in accelerated vaccine development, or to anticipate the source of funds for any vaccine-related programs. The approach adopts a perspective for the developing world as a whole on health benefits and expenditures. This does not imply that the priorities that emerge from this analysis are necessarily those that should be adopted in all circumstances. The methods used in this analysis (and additional procedures used to estimate differences in vaccine utilization and cost savings from treatment averted) can be applied by others for determining priorities for specific countries or regions. The selection of candidates for accelerated vaccine development should be an ongoing process. One of the benefits of the model is that it provides a structured format in which to incorporate new research findings. This is especially important, given the rapid development of new techniques in biotechnology. After the annualized expenditures and the annualized potential health benefits have been determined for each vaccine candidate, the results must be interpreted based on the type of constraints that limit the number of candidates that may be selected. Specific procedures exist for incorporating affordability (willingness to pay) into the rankings based on health benefits or for using expenditures as a decision criterion equal to health benefits. If ranking or dominance considerations alone do not provide a complete slate of candidates, decision makers must make judgments on the basis of various other

OCR for page 30
New Vaccine Development: Establishing Priorities, Volume II, Diseases of Importance in Developing Countries nonquantifiable considerations, such as the availability of alternative control measures. REFERENCE Institute of Medicine. 1985. New Vaccine Development: Establishing Priorities, Volume I. Diseases of Importance in the United States. Washington, D.C.: National Academy Press.