The fifth step—the ranking itself—is left to the judgment of the decision makers or panels and is not an inherent part of the methodology. The only constraint imposed by the methodology is that dominance between pairs be preserved in the final rankings. The rest is left to intuitive judgment, which may be viewed either as an advantage or a limitation of the method.

The method of multiattribute scoring goes beyond multiattribute accounting by generating a composite score for each candidate project. This requires three additional steps: (1) entry of a quantitative score (x_ij) in each cell in the matrix corresponding to the j^th criterion (objective) and the i^th project (vaccine candidate); (2) specifying a set of weights, w_j, by which the individual factor scores will be combined; and (3) computing the weighted scores (s_i),

Projects are ranked according to these scores. As an intermediate step, scores for groups of criteria are often combined into subscores (e.g., a “disease impact” subscore composed of the first three criteria in Table 2.1), and then the subscores are combined. Also, the individual scores are often “normalized” to a 0–100 scale before weighting for computational convenience. Sometimes, multiplicative rather than additive aggregation rules are used.

A hypothetical example of the process of multiattribute scoring is shown in Table 2.2. The end result is that vaccine candidate A is ranked highest, followed by vaccines B, D, and C. If desired, a sensitivity analysis can be performed in which the weights are varied to see whether the rankings change. If only one of the four vaccine candidates in Table 2.2 could be developed, a sensitivity analysis would be desirable because the scores of A and B are so close. However, if two vaccines could be developed, vaccines A and B probably would come out on top for most plausible sets of weights.

Multiattribute scoring and decision analysis with multiple objectives (see below) may incorporate implicit (subjective) judgments about expected outcomes. The committee believes that every effort should be made to use available data in an explicit fashion and to clearly identify and define areas in which personal values may influence the choices.

One obvious limitation of the multiattribute scoring method just described is that the weights are arbitrary. This is especially disconcerting, considering that one is adding such disparate items as likelihood of success and disease mortality.