Page 32

3
Methodologic Considerations in Evaluating the Evidence

THE NATURE OF THE EVIDENCE

The committee has undertaken the task of judging whether each of a set of adverse events may occur as a result of exposure to pertussis or rubella vaccines. These judgments have both quantitative and qualitative aspects, and they reflect the evidence examined and the approach taken to evaluate it. In this chapter, the committee describes more fully how it has approached its task, in the hope that readers may then be in a better position to assess and interpret the committee's findings. By offering this information, the committee wishes to make the report useful to those who may seek to update its conclusions as new information is obtained. This chapter outlines the specific questions the committee posed, the types of evidence it identified, its approaches to evaluating reports both singly and collectively, and the nature of the conclusions it felt that logic and evidence permitted. Against this background, details of the analysis and specific conclusions concerning each type of adverse event appear in subsequent chapters.

Attributes both of the adverse events being considered and of the population exposed to vaccine influenced the committee's analysis. The events can be characterized, for example, by their frequency, discreteness, and specificity and by prior knowledge of their etiology and pathogenesis. Events that occur only rarely in exposed persons are more difficult to study than those that occur more frequently. Conditions that are ill-defined, that are



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 32
Page 32 3 Methodologic Considerations in Evaluating the Evidence THE NATURE OF THE EVIDENCE The committee has undertaken the task of judging whether each of a set of adverse events may occur as a result of exposure to pertussis or rubella vaccines. These judgments have both quantitative and qualitative aspects, and they reflect the evidence examined and the approach taken to evaluate it. In this chapter, the committee describes more fully how it has approached its task, in the hope that readers may then be in a better position to assess and interpret the committee's findings. By offering this information, the committee wishes to make the report useful to those who may seek to update its conclusions as new information is obtained. This chapter outlines the specific questions the committee posed, the types of evidence it identified, its approaches to evaluating reports both singly and collectively, and the nature of the conclusions it felt that logic and evidence permitted. Against this background, details of the analysis and specific conclusions concerning each type of adverse event appear in subsequent chapters. Attributes both of the adverse events being considered and of the population exposed to vaccine influenced the committee's analysis. The events can be characterized, for example, by their frequency, discreteness, and specificity and by prior knowledge of their etiology and pathogenesis. Events that occur only rarely in exposed persons are more difficult to study than those that occur more frequently. Conditions that are ill-defined, that are

OCR for page 32
Page 33 known to occur in the absence of vaccine exposure, or that generally have unknown causes or mechanisms of development are also inherently difficult to investigate. Under these circumstances, epidemiologic studies offer important advantages over clinical experience and intuition, although these limiting characteristics affect epidemiologic studies also. When the great majority of the population is exposed, as is generally true for pertussis and rubella vaccines, comparisons between exposed and nonexposed persons become clouded. This is due to the potential for selective factors against vaccination to confound the relation between immunization status and the occurrence of adverse events. For example, a decreased relative risk of SIDS has been observed in several studies in the time period immediately following DPT immunization. Although a protective effect of vaccine cannot be ruled out, it is more plausible that children who are not immunized by the recommended age are at increased risk for SIDS because of other factors, such as socioeconomic status, that are associated with both delaying immunization and SIDS (see Chapter 5). Other aspects of vaccine exposure, such as changes in vaccine formulation, single versus multiple occasions of administration, and the age pattern of administration also bear on interpretation of the evidence. QUESTIONS TO BE ADDRESSED What would it mean to say that a vaccine causes one or another type of adverse event? It would not mean that exposure invariably produces the adverse event, nor that all cases of the event were due to the vaccine. Such complete correspondence between exposures and events is by far the exception in public health and does not occur in the present context, or the present review would not be required. In general matters of health and disease, different causal explanations may apply even to a single disease. For example, the question of what causes typical cases of a particular disease is quite distinct from the question of what causes epidemic outbreaks of that disease. The answers are also distinct, in that the first might be a specific microorganism and exposure conditions of the individual case, whereas the second could entail complex ecologic and social factors suddenly favoring the widespread transmission of the microorganism. Clearly, different senses of ''cause" are implied in these two questions.  Although each of these questions concerns the causation of disease, the answers require different types of evidence. This example suggests the importance of stating clearly the questions about causation to be answered. In the present review, the committee has been concerned with causal questions of three kinds. The first of these questions about exposure to pertussis or rubella vaccine is, in general, can it cause the specified adverse

OCR for page 32
Page 34 condition? For example, can rubella vaccine cause chronic arthritis? If the conclusion is affirmative, a second question becomes pertinent: How frequently does it cause that condition? Or, how frequently is arthritis a result of rubella vaccination? The third question, which applies to a particular instance or case of an adverse event, is did it cause that specific event? Or, did rubella vaccine cause this particular individual to develop arthritis? Discussion of each of these three types of questions will help to indicate the committee's view of its task. (1) Can vaccine cause the adverse event? While the nature of causation has a deep philosophical underpinning, the work of the committee necessarily focused on a pragmatic question: What is the nature of the evidence relevant to drawing its conclusion about causation? In pursuing this question, the committee recognized that an absolute conclusion about the absence of causation may never be attained. As in science generally, studies of adverse events following vaccination are not capable of demonstrating a zero effect, that is, that the purported effect is impossible or could not ever occur. Any instrument of observation has a limit to its resolving power, and this is true as well of randomized clinical trials and epidemiologic studies. Hence, the committee could not prove the absence of any possibility of an adverse event caused by vaccine. Rather, in the absence of evidence suggesting an effect, and especially in the presence of evidence not consistent with causation, the committee could only conclude that the evidence fails to demonstrate an effect. The evidentiary base that the committee found to be most helpful derived from epidemiologic studies of populations. Here, the primary question is whether an association exists between the exposure (immunization) and the event. To determine whether an association exists, epidemiologists estimate the magnitude of an appropriate quantitative measure (such as the relative risk or the odds ratio1) that describes the joint occurrence of exposures and events in defined populations or groups. Values of relative risk greater than 1 may indicate a positive, or direct (harmful), association and are emphasized in this discussion; values between 1 and 0 may indicate a negative, or inverse (protective), association. The observed relative risk in the study population must be sufficiently distant from unity to meet a stated criterion of significance before an association is said to be apparent. 1 Usage of "relative risk," "odds ratio," or "estimate of relative risk" is not consistent in the literature reviewed and cited in this report. In its own usage, the committee intends that ''relative risk" be used to refer to the results of cohort studies and "estimates of relative risk" or "odds ratio" be used to refer to the results of case-comparison studies (see Glossary of Terms for definitions).

OCR for page 32
Page 35 Formally, in planning an investigation, an epidemiologist poses a hypothesis to the effect that the exposures and events under study are independent, or not associated. Under this hypothesis, the value of the measure of association used is theoretically expected to be approximately 1. This is termed the null hypothesis or the hypothesis of no association. The measure of association derived from the investigation is then tested statistically. To "reject the null hypothesis," or to conclude that exposures and events are not independent, is to conclude that there is evidence of an association. When more than one epidemiologic study has been conducted, it may be instructive to combine their results so as to reach a stronger conclusion than a single study can provide. This process is described more fully later in this chapter. Determining whether an observed association is causal requires additional scrutiny. This is because there may be alternative explanations for an observed association. These include errors in the design, conduct, or analysis of the investigation; bias, or a systematic tendency to distort the measure of association from representing the true relation between exposures and events; confounding, or distortion of the measure of association because another factor, related to both exposures and events, has not been recognized or taken into account in the analysis; and chance, the effect of random variation in producing observations that can, in reality, only be approximations to the truth and can, with some probability, sometimes depart widely from the truth. In deciding whether vaccine can cause any particular type of event, then, it has been the committee's task to judge in each instance whether there is evidence of association from the available studies, and if so, whether it is direct or inverse, and whether it is due to error, bias, confounding, or chance or, instead, due to a causal relation between vaccine and event. (2) How frequently does vaccine cause the event? The second type of causal question, which becomes pertinent only if the answer to the first question is affirmative, concerns the proportion of individuals in a specified population who experience the adverse event because of the exposure. The most desirable evidence as a basis for answering this type of question involves knowledge of the rate of occurrence of the event in those who are exposed, the rate in those who are not exposed (the "background" rate of the event in the population), and the degree to which any other differences between exposed and unexposed persons influence the difference in rates. The term attributable risk is generally used to denote the difference in rates between exposed and unexposed groups. This is a simple measure of the frequency with which the occurrence of the event in exposed persons may be due to the vaccine exposure and not to the other

OCR for page 32
Page 36 causes that account for the event in the absence of vaccine exposure (see Glossary of Terms). By also taking into account the frequency of exposure in the population, a further measure can be calculated which indicates the number of cases in a total population that may be due to the exposure. When information was available, in the presence of an association judged by the committee to be causal, the committee has attempted to indicate how much of the occurrence of the event in question might be caused by vaccine exposure. (3) Did vaccination cause a particular case of the adverse event? A third type of causal question is whether, in a specific instance of exposure and an adverse event, it can be concluded that the event was caused by the vaccine exposure. This question is especially pertinent to those types of adverse events for which case reports constitute a substantial part of the evidence available for review.  Three different sets of circumstances bear on the nature of this question. First, it may have been judged in general that the exposure can cause this type of event. In this instance the question concerning any particular case needs to consider the similarity between the circumstances of that case and the circumstances in which the general conclusion was reached that such causation can occur.  If other causes of the same type of event are known, their possible role in this individual case must be considered also. Second, most evidence for a general proposition of causation may be negative, yet circumstances surrounding a single case may raise the question of whether causation may be attributed to the exposure of this individual. To judge that a particular case was caused by the exposure under these circumstances would entail the reversal of the more general conclusion on the basis of a single case. It seems extremely unlikely that such a reversal could be justified on the basis of the evidence concerning an individual case. Third, there may be no evidence of the sort required to judge the presence or absence of association. Here the judgment in a specific case would depend on the circumstances concerning that case alone, in isolation from any conclusion about a causal relation in general. On consideration of several aspects of the evaluation of case reports, discussed in the section below, even the strongest affirmative answer based on the individual case would be inherently weaker, or less securely supported, than it would be if a general conclusion supporting vaccination as a cause of the event had been reached. This is not to say that any such conclusion reached on the basis of the case report alone is necessarily incorrect. However, it would be highly subject to error in light of the properties of most of the events considered here and the nature of the evidence available about them in the case report.

OCR for page 32
Page 37 THE BURDEN OF PROOF In approaching its task, the committee considered the concept of "burden of proof' and its place in such an evaluation. This concept implies that one position or another concerning causation is presumed to be true unless it is offset by evidence to the contrary.  The prior position might be either affirmative or negative. That is, it may be assumed that an exposure is harmful unless sufficient evidence of safety is present; alternatively, it may be assumed that an exposure is safe unless convincing evidence of harmful effects is present. In either case, it is sometimes argued that a burden of proof must be fulfilled before the presumed position is rejected. In general, it is desirable to avoid making an error in either direction, concluding either that there is or that there is not a causal relation when the opposite is true. Reducing the chance of such mistaken conclusions depends on careful assessment of the evidence, including consideration of possible errors, bias, and confounding. The role of chance in leading to erroneous conclusions as a result of random variation in sampling or in other respects is customarily handled through formal statistical analyses, which are based on assumptions from probability theory. Statistical measures can suggest the likelihood that conclusions of the presence or absence of association will each be in error. In general, a result is said to have greater statistical significance as the probability of error in accepting an association becomes smaller.2The likelihood that a true association will be correctly detected in an investigation is a statistical property of the investigation, termed its power. Both statistical significance and power reflect the role of chance in scientific observations and the concomitant uncertainty in all scientific conclusions. One obvious implication of this understanding is that the concept of "proof" in its common-sense meaning is not strictly applicable to scientific observations. Even when scientists conclude that an experiment demonstrates ("proves") causation, they know there is a small, statistically definable probability that the conclusion is incorrect. The committee began its evaluation presuming neither the existence nor the absence of association. It has sought to characterize and weigh the strengths and limitations of the available evidence. Subsequent chapters of the report summarize the evidence concerning each vaccine-event relation under review and present the committee's conclusions. If the first question (can the vaccine cause the adverse event?) was answered affirmatively, and 2 More technically, the level of statistical significance is the probability of observing by chance at least as great a difference as that observed between an "experimental" (exposed) and a "control" (unexposed) group, if the risk of the adverse event were, in truth, identical in the two groups.

OCR for page 32
Page 38 if available information permitted, the second question (how frequently does the vaccine cause the adverse event?) was answered. In a few cases, conclusions based on the answer to the third question (did the vaccine cause the adverse event in this individual case?) are proposed (see, for example, the section on hemolytic anemia in Chapter 6). Of necessity, such a conclusion is especially tentative if the more general question of whether the vaccine can cause the adverse event remains unanswered. Furthermore, the committee's task was not to judge individual cases, except when this was the only way to shed light on the general question of causality. It should be noted that the committee's charge was to focus on questions of causation and not broader topics, such as cost-benefit or risk-benefit analyses of vaccination, which are not considered in this report. With this orientation to the committee's task and approach in mind, the following sections discuss the characteristics of the types of evidence that bear on the causal questions at hand. CATEGORIES OF EVIDENCE Experiments in Humans Randomized Controlled Trials Theoretically, the ideal method for assessment of causal relations between treatments and adverse events is the randomized controlled trial because, when appropriate and feasible, it is the most scientifically rigorous method for testing such hypotheses. Randomized controlled trials are experiments in which subjects are randomly allocated, often in a masked fashion, into "treatment" and "control" groups, to receive or not to receive an intervention such as, in the present context, a monovalent vaccine; the control group receives an injection of an inert substance (placebo) or an established alternative vaccine; and both groups are followed up in a strictly comparable manner to determine the relative frequencies of outcomes and events of interest. Although they are theoretically ideal, such trials have ethical and practical limitations for investigating the causal connections of concern here. Widespread acceptance of routine immunization against pertussis and rubella creates ethical barriers to withholding vaccination from some participants to permit a placebo-controlled trial (Cherry et al., 1988). In addition, because these adverse events are generally rare, the sample sizes required to detect them reliably would be much larger than those required to evaluate vaccine efficacy. In fact, since the first reports of serious adverse events following administration of pertussis and rubella vaccines (Madsen, 1933; Modlin et al., 1975), virtually no placebo-controlled or other experimental

OCR for page 32
Page 39 studies in humans of the adverse events covered in this report have been published (Cherry et al., 1988; Plotkin, 1988; Preblud, 1985). A number of early studies of pertussis vaccine in the United States and the United Kingdom did include unexposed controls, but these studies were primarily concerned with efficacy and not with adverse events (e.g., Kendrick and Eldering, 1939; Lapin, 1943; Medical Research Council, 1951, 1956, 1959). More recently, a small number of studies have compared adverse reaction rates in children following DPT versus DT vaccine administration (Cody et al., 1981; Pollock et al., 1984) or in adults following rubella vaccine versus natural rubella infection (Tingle et al., 1986). Predominantly, however, pertinent randomized trials have focused on comparisons of pertussis vaccine reaction rates between different injection sites, vaccines of different manufacturers, prior reaction histories, or different immunization schedules (Baraff et al., 1984, 1985; Barkin and Pichichero, 1979). Other studies have compared reaction rates by vaccine type, for example, whole-cell versus acellular pertussis vaccines (e.g., Edwards et al., 1986; Lewis et al., 1986; Pichichero et al., 1987) or between rubella vaccine strains (e.g., Barnes et al., 1972; Isacson et al., 1971; Polk et al., 1982). In general, these studies have been primarily concerned with evaluation of transient reactions, whether local or systemic, and not chronic adverse events of the types included in this review. One exception is a randomized, double-masked, placebo-controlled trial of rubella vaccine and chronic arthritis that is currently in progress in Vancouver, British Columbia, Canada. Results are expected in 1992 (A. Tingle, British Columbia Children's Hospital, personal communication, 1991). Other Experimental Studies Thus, few randomized controlled trials contribute to the assessment of the causal questions considered in this report. And although it should be noted for completeness that other experimental approaches, such as formal communitywide comparisons of the impact of vaccination programs, including both beneficial outcomes and adverse events, are applicable in principle, evidence of this type is also generally unavailable. Experiments in Animals: Animal Models In principle, experimental studies in animals allow for both rigid control over vaccine exposure and intensive observation for any adverse events that may follow. If an animal model is to be considered valid for the study of a human disease, however, the manifestations of the disease should be similar in the two species. The starting point is generally what is currently known about the human disease.

OCR for page 32
Page 40 With respect to evaluation of pertussis vaccine, the committee found that the information gained from animal models of whooping cough is difficult to apply to humans for two critical reasons: Knowledge of virulence factors for the organism and of the pathogenesis of whooping cough is incomplete and largely superficial, and Bordetella pertussis is not a natural pathogen for animals. Nonetheless, several studies have been conducted and are reviewed below. With respect to rubella, the committee could find no studies of animal models of either the disease or rubella vaccine-related illness that were specific to the adverse events under consideration. The discussion below therefore refers specifically to pertussis. Several additional factors make it difficult to apply findings in animal models of infection to their human counterparts, and these difficulties hold for the study of pertussis. These factors are either specific characteristics of infection and response in a particular species or more general considerations in judging the relevance to humans of studies in animals. First, the capacity of any particular organism to infect (i.e., to colonize and replicate in) a host varies a great deal across host species. This variability depends in turn on several factors, including how closely the organism's surface antigens resemble those of the host. If the microbe's prominent antigens are like those of the host, rapid replication might occur before the host would recognize the invader as foreign. Some organisms initiate their relationship with the host by binding to specific host receptors or antigens; these binding sites vary among species. An animal's immunologic repertoire, consisting of antibodies and programmed lymphocytes, is generated by natural exposure to antigens. Across species this experience with antigens varies with diet and habitat, particularly proximity to humans or other animals. Second, disease is the product of a stimulus (such as a bacterium and its toxins) and the host response. The inflammatory response is similar but far from identical across mammalian species, and the disease can be expressed differently in different species. For example, different animals vary in their capacity to inactivate bacterial endotoxin and in their susceptibility to allergic anaphylaxis. Third, results in animal models of infectious diseases can vary greatly with the conditions of the initial exposure of the host to the organism. If animal models of infection involve the same point of entry, the same vehicle (e.g., dust or aerosolized droplets), and similar numbers of organisms in order to produce natural infection, they are more likely than less analogous models to predict the results that will be obtained in humans. Fourth, the material used for studies in animals (in this case, the organism and its products) should, ideally, be the same as that to which the human is exposed. Attention must be paid to the bacterial strain and to possible differences in metabolism and virulence within the strain.

OCR for page 32
Page 41 Fifth, host defense mechanisms mature at markedly different rates in different species. The stage of immunologic development in the test animal needs to be understood in relation to development of the immune system in the human. Results with animal models can only suggest possible relations or outcomes in humans. Observations made in animal models represent only an initial step in the process of applying animal experience to human disease and its prevention. Findings from the animal model must be confirmed by the study of humans, and that principle is clearly relevant to the study of either whooping cough or the adverse events that follow pertussis vaccination. With respect to the study of adverse events following exposure to rubella or rubella vaccine, such questions of relevance are moot, since it has not been possible to develop an animal model for rubella infection. These general requirements for the applicability of data from animal models to human conditions limit the usefulness of the information currently available regarding pertussis vaccine. No pertinent information is available from animal models regarding rubella vaccine. Observational studies in humans have been a more useful basis for making judgments about the possibility of causation of adverse events by pertussis and rubella vaccines. (See Appendix C for further discussion of the animal models used to study pertussis and pertussis vaccine.) Controlled Epidemiologic Studies (Observational) In contrast to randomized controlled trials and other experimental studies in humans, many epidemiologic investigations are observational. This means simply that the occurrences of exposures or events of concern, such as pertussis vaccination or a particular adverse event, are studied as they arise in the usual course of life and not under the conditions of a planned experiment. Observational studies in populations are often controlled, however, through various strategies of formal comparative investigation. For example, the experience of adverse events in a group after receiving pertussis vaccine can be compared with that in an unvaccinated group (unexposed control group). Alternatively, the prior vaccination history of a group that has developed irreversible encephalopathy can be compared with that of a group free of this condition (unaffected control group). In these two strategies, the experience of the control or comparison group provides an estimate of the frequency either of events in the absence of exposure or of exposure in the absence of the event, as experienced in the general population. Thus, the contribution of the control group in such studies is analogous to that of the placebo group in a controlled trial. The most relevant types of such controlled, observational studies for the

OCR for page 32
Page 42 present review and their main characteristics are described in this section. Examples of studies related to pertussis or rubella vaccine and adverse events serve for illustration. Cohort Studies Cohort studies track groups that are defined by common characteristics, including their exposure status, for example, vaccinated and unvaccinated, at the starting point of observation. The rates of occurrence of adverse events are compared between these groups over time. All study participants are known or presumed to be free of the disease or events under investigation at the start of the study. In the well-designed cohort study, reliable estimates of event rates in each group can be obtained.  Especially for uncommon events, large populations, prolonged periods of observation, or both are required (Last, 1988). Such studies can provide evidence that bears on both the first and second types of causal questions discussed earlier in this chapter. By direct comparison of rates in exposed versus nonexposed groups, a measure of association, termed the relative risk, is derived. From the same results, the attributable risk can be determined, and with knowledge of the frequency of exposure in the general population, the population attributable risk can also be calculated. Thus, the questions of whether exposure is associated with the event and, if so, to what degree can both be answered with the results of the cohort study. The starting point of the investigation can be either contemporaneous or in the past. In the first case, termed concurrent cohort studies, all observations, including both exposures and events, may be subject to direct observation by the investigator. In the second case, which typically depends on the availability of records of past exposures and events, the entire study may relate to experience prior to the start of the investigation. Such studies are termed historical cohort studies. Some features are common to both types of cohort studies, and others are distinct in accordance with their different temporal strategies. For example, in a historical cohort study, Griffin and colleagues (1990) evaluated the risks of seizures and encephalopathy in a cohort of 38,171 children in Tennessee on Medicaid who had collectively received a standard schedule of 107,154 DPT immunizations in the first 3 years of life (see Chapter 4 for details). The use of historical records permitted a more rapid and less expensive investigation than would have been afforded by a concurrent cohort design. One potential weakness of the historical cohort study design, the dependence on possibly incomplete and unreliable historical records for information, was reduced in the study of Griffin and colleagues because of the

OCR for page 32
Page 54 recognized, however, that perfect specificity could not be expected given the multifactorial etiology of many of the adverse events under examination. Biologic Plausibility  Biologic plausibility is based on whether a possible causal association fits existing biologic or medical knowledge. The existence of a possible mechanism, such as an established association of the adverse event with natural disease (e.g., thrombocytopenic purpura following natural rubella infection), was thought to increase the likelihood that the vaccine-event association could be causal. Other Considerations  As noted above, it is important also to consider whether alternative explanations—error, bias, confounding, or chance—might account for the finding of an association. If an association could be sufficiently explained by one or more of these considerations, there would be no need to invoke the several considerations listed above. From this viewpoint, an inference of causation could be based solely on the exclusion of these alternatives.  Because these alternative explanations can rarely be excluded sufficiently, however, assessment of the applicable considerations listed above almost invariably remains. The final judgment is then a balance between the strength of support for the causal interpretation and the degree of exclusion of alternatives. Other considerations were also entertained in the evaluation of the evidence of association. One special consideration in evaluating summary evidence on the relation of adverse events to pertussis or rubella immunization was that of the variation in vaccine composition observed across manufacturers. With respect to the whole-cell pertussis vaccine, for example, the committee recognized that methods of production, seed bacteria, preservatives, and adjuvants used in manufacturing the vaccine have varied substantially over the years (Cox et al., 1987; Ross, 1988) and vary even now by manufacturer and country. In some countries, such as the United States, adsorbed vaccines are used exclusively. In others, both plain (fluid) and adsorbed vaccines are available, and in some, only plain vaccines are available. In most countries, pertussis immunization is given in conjunction with diphtheria and tetanus toxoids in a combined DPT product. In countries where inactivated polio vaccine is used, a quadruple antigen (diphtheria, tetanus, pertussis, and polio) vaccine is used. In yet other countries, pertussis vaccine is given primarily as a monovalent vaccine. The committee also recognized that pertussis immunization schedules have varied markedly by country and time period. In the United States, for example, DPT vaccine is currently administered in five doses at ages 2, 4, 6, and 18 months and 4 to 6 years (American Academy of Pediatrics, 1988).

OCR for page 32
Page 55 In The Netherlands, DPT-polio vaccine is given in four doses at ages 3, 4, 5, and 11 to 14 months (Health Council of The Netherlands, 1987). Rubella-containing vaccines, like pertussis vaccine, have also varied considerably across place and time. For example, three vaccine strains were used initially in the United States following licensure in 1969-1970: HPV-77 (duck embryo), HPV-77 (dog kidney), and Cendehill (rabbit kidney) (Hilleman et al., 1969; Meyer et al., 1969; Prinzie et al., 1969). Soon after, the RA 27/3 human diploid fibroblast vaccine was licensed in Europe (Plotkin et al., 1969), and both the HPV-77 (dog kidney) and the Cendehill vaccines were subsequently withdrawn from U.S. licensure. In 1979, RA 27/3 was licensed in the United States, and the HPV-77 (duck embryo) vaccine was withdrawn, leaving RA 27/3 as the only U.S.-licensed rubella vaccine (Perkins, 1985). In addition to strain variations, rubella-containing vaccines also vary in composition. In the United States, and increasingly elsewhere, rubella vaccines are combined in a triple vaccine also containing measles and mumps vaccine viruses (MMR). However, bivalent vaccines containing rubella and measles or rubella and mumps vaccines are also used. What, then, are the implications of these variations in vaccine composition and schedules for the evaluation of vaccine-adverse event associations? With respect to rubella vaccines, rates of arthritis and arthralgia following immunization were found to differ by strain (see Chapter 7). Integration of evidence across studies was therefore problematic. This issue was considered moot with respect to radiculoneuritis and other neuropathies and thrombocytopenic purpura, since the evidence for these adverse events was limited to isolated case reports. Unlike rubella vaccines, however, the committee considered the potential problem of variability in whole-cell pertussis vaccine composition to be much greater. For example, in the last 10 years of testing of pertussis vaccines at the National Institute for Biological Standards and Control (NIBSC) in the United Kingdom, whole-cell vaccines have shown wide variation in biologic activity from batch to batch, although with no significant time-related trends (K. Redhead, National Institute for Biological Standards and Control, personal communication, 1990). Testing did reveal that plain pertussis vaccines are usually more active than adsorbed vaccines, a finding consistent with studies cited in the report (e.g., Pollock et al., 1984) which indicate that the frequency of moderate systemic reactions following primary immunization with adsorbed DPT vaccine is similar to that with adsorbed DT vaccine, but considerably less than that with plain DPT vaccine. In summary, the general approach to evaluation outlined in this chapter was applied to each type of adverse event as dictated by the nature of the available evidence.

OCR for page 32
Page 56 THE NATURE OF THE CONCLUSIONS This chapter has demonstrated that judgments about the possible causation of adverse events by pertussis or rubella vaccine and similar causal questions reflect both quantitative and qualitative reasoning. Some final observations will help to clarify the nature of the committee's conclusions. Quantitation Resolution Resolution refers to the fineness or sharpness of detail that can be discriminated by a particular mode of observation. In light microscopy, for example, observations are described by reference to the optical properties of the lens, such as 10x, 100x, or higher magnification. Electron microscopy, with very much higher resolution, distinguishes structural features not detectable with light microscopy. Resolution in epidemiologic studies concerns the capacity of a study to discriminate between the frequencies of events or of exposures between groups in order to determine the presence or absence of associations. By analogy, resolution in epidemiology also depends in a sense on magnification, that is, on the order of magnitude of the numbers of participants—for example, from tens to hundreds of cases and controls in case-comparison studies and from hundreds to thousands of exposed and unexposed subjects in cohort studies. With equally valid observations, results based on the experience of increasing numbers of persons, from single individuals to tens, hundreds, or thousands of individuals, provide successively greater resolution. The resolution or discriminating capacity of epidemiologic studies could theoretically be increased indefinitely through ever larger study populations. However, there are many constraints on the feasibility of large studies. Rarity of exposures or events or other circumstances may limit the resolution even of large studies. Meta-analysis can, under the appropriate circumstances discussed above, be used to offset the limited size of individual studies, but the collective magnitude of the contributing studies may still be less than desired. It should be emphasized that in all such studies the potential for bias is a key problem and that enlarging the study only reduces random error, not systematic error. Therefore, if bias is present, a firmer but still erroneous conclusion will result from a larger study than from a smaller one. Power calculations indicate the probability of achieving discrimination of a predetermined degree under the design of a given study. Power is thus a quantitative measure of the capacity of a study to achieve a given degree

OCR for page 32
Page 57 of resolution. In particular, it provides guidance against overconfidence in the absence of an association when a study with relatively low power has failed to demonstrate one. As such, it is an aid in appreciating the nature of the evidence that underlies conclusions about causation and is described further here to indicate its role in the present review. As discussed earlier in this chapter, two types of error must be taken into account in designing and interpreting statistical tests. Epidemiologic studies are often designed to provide statistical tests that minimize type I error, the probability that the null hypothesis is falsely rejected. Commonly, such tests are designed so that there is less than a 5 percent chance that the test will incorrectly indicate an association between a vaccine and an adverse event if no association truly exists. For any given test and sample size, on the other hand, there is some chance that the test will err in failing to find an association when one truly exists. This is called a type II error. The chance of making such an error increases when both the true excess risk and the sample size are small. From another perspective, given a particular sample size and a specified probability of a type I error, one can calculate the power of a test to detect an assumed association of a given magnitude. Because the power of a test is the opposite (technically, the complement) of the probability of making a type II error, the power of a test increases when both the true excess risk and the sample size are large. For example, Shields and colleagues (Melchior, 1977; Shields et al., 1988) compared the ages of onset of various disorders under different vaccination schedules in two time periods in Denmark and found no statistically significant differences between the two periods in the onset of infantile spasms. However, there were only 80 cases of infantile spasms in the two study periods combined, and the lack of a significant statistical association may therefore reflect the small sample size rather than a true absence of association. In other words, if this investigation were replicated in a country with more cases of infantile spasms than occurred in Denmark, a statistically significant difference might be detected, if the relation was, in truth, causal. Furthermore, according to the power calculations described in Appendix D, if 50 percent of cases of infantile spasms were caused by pertussis vaccine, there would be nearly a 90 percent chance that a sample of the size used by Shields and colleagues would detect this relationship. If, on the other hand, only 25 percent of the cases of infantile spasms were caused by pertussis vaccine, Shields and colleagues' test has about a 45 percent chance of detecting the relationship. Power calculations are also valuable in interpreting apparently conflicting results of multiple studies of the same vaccine-adverse event combination. If findings of no association were concentrated in the low-power studies, for instance, the suggestion that no association exists would be weakened.

OCR for page 32
Page 58 Because power calculations help to illuminate an important aspect of the uncertainty in the evidence it evaluated, the committee decided to calculate, whenever possible, the power of the statistical tests on which its conclusions were based. These calculations are described in Appendix D. Uncertainty and Confidence All science, including the spectrum from particle physics to astrophysics, is characterized by uncertainty. Scientific conclusions concerning the result of a particular analysis or set of analyses can range from highly uncertain to highly confident. As discussed earlier in this chapter, the theoretical concept of proof does not apply in evaluating actual observations. In its review, the committee attempted to assess the degree of uncertainty associated with the results on which it had to base its conclusions. For individual studies, confidence intervals around estimated results such as relative risks represent a quantitative measure of uncertainty. Confidence intervals present a range of results that, with a predetermined level of probability, include the true relative risk being estimated. When it is possible to use meta-analysis to combine the results of different studies, a combined estimate of the relative risk and confidence interval may be obtained. Appendix D describes the methods used for meta-analysis in the report. For an overall judgment about causation based on a whole body of evidence, beyond the results of single studies or of meta-analyses, no quantitative method exists for characterizing the uncertainty of the conclusions. Thus, to assess the appropriate level of confidence to be placed on the ultimate causal conclusions, it may be useful to consider qualitative as well as quantitative aspects. Quality Comprehensiveness An important aspect of the quality of a review such as the present one is comprehensiveness. This is to ensure against the possibility of any serious omission or inappropriate exclusion of evidence from consideration. If any such omission should be identified, a determination would be needed of whether its inclusion would likely affect the overall results and, if so, in what way. In this report the committee has documented in detail its approach to seeking and identifying the evidence to be reviewed (see Appendix A, Strategies for Gathering Information). Numerous parties were invited to supplement the materials already under review and to notify the committee of any recognized omissions of importance.

OCR for page 32
Page 59 Neutrality Neutrality is another important consideration in the quality of such conclusions as those presented by the committee. This is to ensure a fair weighing of all of the evidence. In this connection, the committee avoided the posture of the burden of proof approach, as discussed earlier in this chapter. The essential evidence, its main strengths and limitations, and the conclusions that follow are stated for each adverse event considered. Judgment The evaluation of evidence to reach conclusions about causation goes beyond quantitative procedures, at several stages: assessing the relevance and validity of individual reports; deciding on the possible influence of error, bias, or confounding on the reported results; integrating the overall evidence, within and across diverse areas of research; and formulating the conclusions themselves. These aspects of the review required thoughtful consideration of alternative approaches at several points. They could not be accomplished by adherence to a prescribed formula. Rather, the approach described here evolved throughout the process of the review and was determined in important respects by the nature of the evidence, exposures, and events at issue.  Both the quantitative and the qualitative aspects of the process that could be made explicit were important to the overall review. Ultimately, the conclusions expressed in this report about causation are based on the committee's collective judgment. The committee endeavored to express its judgments as clearly and precisely as the available data allowed. SUMMARY OF THE EVIDENCE Table 3-1 summarizes the types of evidence reviewed for each adverse event and the respective contribution of each to the committee's judgments about causation. The evidence is organized under five headings: (1) human experiments; (2) animal experiments; (3) case-comparison, cohort, and other controlled studies, (4) case reports and case series; and (5) biologic plausibility. The first four categories were discussed earlier in this chapter. The fifth category, biologic plausibility, includes background knowledge concerning the pathophysiology of an adverse event, attributes of a particular vaccine, or other biologic information derived from research in such areas as immunology and physiology. Where evidence was available in a particular category, the committee judged whether that evidence was generally supportive or not supportive of causation or whether it was insufficient for a determination. For example,

OCR for page 32
Page 60 TABLE 3-1 Categories of Evidence Reviewed for Each Adverse Event: Is the Evidence Supportive of Causation?a  Vaccine and Adverse Event Human Experiments Animal Experiments Case-Comparison, Cohort, and Other Controlled Studies Case Reports and Case Series Biologic Plausibility (Chapter of Report) Yesb ?c Nod Yes ? No Yes ? No Yes ? No Yes ? No                                 DPT                               Infantile spasms (4)                 X   X         Hypsarrhythmia (4)     X               X         Aseptic meningitis (4)               X     X         Acute encephalopathye (4)         X   X     X       X   Chronic neurologic damage (4)         X     X     X     X   Sudden infant death syndrome (5)                 X   X         Anaphylaxis (6)         X     X   X     X     Autism (6)                               Erythema multiforme or other rash (6)                     X   X     Guillain-Barrè syndrome (polyneuropathy) (6)                     X         Peripheral mononeuropathy (6)                     X         Hemolytic anemia (6)                     X   X    

OCR for page 32
Page 61 Vaccine and Adverse Event Human Experiments Animal Experiments Case-Comparison, Cohort, and Other Controlled Studies Case Reports and Case Series Biologic Plausibility (Chapter of Report) Yesb ?c Nod Yes ? No Yes ? No Yes ? No Yes ? No Juvenile diabetes (6)         X     X     X         Learning disabilities and hyperactivity (6)                 X   X         Protracted inconsolable crying and screaming (6)             X     X     X     Reye syndrome (6)               X       X       Shock and ''unusual shocklike state" (6)               X           X                                   Thrombocytopenia (6                     X                                         RA 27/3 Rubella                               Arthritis (7)                               Acute X           X     X     X     Chronic   X               X     X     Radiculoneuritis and other neuropathies (7)                     X   X     Thrombocytopenic purpura (7)                     X   X     a Blanks for any given category of evidence indicate that evidence of this kind is lacking. b Yes, Evidence of this kind is supportive of causation. c ?, Evidence of this kind cannot be classified either as supportive or as not supportive of causation. d No, Evidence of this kind is not supportive of causation. e Defined in controlled studies reviewed as encephalopathy, encephalitis, or encephalomyelitis.

OCR for page 32
Page 62 where there were relevant controlled studies which, overall, had relative risks of greater than 1, the evidence was classified as "supportive of causation." Blanks for any given category of evidence indicate that evidence of that type was lacking.  It is important to note that any one category of evidence generally was not sufficient in itself to support a conclusion of causality, since other aspects of the evidence, including the number and quality of contributing studies, the details of results obtained, and other considerations outlined earlier in this chapter all weighed into the committee's evaluation. The committee found it convenient to classify its conclusions about each adverse event under one of five categories, reflecting the strength and direction of its conclusions. These categories are: 1. No evidence bearing on a causal relation 2. Evidence insufficient to indicate a causal relation 3. Evidence does not indicate a causal relation 4. Evidence is consistent with a causal relation 5. Evidence indicates a causal relation. The remaining chapters elaborate on the evidence assembled as the basis of the committee's findings and conclusions. REFERENCES Alderslade R, Bellman MH, Rawson NSB, Ross EM, Miller DL. 1981. The National Childhood Encephalopathy Study: a report on 1000 cases of serious neurological disorders in infants and young children from the NCES research team. In: Whooping Cough: Reports from the Committee on the Safety of Medicines and the Joint Committee on Vaccination and Immunisation. Department of Health and Social Security. London: Her Majesty's Stationery Office. American Academy of Pediatrics. 1988. The Red Book. Report of the Committee on Infectious Diseases, 21st edition. Peter G, ed. Elk Grove, IL: American Academy of Pediatrics. Baraff LJ, Cody CL, Cherry JD. 1984. DTP-associated reactions: an analysis by injection site, prior reactions, and dose. Pediatrics 73:31-36. Baraff LJ, Cherry JD, Cody CL, Marcy SM, Manclark CR. 1985. DTP vaccine reactions: effect of prior reactions on rate of subsequent reactions. Developments in Biological Standardization 61:423-428. Barkin RM, Pichichero ME. 1979. Diphtheria-pertussis-tetanus vaccine: reactogenicity of commercial products. Pediatrics 63:256-260. Barnes EK, Altman R. Austin SM, Dougherty WJ. 1972. Joint reactions in children vaccinated against rubella. Study II: comparison of three vaccines. American Journal of Epidemiology 95:59-66. Berg JM. 1958. Neurological complications of pertussis immunization. British Medical Journal 2:24-27. Byers RK, Moll FC. 1948. Encephalopathies following prophylactic pertussis vaccination. Pediatrics 1:437-457. Centers for Disease Control. 1984. Adverse Events Following Immunization: Surveillance

OCR for page 32
Page 63 Report No.1, 1979-1982. Atlanta: Public Health Service, U.S. Department of Health and Human Services. Centers for Disease Control. 1986. Adverse Events Following Immunization: Surveillance Report No.2, 1982-1984. Atlanta: Public Health Service, U.S. Department of Health and Human Services. Centers for Disease Control. 1989. Adverse Events Following Immunization: Surveillance Report No.3, 1985-1986. Atlanta: Public Health Service, U.S. Department of Health and Human Services. Centers for Disease Control. 1990. Vaccine adverse event reporting system—United States. Morbidity and Mortality Weekly Report 39:730-733. Cherry JD, Brunell PA, Golden GS, Karzon DT. 1988. Report of the task force on pertussis and pertussis immunization—1988. Pediatrics 81(6, part 21):939-984. Cody CL, Baraff LJ, Cherry JD, Marcy SM, Manclark CR. 1981. Nature and rates of adverse reactions associated with DTP and DT immunizations in infants and children. Pediatrics 68:650-660. Corsellis JAN, Janota I, Marshall AK. 1983. Immunization against whooping cough: a neuropathological review. Neuropathology and Applied Neurobiology 9:261-270. Coulter HL, Fisher BL. 1985. DPT: A Shot in the Dark. New York: Harcourt Brace Jovanovich. Cox NH, Morley WN, Forsythe A. 1987. Vaccine reactions and thiomersal. British Medical Journal 294:250. Edwards KM, Lawrence E, Wright PF. 1986. Diphtheria, tetanus, and pertussis vaccine: a comparison of the immune response and adverse reactions to conventional and acellular pertussis components. American Journal of Diseases of Children 140:867-871. Griffin MR, Ray WA, Mortimer EA, Fenichel GM, Schaffner W. 1990. Risk of seizures and encephalopathy after immunization with the diphtheria-tetanus-pertussis vaccine. Journal of the American Medical Association 263:1641-1645. Health Council of The Netherlands. 1987. Adverse Reactions to Vaccines in the National Vaccination Programme in 1986. The Hague, The Netherlands: Gezondheidsraad. Hill AB. 1971. Principles of Medical Statistics, 9th edition. New York: Oxford University Press. Hilleman MR, Buynak EV, Whitman JE, Weibel RD, Stokes J. 1969. Live attenuated rubella virus vaccines: experiences with duck embryo cell preparations. American Journal of Diseases of Children 118:166-171. Isacson P, Kehrer AF, Wilson H, Williams S. 1971. Comparative study of live, attenuated rubella virus vaccines during the immediate puerperium. Obstetrics and Gynecology 37:332-337. Kendrick PL, Eldering G. 1939. A study in active immunization against pertussis. American Journal of Hygiene 29:133-153. Kleinbaum DG, Kupper LL, Morgenstern H. 1982. Epidemiologic Research: Principles and Quantitative Methods. Belmont, CA: Lifetime Learning Publications. Lapin JH. 1943. Whooping Cough. Springfield, IL: Charles C Thomas. Last JM, ed. 1988. A Dictionary of Epidemiology, 2nd edition. New York: Oxford University Press. Lewis K, Cherry JD, Holroyd HJ, Baker LR, Dudenhoeffer FE, Robinson RG. 1986. A doubleblind study comparing an acellular pertussis-component DTP vaccine with a whole-cell pertussis-component DTP vaccine in 18-month old children. American Journal of Diseases of Children 140:872-876. Madsen T. 1933. Vaccination against whooping cough. Journal of the American Medical Association 101:187-188. Martin GI, Weintraub MI. 1973. Brachial neuritis and seventh nerve palsy: a rare hazard of DPT vaccination. Clinical Pediatrics 12:506-507.

OCR for page 32
Page 64 Mausner JS, Kramer S. 1985. Epidemiology: An Introductory Text, 2nd edition. Philadelphia: W.B. Saunders Co. Medical Research Council. 1951. The prevention of whooping cough by vaccination. British Medical Journal 1:1463-1471. Medical Research Council. 1956. Vaccination against whooping cough: relation between protection in children and results of laboratory tests. British Medical Journal 2:454-462. Medical Research Council. 1959. Vaccination against whooping cough: the final report. British Medical Journal 1:994-1000. Melchior JC. 1977. Infantile spasms and early immunization against whooping cough. Danish survey from 1970 to 1975. Archives of Disease in Childhood 52:134-137. Meyer HM, Parkman PD, Hobbins TE, Larson HE, Davis WJ, Simsarian JP, Hopps HE. 1969. Attenuated rubella viruses: laboratory and clinical characteristics. American Journal of Diseases of Children 118:155-165. Modlin JF, Brandling-Bennett AD, Witte JJ, Campbell CC, Meyers JD. 1975. A review of five years' experience with rubella vaccine in the United States. Pediatrics 55:20-29. Perkins FT. 1985. Licensed vaccines. Reviews of Infectious Diseases 7:S73-S76. Pichichero ME, Badgett JT, Rodgers GC, McLinn S, Trevino-Scatterday B, Nelson JD. 1987. Acellular pertussis vaccine: immunogenicity and safety of an acellular pertussis vs. a whole cell pertussis vaccine combined with diphtheria and tetanus toxoids as a booster in 18- to 24-month old children. Pediatric Infectious Disease Journal 6:352-363. Plotkin SA. 1988. Rubella vaccine. In: Plotkin SA, Mortimer EA, eds. Vaccines. Philadelphia: W.B. Saunders Co. Plotkin SA, Farquhar JD, Katz M, Buser F. 1969. Attenuation of RA27/3 rubella virus in WI38 human diploid cells. American Journal of Diseases of Children 118: 178-185. Polk BF, Modlin JF, White JA, DeGirolami PC. 1982. A controlled comparison of joint reactions among women receiving one of two rubella vaccines. American Journal of Epidemiology 115:19-25. Pollock TM, Miller E, Mortimer JY, Smith G. 1984. Symptoms after primary immunisation with DTP and with DT vaccine. Lancet 2:146-149. Preblud SR. 1985. Some current issues relating to rubella vaccine. Journal of the American Medical Association 254:253-256. Prinzie A, Huygelen C, Gold J, Farquhar J, McKee J. 1969. Experimental live attenuated rubella virus vaccine: clinical evaluation of Cendehill strain. American Journal of Diseases of Children 118:172-177. Ross EM. 1988. Reactions to whole-cell pertussis vaccines. In: Wardlaw AC, Parton R, eds. Pathogenesis and Immunity in Pertussis. New York: John Wiley & Sons. Rothman KJ. 1986. Modern Epidemiology. Boston: Little, Brown & Co. Shields WD, Nielsen C, Buch D, Jacobsen V, Christenson P, Zachau-Christiansen B, Cherry JD. 1988. Relationship of pertussis immunization to the onset of neurologic disorders: a retrospective epidemiologic study. Journal of Pediatrics 113:801-805. Tingle AJ, Ford DK, Price GE, Kettyls DWG. 1979. Prolonged arthritis in identical twins after rubella immunization (brief report). Annals of Internal Medicine 90:203-204. Tingle AJ, Chantler JK, Pot KH, Paty DW, Ford DK. 1985. Postpartum rubella immunization: association with development of prolonged arthritis, neurological sequelae, and chronic rubella viremia. Journal of Infectious Diseases 152:606-612. Tingle AJ, Allen M, Petty RE, Kettyls GD, Chantler JK. 1986. Rubella-associated arthritis. I. Comparative study of joint manifestations associated with natural rubella infection and RA 27/3 rubella immunisation. Annals of the Rheumatic Diseases 45:110-114.