Cover Image

HARDBACK
$59.95



View/Hide Left Panel

4
Categories of Scientific Evidence—Human Information and Data

GUIDING PRINCIPLES: A credible report or study finding of a serious adverse event (or experience)1 in humans that is associated with use of a dietary supplement ingredient raises concern about the ingredient’s safety and requires further information gathering and evaluation. A final judgment, however, will require consideration of the totality of the evidence. In considering the evidence, historical use should not be used as prima facie evidence that the ingredient does not cause harm. It may be appropriate, however, to give considerable weight to a lack of adverse events in large, high-quality, randomized clinical trials or epidemiological studies that are adequately powered and designed to detect adverse effects, including those adverse effects with established serious risks for human morbidity or mortality and that are known to rarely occur de novo in the population.

1  

“Serious Adverse Experience. Any adverse experience occurring at any dose that results in any of the following outcomes: Death, a life-threatening adverse experience, inpatient hospitalization or prolongation of existing hospitalization, a persistent or significant disability/ incapacity, or a congenital anomaly/birth defect. Important medical events that may not result in death, be life-threatening, or require hospitalization may be considered a serious adverse experience when, based upon appropriate medical judgment, they may jeopardize the patient or subject and may require medical or surgical intervention to prevent one of the outcomes listed in this definition. Examples of such medical events include allergic broncho



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety 4 Categories of Scientific Evidence—Human Information and Data GUIDING PRINCIPLES: A credible report or study finding of a serious adverse event (or experience)1 in humans that is associated with use of a dietary supplement ingredient raises concern about the ingredient’s safety and requires further information gathering and evaluation. A final judgment, however, will require consideration of the totality of the evidence. In considering the evidence, historical use should not be used as prima facie evidence that the ingredient does not cause harm. It may be appropriate, however, to give considerable weight to a lack of adverse events in large, high-quality, randomized clinical trials or epidemiological studies that are adequately powered and designed to detect adverse effects, including those adverse effects with established serious risks for human morbidity or mortality and that are known to rarely occur de novo in the population. 1   “Serious Adverse Experience. Any adverse experience occurring at any dose that results in any of the following outcomes: Death, a life-threatening adverse experience, inpatient hospitalization or prolongation of existing hospitalization, a persistent or significant disability/ incapacity, or a congenital anomaly/birth defect. Important medical events that may not result in death, be life-threatening, or require hospitalization may be considered a serious adverse experience when, based upon appropriate medical judgment, they may jeopardize the patient or subject and may require medical or surgical intervention to prevent one of the outcomes listed in this definition. Examples of such medical events include allergic broncho

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety Information about human use of dietary supplement ingredients may be in the form of formal studies, such as clinical studies or trials and epidemiological studies; in the form of spontaneously reported adverse event reports and literature case reports; or in the form of information about historical use of the ingredient. Because dietary supplements are not required to undergo formal studies before marketing, formal study data available on dietary supplements are less commonly available than adverse event reports or information about historical use. The different types of data about human use can be useful either as (1) an indicator of possible risk or (2) a mitigator of concerns raised by other data. For example, spontaneous reports are generally used to detect concerns, and historical use information is often presented as a mitigator of concern. Formal studies are less likely to be available, but if they are, they can be the source of information about adverse events in individuals, or they can be used to demonstrate an overall increase in risk of a particular adverse event associated with ingestion of an ingredient. Formal studies can also be used as mitigators of concern if they are adequately designed and powered to detect adverse events. Within each type of human data, questions can be asked about the nature and quality of the scientific information to determine whether the information raises the level of concern regarding the probability to cause harm. In the sections that follow, the nature of evidence that increases concern is described and illustrated in a spectrum of concern figure. Information that independently raises a higher level of concern requires immediate attention to evaluate the potential of the ingredient to cause harm. For observations categorized as lower to moderate concern based on their placement on the spectrum, it is important to consider whether other types of human, animal, in vitro, or related data, as well as information about potential interactions (see Chapter 8), add to the level of concern. For many dietary supplement ingredients, human data regarding their safety or risk will not be available. A lack of data should not be interpreted as an absence of risk. Other types of data must be examined and weighed appropriately to understand the risk. SPONTANEOUSLY REPORTED ADVERSE EVENTS Adverse events associated with product use—whether reported directly to the Food and Drug Administration (FDA), manufacturers, distributors,     spasm requiring intensive treatment in an emergency room or at home, blood dyscrasias or convulsions that do not result in inpatient hospitalization, or the development of drug dependency or drug abuse” (21 C.F.R. § 600.80 [2002] and 21 C.F.R. § 314.80 [2002]).

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety or poison control centers—constitute important sources of safety information, along with published case reports and case series about adverse events. All unsolicited reports from health professionals or consumers received by FDA via either the voluntary or mandatory route are called spontaneous reports, so classified because they are clinical observations that originate outside of a formal study (Faich, 1986). The large-scale regulatory agency safety databases are composed of adverse event information generated by reporting from all sources, including scientific literature case reports and case series; each type of product (e.g., drug, biologic, device, and dietary supplement) is represented. While different products have aspects unique to their specific type, the principles of postmarketing safety monitoring apply to all; thus, what has been, and what will be, learned from one product realm is generally globally applicable. Dietary supplements, many of which contain biologically active ingredients, are no exception; that the guiding scientific principles for postmarketing safety surveillance have predominantly resulted from experience with pharmaceuticals (drugs and biologics) in no way invalidates their applicability to other substances, such as dietary supplement ingredients. However, while the underlying principles for safety monitoring are globally valid, the regulatory situation of dietary supplements impacts the use of their associated adverse event reports in several ways. Unlike drugs, biologics, and medical devices, dietary supplements do not undergo premarketing evaluation for safety and efficacy by FDA, and a formal benefit/risk assessment is not performed as part of an approval process. There is no evaluation of product quality (including purity, content uniformity, and stability) prior to marketing, and there is no requirement for manufacturers to collect or report adverse events to FDA. Yet, at the same time, the threshold for concern or action for a dietary supplement is lower than for regulated medical products, as under the applicable law they are considered to be similar to foods. In this section, the strengths and limitations of using adverse event reports for medical products in general are described, as well as how significant differences in laws and regulations impact the evaluation of adverse event reports associated with dietary supplement use. Strengths and Limitations The limitations of spontaneous adverse event reporting systems are well recognized, and include subjectivity and imprecision of adverse event recognition (Karch et al., 1976; Koch-Weser et al., 1977), underreporting (Chen et al., 1994; Chyka and McCommon, 2000; Rawlins, 1995), reporting biases (Sachs and Bortnichak, 1986), lack of precise exposure data (Begaud et al., 1994), and variability in report quality (Goldman, 1998). In

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety particular, the lack of precise numerator (number of cases) and denominator (number of patients exposed) data render the computation of incidence rates from spontaneous reports problematic (Begaud et al., 1994), if not totally unfeasible. In addition, as spontaneous reports originate under conditions of everyday use rather than under study conditions, there are possible confounding factors to be considered when evaluating reports, such as multiple concomitant medications (prescribed or over-the-counter [OTC]), multiple concomitant dietary supplements, concomitant medical devices, underlying disease states, or alcohol use. At the same time, these systems entail considerable concomitant strengths. Large-scale and relatively inexpensive (Fletcher, 1991), spontaneous adverse event reporting systems serve as the basis for safety-related hypothesis generation (Strom and Tugwell, 1990) and foster suspicions (Finney, 1971) that generate signals of potential problems warranting further study, while enabling individuals (health professionals and consumers alike) to contribute to public health (Goldman, 1998). This sentinel signaling function is critical, and the appropriateness of using a spontaneous reporting system in this regard is well documented and scientifically accepted (Blum et al., 1994; Goldman, 1996; Rossi and Knapp, 1984). The cases spontaneously reported to any surveillance program will generally represent only a small percentage of the number that have actually occurred.2 However, if the submitted reports are of high quality, irrespective of number, the effect of underreporting can be somewhat mitigated (Goldman, 1998). In the particular case of dietary supplements, a recent FDA-commissioned study estimated that FDA receives reports on less than 1 percent of all adverse events associated with their use (Walker, 2000). (With the majority of dietary supplement adverse event reporting to FDA done by consumers rather than by health professionals, it is possible that consumers might be less likely to associate dietary supplements with untoward effects, as opposed to making such attribution with a drug product, either prescribed or OTC). In addition, it has been found that consumers often do not inform their physicians about their use of dietary supplements (Eisenberg et al., 1998). Hence, reporting by physicians and other health professionals of adverse events on these products may well be minimal. 2   Concerning the extent of underreporting of drug-associated adverse events, a study performed in the United States before establishment of the FDA MedWatch adverse event reporting program (Kessler, 1993) estimated that the FDA received by direct report less than 1 percent of suspected serious adverse drug reactions (Scott et al., 1987), while rarely more than 10 percent of serious adverse drug reactions were estimated to be reported to the British spontaneous reporting program (Rawlins, 1995). This latter figure of 10 percent is consistent with other studies (Skjeldestad et al., 2000). It is not clear that these relative percentages hold for every serious adverse event and associated drug.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety Challenges Particular to Dietary Supplements Scientifically, the use of adverse event reports to assess the safety of dietary supplements should be very similar to how the safety of drug and other medical products are assessed, but the unique regulatory situation of dietary supplements provides some additional challenges. Beyond the limitations inherent in postmarketing surveillance systems, this unique regulatory environment renders the assessment of dietary supplement adverse event reports of greater complexity versus that performed on reports regarding drugs, biologics, or medical devices. Because premarketing safety studies are not required for dietary supplements, standard drug premarketing data, such as clinical pharmacology studies, are not likely to be available. As a result, significant clinical information such as how the product is absorbed, metabolized, and excreted is generally not available, nor is the product evaluated for possible interactions with foods, drugs, biologics, or devices. The lack of such clinical information generally does not allow specific populations at possible increased risk for adverse effects (such as children, the elderly, or those with renal and/or hepatic dysfunction) to be identified in a systematic manner. In further important distinction from medical products, there are no current FDA regulations establishing a baseline mandatory standard for dietary supplement manufacturing (CFSAN, 2001). As a result, the situation of multiple manufacturers of a specific dietary supplement can result in significant variation from product to product. An illustrative example is a study of the St. John’s wort products available in Germany, which detailed wide differences among the content of hypericin and hyperforin products, and notable interbatch variability in some of the products (Wurglics et al., 2001). Thus, unlike the situation with prescription drugs, in which there is standardized quality control of the innovator product and generic versions in manufacturing, adverse event reports on a particular dietary supplement may entail several different products that can vary significantly in dietary ingredient concentration, both among and within individual products. FDA’s proposed rule to establish current good manufacturing practices for manufacturers, with respect to both the production and labeling of dietary supplements (FDA, 2003a), addresses some manufacturing issues, but not all aspects of product-to-product variability. According to FDA, this proposed rule “would, for the first time, establish standards to ensure that dietary supplements and dietary ingredients are not adulterated with contaminants or impurities, and are labeled to accurately to reflect the active ingredients and other ingredients in the product” (FDA, 2003b). The challenge of underreporting adverse events was described above as a general challenge in using spontaneous adverse event reports. A factor particular to dietary supplements, however, is that, unlike drugs, biologics,

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety and medical devices, dietary supplement manufacturers and distributors are not required to disclose to FDA the adverse event reports that they receive (CFSAN, 2001; OIG, 2001). As a result, an important established source of reports for medical products is not duplicated for dietary supplements, and the evolution of a concomitant culture of adverse event reporting among manufacturers and distributors is not encouraged. The lack of manufacturer disclosure requirements is especially important given that consumers may be less likely to report adverse reactions to practitioners, a usual source of adverse event reports. A study sought to determine whether botanical remedy users would report adverse reactions to such products differently from similar adverse reactions experienced with the use of OTC medications (Barnes et al., 1998). While approximately 30 percent would consult their general practitioner irrespective of which type of ingested product was being used, and another 43 percent would not consult in either case, 26 percent would consult their general practitioner for a serious OTC-associated adverse reaction, but not for a similar adverse reaction associated with use of an herbal remedy.3 In summary, adverse event report assessment of dietary supplement ingredients is of heightened complexity and ambiguity compared with that of medical products. Using Spontaneous Reports Assessing the Strength of Association Between Event and Product Spontaneous reports entail an assumed association between the denoted adverse event and product in question, but careful evaluation of accumulated cases is needed to assess the actual strength of the association. Achieving certain proof of causality is not necessary to determine that an unreasonable or significant risk exists, especially if other types of data support the same conclusion. Regarding numbers of cases needed for such assessment, when the medical product-adverse event relationship is stronger and the incidence of the adverse event occurring de novo/naturalistically is lower (i.e., the event 3   Of further significance is the international variation with respect to consumer reporting. The European Union regulatory agencies do not routinely accept drug adverse event reports from consumers that are not confirmed by a health professional, while few countries (the United States and Canada being notable exceptions) require pharmaceutical companies to submit consumer reports that they receive (CIOMS, 2001). In addition, while U.S. consumers are encouraged to report adverse events directly to FDA (FDA, 2003c), it is conceivable that the lack of encouragement for consumer reporting in other countries might negatively affect accumulation of dietary supplement-associated adverse event data on a global basis.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety is rarer), fewer case reports are needed to perceive causality (Auriche and Loupi, 1993). For rare4 serious adverse events, such as toxic epidermal necrolysis, coincidental medical product-event associations have been found to be so unlikely that they merit little concern in spontaneous reporting, with more than three reports seen to represent a signal necessitating further study (Begaud et al., 1994). Further, it has been suggested that the combination of a temporal relationship between medical product and adverse event, positive dechallenge, and rechallenge can make individual reports conclusive as to product-event association (Temple et al., 1979). Notwithstanding, there is no definitive number of cases for generating a signal of safety concern. It is dependent on the characteristics of the individual adverse event itself, including such factors as clinical manifestations and severity, potential for significant morbidity (reversible and irreversible) or mortality, potential populations at risk, and the overall risk to public health. In judging any association to be causal, biological plausibility and reasonable strength of association are useful (Rawlins, 1995). However, given the outlined limitations of spontaneous reports, achieving certain proof of causality through postmarketing surveillance is unusual (Auriche and Loupi, 1993). Attaining a prominent degree of suspicion is much more likely, and, given that it may be deemed an ample basis for regulatory decisions on drugs (Auriche and Loupi, 1993), it is an ample basis for regulatory decisions on dietary supplement ingredients. Regarding factors that can be useful in assessing the strength of association between any medical product and a reported adverse event, international consensus produced the following list (CIOMS, 1990): The chronology of administration of agent, including beginning and ending of treatment and adverse event onset, The course of the adverse event when the suspected agent stopped (dechallenge) or continued, The etiologic roles of agents and diseases in regard to an adverse event, Response to readministration (rechallenge) of the agent, and Laboratory test results. 4   The United States has no regulatory definition that explicitly delineates events as common, infrequent, or rare based on their frequency of occurrence; the Council of International Organizations of Medical Sciences (CIOMS) III/V working groups have recommended the following standard categories of frequency: common (frequent): > 1/100 and < 1/10 (> 1 and < 10 percent); uncommon (infrequent): > 1/1,000 and < 1/100 (> 0.1 percent and < 1 percent); rare: > 1/10,000 and < 1/,1000 (> 0.01 and < 0.1 percent) (CIOMS, 1999).

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety Considering the Nature of the Adverse Event Beyond the factors listed above, the nature of the adverse event itself is an important consideration in assessing spontaneous reports. The nature of the adverse event encompasses knowledge about its naturalistic frequency of occurrence and its potential for significant morbidity or mortality. As previously discussed, premarket clinical trials in humans have inherent limitations that significantly affect their ability to detect adverse events (Goldman et al., 1995); to have a 95 percent chance of detecting an adverse event that occurs in 1 in 1,000 people, 3,000 people must be exposed (Lewis, 1981). For adverse events that occur de novo even more rarely, such as 1 in 100,000, 300,000 people must be exposed for there to be a good chance of detection. Many of the most serious adverse events occur relatively infrequently; that is why spontaneous report systems, which are designed to cover entire populations, are able to detect rare, serious events not discovered during premarket testing. This important outcome of spontaneous report systems for dietary supplement ingredients is the same as for other products. There are serious adverse events that by their very nature necessitate increased attention and scrutiny due to their potential for significant morbidity and/or mortality. FDA, in its recently published Proposed Rule for Safety Reporting Requirements for Human Drug and Biological Products, specifically addressed this issue by proposing a new designation, “Always Expedited Reports” for specific suspected adverse drug reactions (SADRs) of medical significance (FDA, 2003c). The following SADRs,5 regardless of expectedness, would be subject to expedited reporting due to their very nature: “congenital anomalies, acute respiratory failure, ventricular fibrillation, torsades de pointe, malignant hypertension, seizure, agranulocytosis, aplastic anemia, toxic epidermal necrolysis, liver necrosis, acute liver failure, anaphylaxis, acute renal failure, sclerosing syndromes, pulmonary hypertension, pulmonary fibrosis, confirmed or suspected transmission of an infectious agent by a marketed drug or biological product, confirmed or suspected endotoxin shock” (FDA, 2003c). Reporting of any of these adverse events in association with dietary supplement use should trigger the same degree of concern as for any other product, and the same heightened need for a timely assessment. 5   Also subjected to expedited reporting within this new designation are any other medically significant suspected adverse drug reactions that FDA determines to be the subject of an always expedited report (i.e., may jeopardize the patient and/or require medical or surgical intervention to treat the patient).

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety Considering the Amount Ingested Some reports of adverse events are based on intakes that exceed the amount specified on the particular dietary supplement label for “intended use.” A number of factors make it difficult, if not impossible, to determine the contribution of dose to the effects reported. The lack of adequate premarketing data to establish the validity of labeled dosing for many dietary supplements is coupled with significant variability in the amount of the particular dietary supplement constituents in different preparations. Known pharmacokinetic variability among individuals also makes it difficult, if not impossible, to determine the contribution of dose to the effects reported. However, in conclusion, if an adverse event report provides credible evidence that a serious adverse event is associated with a dietary supplement ingredient (see criteria in the earlier section), its utility in establishing a level of concern should not be discounted simply because the intake (resulting in adverse events) exceeds that specified in current dietary supplement labeling. Summary of Spontaneous Report Use In summary, higher concern is warranted in situations where one or more well-documented serious adverse events manifests positive temporality, and other factors (e.g., positive dechallenge, biological plausibility, or laboratory results) combine to strengthen the perceived association between the dietary supplement ingested and the adverse event in question. Given the inherent limitations of spontaneous reports (including report quality) in general, and those concerning dietary supplement ingredients in particular, not all of this information will be available in many cases. Tables 4-1 and 4-2 illustrate the relative spectra of concern. The columns on the right describe situations that warrant higher concern because the greatest risk to public health exists, while situations described on the left are of lower concern. The level of concern increases in proportion to the completeness of information provided and the likelihood of confounding decreasing. However, with serious, unexpected adverse events, possible confounding in the associated reports should not automatically lessen the level of concern, but rather heighten the attempt to obtain more reports of the highest possible quality to maximize the signaling function of spontaneous report systems. In addition, the nature of the serious adverse event itself and its potential for significant harm should stimulate appropriate heightened concern. The great utility of spontaneous reports lies in the generation of hypotheses about relationships between supplement ingredients and untoward effects, thus highlighting potential problems and signals that the agency may want to explore in greater depth. Evaluation of signals from

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety TABLE 4-1 Relative Spectrum of Concern for Individual Spontaneous Adverse Event Reports Increasing Concern Describes a serious adverse event with less information than would justify moderate or strong concern, and/or with prominent confounding factors (e.g., multiple concomitant substances and/or conditions) Describes a serious adverse event with some, but not all, characteristics associated with strong concern Describes a well-documented serious adverse event with plasma levels (if available) at a relevant range and demonstrates dechallenge and rechallenge (if possible), temporality, and strong attribution TABLE 4-2 Relative Spectrum of Concern for Case Series of Spontaneous Adverse Event Reports Increasing Concern Describes a series of serious adverse events, with less information than would justify moderate or strong concern, and/or prominent confounding factors (e.g., multiple concomitant substances and/or conditions) Describes a series of serious adverse events, with some, but not all, characteristics associated with strong concern Describes a series of well-documented cases demonstrating consistent serious adverse events and clinical findings, and dechallenge (if possible), temporality, and strong attribution spontaneous reports should entail use of supplementary information available about the substance, such as animal data, in vitro data, epidemiological studies, or clinical trials in which formal hypothesis testing can occur. Such studies can be used to further evaluate the strength of the association between the adverse event and dietary supplement in question because, as described in the previous paragraphs, establishing a definitive causal relationship solely through use of spontaneous reports is rarely possible. However, as noted, regulatory decisions that directly involve the safety of the public’s health do not necessitate definitive proof. In the specific case of dietary supplements, the threshold for action is clearly stated in the Dietary Supplement and Health Education Act: demonstration of a “significant or unreasonable risk of illness or injury.” Given the state of the art in adverse event report evaluation, pharmaco-

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety vigilance, and risk management, and given the potential risk to the public entailed by serious adverse events, regulatory action can be justified on the basis of adverse event report analysis alone or as the predominant source of information. There is ample precedent for this approach from the realms of regulated medical products, such as drugs, biologics, and medical devices. Dietary supplements, as agents with biological activity, should be no exception. HISTORICAL USE Experience from generations of use by humans is often referred to as evidence of safety for modern dietary supplements that bear resemblance to substances used historically. Some botanicals, for example, have a long history of medicinal use in many cultures. It may be useful to consider that there is both ancient (thousands of years) and recent history (perhaps the last 100 years). Ancient history may include traditional Chinese, Ayurvedic, and Native American medicines. Information about the preparation and use of ancient remedies is more difficult to locate and verify. Information about historical use is of less importance when relevant clinical, epidemiological, or animal toxicity data exist because if these types of data document harm, then this outweighs historical use that may show no harm. However, for many dietary supplement ingredients, the amount of scientific and experimental data are insufficient for a critical analysis of safety. Recognizing that a full range of data are unlikely to be available for many dietary supplement ingredients, historical use may be taken into account as a surrogate measure for safety in the absence of relevant scientific and experimental data. In doing so, it is important to consider the relevance of the traditional use to the current use and, as such, FDA must have information regarding both the traditional use and the current use to determine if the traditional use sheds any light on the potential risk associated with current use. Identifying Historical Uses of a Related Substance A starting place for obtaining information regarding traditional use of dietary supplement ingredients is secondary references. If secondary references suggest that the traditional use is similar to the current use and that the historical use has been without observed complications, then it is important to verify this information with primary sources before relying on it as even a weak surrogate indicator of safety. The quality of the original source of information should be considered before placing much value in secondary sources, as questionable information appears to be cited repeatedly, with problems in obtaining original information.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety TABLE 4-4 Spectrum of Relative Concerns with Clinical Studies Data Increasing Concern Describes a serious adverse event, but with less information than would justify moderate or strong concern, and/or the interpretation of the clinical study is hampered by the presence of prominent confounding factors (e.g., multiple concomitant substances and/or conditions) that could not be controlled by balancing AND/OR Prominent methodological concerns (e.g., unexplained high level of dropouts, lack of control groups) Nonsignificant, but clinically important, trend of a higher rate of a serious adverse event OR Abnormalities in clinical laboratory values OR Other abnormalities, such as electrocardiographic findings in the dietary supplement ingredient group A significantly higher incidence of a serious adverse event OR Other potentially dangerous abnormalities, such as in clinical laboratory values that are associated with risk of serious adverse events OR Other abnormalities, such as electrocardiographic findings in the dietary supplement ingredient group may occur if the study is small, too few adverse events were observed, or the study is otherwise underpowered. For example, the appearance of liver enzyme abnormalities in two or three subjects taking a dietary supplement may not produce a statistically significant difference compared with the control group. Therefore, it cannot be concluded with confidence that the liver abnormality is due to the dietary supplement, but the information can be used to augment other data related to safety of the dietary supplement ingredient. However, in populations and settings where no adverse event is expected, a nonsignificant trend also warrants attention. Finally, a concern still exists even if there is not a trend that would justify a moderate or statistically significant concern. This often occurs when a firm conclusion cannot be reached due to confounding factors. A frequent confounding factor with dietary supplements, for example, is the concomitant consumption of other xenobiotics. Single cases of serious adverse events, such as death or liver failure in the dietary supplement group, also warrant special attention.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety EPIDEMIOLOGICAL STUDIES6 Value of Epidemiological Studies Epidemiological studies that contain information on the use of dietary supplements, when available, are valuable sources of information for evaluating their safety. Data from these studies complement information from RCTs and adverse events reports. This type of data is rare; however, this is likely to change as additional research is conducted on dietary supplement use, efficacy, and safety. The considerations described below will be helpful in considering epidemiological studies to assess unreasonable or significant risk. As discussed above, among the limitations inherent to many RCTs is that relatively small size and short duration limit sensitivity to detect adverse events (Goldman et al., 1995). Latent or delayed effects that occur long after exposure may not be detected. Information about these latent and infrequent effects often comes from epidemiological studies that retrospectively or prospectively examine the effects of ingested substances on large populations. Another benefit of epidemiological studies is that the number of individuals exposed to supplements is expanded to the general population. Data from these studies usually contain a large number of individuals compared with the number exposed in clinical trials. Like RCTs, the value of epidemiological studies also depends on the endpoints examined. For example, if a study evaluates the incidence of cancer, death, or liver damage but does not evaluate anemia, the study is unlikely to detect interference with iron absorption. The assessment of the level of concern regarding the safety of a dietary supplement ingredient is dependent on the quality of the study and analyses, the estimated risk or odds ratio, the clinical significance of the risk, and the statistical significance of the estimate. Situations in the right column of Table 4-5 indicate higher levels of concern. 6   “Observational” and “epidemiological” studies are referred to here as “epidemiological studies.” Thus the term here includes (1) investigations in which the researcher has little or no control over events, and the relationships between risk factors and outcome measures are studied without the intervention of the investigator (e.g., surveys), and (2) the study of the distribution and size of disease problems in human populations, in particular to identify etiological factors in the pathogenesis of disease and to provide the data essential for the management, evaluation, and planning of services for the prevention, control, and treatment of disease.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety TABLE 4-5 Spectrum of Relative Concerns with Epidemiological Data Increasing Concern Case-control or cohort study (including registries), with small,a but statistically significant, relative risk or odds ratio of a serious adverse event OR Large relative risk or odds ratio of a serious adverse event that is not statistically significant OR Poorly conducted studies with large or significant effects Case-control or cohort study (including registries), with moderate, statistically significant relative risk or odds ratio of a serious adverse event OR Moderate relative risk or odds ratio of a serious adverse event that is not statistically significant but that implies a trend Well-conducted case-control or cohort study (including registries), with large, statistically significant relative risk or odds ratio of a serious adverse event a Two or less is generally considered a weak association, and 3 or more is considered strong, but this is only a very general “rule of thumb” guidance and somewhat debatable. Using Epidemiological Data on Dietary Supplements Assessing causality from epidemiological data requires the specific study designs that are described here, for example, case-control and cohort studies. However, other types of epidemiological studies often have been used in combination with other study designs (e.g., RCTs, case reports) to draw or strengthen conclusions (GAO, 1992). For any type of epidemiological study, the quality of the study and the analyses depends on the quality of the data. Missing data or, in the case of surveys, poor participation may cause biased results. Quality also may be affected by a conflict of interest by an author or study sponsor. Safeguards should be in place to prevent biased reporting of study results or where a conflict of interest is present. Errors in design, data collection, and analyses can also lead to poor-quality studies. Possible flaws of epidemiological studies have been well described in the medical and health care literature (Altman, 1998; Gardner et al., 1986). Cohort Studies Cohort studies generally evaluate a group of individuals (either prospectively or retrospectively) and estimate incidence rates of an event in

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety exposed and unexposed individuals. From this information, the risk of the event occurring in the exposed group relative to the risk for the unexposed group (the relative risk) can be estimated. Unlike well-conducted RCTs, results from cohort studies can be influenced by selection bias and confounding—if they exist. Selection bias occurs when there are unmeasured factors that are related to the outcome of safety and also affect selection for use or non-use of a supplement. Confounding can occur when use of a dietary supplement is strongly correlated to other characteristics of individuals that also affect the safety outcome. These two potential study design problems should be considered when adjusting the level of concern warranted by cohort studies. Primarily due to their relative expense, greater need for dedicated personnel, and inability to detect rare, serious adverse events, cohort studies are likely to be used less frequently than case-control studies in a postmarketing environment. Expense of cohort studies decreases when computerized medical records can be used, and thus as more records include information about intake of dietary supplement ingredients, the availability of cohort studies for dietary supplements is likely to increase. Nonetheless, for identified rare, serious adverse events, case-control studies may be more common. Case-Control Epidemiological Studies Case-control epidemiological studies are uniquely useful at estimating the likelihood that an ingested substance causes an adverse event when the occurrence of the event is rare or occurs following a long latency period. In a case-control study, cases (persons with the event of interest) and controls (persons who do not have the event of interest) are identified. The exposure rates among cases and among controls are then estimated.7 Using the estimated exposure (dose is rarely known) in the controls and in the cases, the odds of the event in the exposed group relative to that in the unexposed group (the odds ratio) is estimated. (The odds of an event are equal to the probability of the event divided by one minus this probability). Case-control studies require information on fewer individuals than cohort studies (see next section). However, in case-control studies there is the potential for bias caused by inappropriate selection of the control group or 7   When used in epidemiology, the word “exposure” often has a different meaning than in the field of toxicology. It is unusual for an epidemiological study to state exposure in the quantitative terms of dose. Rather, there may be some evidence, direct or indirect, that exposure to an agent of concern occurred without regard to consideration of the actual ingested dose. In toxicology studies, dosages are explicitly stated.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety inaccurate assignment of exposure status. Most textbooks on epidemiology cover this topic in detail (e.g., Rothman and Greenland, 1998). If exposure to a dietary supplement is widespread, case-control studies could be useful for assessing the association of the supplement with an adverse event—even if the adverse event is rare. However, a case-control study also will be useful when assessing risk from a dietary supplement even when use is not widespread if the adverse event rate among users is high. Both situations—wide exposure and a rare outcome and limited exposure with a common outcome—make case-control studies a useful tool for assessing safety of dietary supplements. Clinical Significance of Estimated Relative Risk or Odds Ratios from Cohort and Case-Control Studies: Using the Statistics Values Relative risk is a measure of the association between the exposure to some factor (in this case, a dietary supplement ingredient) and the risk of some outcome (e.g., a serious adverse event). It is calculated as the incidence rate of a serious adverse event among persons taking the dietary supplement ingredient divided by the incidence rate of the serious adverse event among persons not taking the ingredient. An incidence rate is the ratio of the number of events (e.g., of a serious adverse event) over a period of time and the number in the population being studied during the time period. For example, relative risk of 2.5 means that the group exposed to the ingredient is 2.5 times (or 150 percent) more likely to have a particular serious adverse event than those not exposed to the ingredient. A relative risk of 1.0 shows no additional risk in the exposed group while a relative risk < 1.0 indicates less risk in the exposed group. For example, a relative risk of 0.67 means the exposed group has 0.67 times the risk (two-thirds the risk or 33 percent less risk) of the event than does the unexposed group. An odds ratio is approximately equal to the relative risk when the probability of the adverse event is small in both the exposed and unexposed groups. Therefore, odds ratios often are described and interpreted as if they were relative risk values. If case-control or cohort studies on a dietary supplement ingredient have been completed, the reviewer should consider the magnitude of the odds ratio or relative risk values, p values, or confidence intervals (see below), and the seriousness and severity of the adverse event in question when determining the relative concern about the safety of a dietary supplement ingredient. In general, the reviewer should be alert to relative risks or odds ratios of greater than 2, as described in Table 4-5. That said, a numerical cutoff is not appropriate and greater sensitivity to low values may be appropriate as the seriousness of the adverse event and the number of individuals exposed increases. As a “rule of thumb,” a relative risk or odds

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety ratio of 3 or more generally represents a strong association, although choice of this cut-point is debatable (Stolley, 1990; Temple, 1999). Similarly, as a rule of thumb, a relative risk or odds ratio of 2 or less generally represents a weak association (Temple, 1999). In addition to the relative risk or odds ratio values, a measurement of uncertainty (e.g., standard error) is needed to properly assess the level of concern that should be attached to a finding. Confidence intervals and p values use both the estimated relative risk or odds ratio and the standard error of these estimates in their calculations. In most medical, biological, and health services literature, a p value (the probability of the observed data if the null hypothesis is true) of 0.05 or less is considered statistically significant. This cut-point translates into incorrectly rejecting the null hypothesis 1 time or fewer out of 20, on average. Although a p value of ≤ 0.05 is commonly used to determine when a result warrants attention, in interpreting studies regarding safety, a cutoff point of p < 0.05 is often not appropriate because of its implications. Dietary supplements are regulated similarly to foods and are presumed to be safe (the null hypothesis). A p value under this null hypothesis reflects the probability of the observed data when assuming that the dietary supplement is safe. A p value of 0.05 means that on average, one would incorrectly reject the assumption that a supplement is safe 1 time out of 20. For serious adverse events, when there is a high prevalence of use or when the supplement is used in special populations, a p value greater that 0.05 might raise the level of concern substantially. A p value of 0.10 would mean that the probability of the observed number of adverse events is 0.10 if one assumes that the supplement is safe (i.e., on average one expects to see this number of adverse events 1 time out of 10 if the supplement is safe). This type of finding could be enough to raise the concern level. Knowledge that at this level one incorrectly rejects the hypothesis of safety 1 out of 10 times on average should enable this information to be appropriately integrated with other types of information (e.g., animal data). Finally, confidence intervals of relative risks or odds ratios may be more useful than p values in interpreting results. A 95 percent confidence interval typically is used and means that 95 percent of the calculated confidence intervals are expected to contain the true relative risk if the estimation were repeated a large number of times in similar study settings. However, as with p values, the choice of the value 95 percent should be used with full understanding of its consequences and meaning. Studies where a 95 percent confidence interval for a relative risk or an odds ratio covers 1 (thus indicating lower significance statistically) may still offer important information about safety. A recent example where it was decided that there existed a high safety concern even without having statistical significance is

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety the hormone replacement therapy randomized control trial designed to look at benefits of the therapy over time (Rossouw et al., 2002). Assessing the Strength of Association with Epidemiological Studies Epidemiologists tend to agree on characteristics of epidemiological studies that suggest an increasingly strong association between an adverse effect and an ingested substance, describing this in terms of establishing “causality” between a substance and an effect (Hennekens et al., 1987; Hill, 1971; Rothman and Greenland, 1998; Sackett et al., 1991). While these characteristics can certainly be used to demonstrate whether the threshold of causality has been met, meeting such a threshold is less important for dietary supplements for which it is only necessary to determine whether an unreasonable or significant risk exists. For a dietary supplement ingredient studied in epidemiological studies, concern increases as more of the following criteria are met: large relative risk; consistency of findings in different studies or in different populations; association that “makes sense” because other plausible causes are ruled out and results are consistent with current knowledge of cause and effect in humans, animals, and cells in vitro; association that is limited to a single potential cause and a single type of adverse event;8 a dose-response relationship9 and temporality (i.e., the adverse event occurs after a dietary supplement is ingested). In the evaluation of safety of dietary supplements, one high-quality epidemiological study alone can cause a high level of concern, as shown on the right side of Table 4-5. Epidemiological studies that do not meet these criteria may be used to form hypotheses about safety or to strengthen or generalize the results from RCTs or other data. SUMMARY As described in the guiding principles, a credible report or study finding of a serious adverse event in humans that is associated with use of a dietary supplement ingredient raises concern about the ingredient’s safety. While historical use should not be used as prima facie evidence that the ingredient does not cause harm, it may be appropriate to give considerable weight to a lack of adverse events in large, high-quality, randomized clinical trials or 8   One example of this is thalidomide—no other drugs were associated with the particular birth defects and the type of birth defect was unique. 9   For example, people who smoke more cigarettes per day have a greater likelihood of disease.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety epidemiological studies that are adequately powered and designed to detect adverse effects. The basis for this guiding principle is described in the sections of this chapter, as are the following important corollaries and specific guidance: There are significant limitations in using clinical efficacy trials to predict that an adverse event will not occur because of their limited sensitivity. Clinical trials do provide valuable information when adverse events are detected. A statistically significant increased rate in adverse events indicates that a sufficiently large number of events have occurred to allow one to conclude that the observation is unlikely due to chance alone. However, in populations and settings where no adverse event is expected, a nonsignificant trend also warrants some concern and consideration. Epidemiological studies that contain information on the use of dietary supplements, when available, are valuable sources of information for evaluating their safety. Given the state of the art in adverse event report evaluation and pharmacovigilance and risk management, and given the potential risk to the public entailed by serious adverse events, regulatory action can be justified on the basis of spontaneously reported adverse event report analysis alone or as the predominant source of information. Reports of certain adverse events warrant heightened concern because they have a known potential for significant morbidity (and in some cases, mortality). If spontaneously reported adverse event reports are of high quality, irrespective of number, the effect of underreporting can be somewhat mitigated. The stronger the product-adverse event relationship and the lower the incidence of (and thus rarer) the adverse event occurring de novo/ naturalistically, the fewer the number of case reports that will be needed to perceive causality. Recognizing that a full range of data is unlikely to be available for many dietary supplement ingredients, historical use may be taken into account as a surrogate measure for safety in the absence of relevant scientific and experimental data. In doing so, it is important to consider the relevance of the traditional use to the current use and, as such, FDA must have information regarding both the traditional use and the current use to determine if the traditional use sheds any light on the potential risk associated with current use. The fact that a substance was consumed over a number of years does not indicate that it was consumed without adverse effects. Safe administration by non-oral routes of administration should not be taken as an indication of safety via oral administration. Historical information is useful only if the product in question is not so far removed from the original

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety substance as to constitute a distinct entity. If the current level of intake is significantly above what has been traditionally recommended, then the level of concern should be increased. REFERENCES Altman DG. 1998. Statistical reviewing for medical journals. Stat Med 17:2661–2674. Auriche M, Loupi E. 1993. Does proof of casualty ever exist in pharmacovigilance? Drug Safe 9:230–235. Bannerman RH, Burton J, Wen-Chieh C, eds. 1983. Traditional Medicine and Health Care Coverage. A Reader for Health Administrators and Practitioners. Geneva: World Health Organization. Barnes J, Mills SY, Abbot NC, Willoughby M, Ernst E. 1998. Different standards for reporting ADRs to herbal remedies and conventional OTC medicines: Face-to-face interview with 515 users of herbal remedies. Br J Clin Pharmacol 45:496–500. Begaud B, Moride Y, Tubert-Bitter P, Chaslerie A, Haramburu F. 1994. False-positives in spontaneous reporting: Should we worry about them? Br J Clin Pharmacol 38:401–404. Blum MD, Graham DJ, McCloskey CA. 1994. Temafloxacin syndrome: Review of 95 cases. Clin Infect Dis 18:946–950. CFSAN (Center for Food Safety and Applied Nutrition). 2001. Overview of Dietary Supplements. Online. Food and Drug Administration. Available at http://www.cfsan.fda.gov/~dms/ds-oview.html. Accessed February 22, 2002. Chadwick DJ, Marsh J, eds. 1990. Bioactive Compounds from Plants. Ciba Foundation Symposium 154. New York: John Wiley and Sons. Chen RT, Rastogi SC, Mullen JR, Hayes SW, Cochi SL, Donlon JA, Wassilak SG. 1994. The Vaccine Adverse Event Reporting System (VAERS). Vaccine 12:542–550. Chyka PA, McCommon SW. 2000. Reporting of adverse drug reactions by poison control centers in the US. Drug Safe 23:87–93. CIOMS (Council for International Organizations of Medical Sciences). 1990. Standardization of definitions and criteria of causality assessment of adverse drug reaction. Drug-induced liver disorders: Report of an International Consensus Meeting. Int J Clin Pharmacol 28:317–322. CIOMS. 1999. Guidelines for Preparing Core Clinical-Safety Information on Drugs, 2nd ed. Report of CIOMS Working Groups III and V. Geneva: CIOMS. CIOMS. 2001. Current Challenges in Pharmacolvigilance: Pragmatic Approaches. Report of CIOMS Working Group V. Geneva: CIOMS. Counsell C. 1997. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 127:380–387. Dobelis IN. 1997. Magic and Medicine of Plants. A Practical Guide to the Science, History, Folklore, and Everyday Uses of Medical Plants. Pleasantville, NY: Reader’s Digest Association. Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, Van Rompay M, Kessler RC. 1998. Trends in alternative medicine use in the United States, 1990–1997: Results of a follow-up national survey. J Am Med Assoc 280:1569–1575. Faich GA. 1986. Adverse-drug-reaction monitoring. N Engl J Med 314:1589–1592. Farnsworth NR. 2003. NAPRALERT (Natural Products ALERT) Database. Chicago: University of Chicago. FDA (Food and Drug Administration). 2003a. Current good manufacturing practice in manufacturing, packing, or holding dietary ingredients and dietary supplements. Proposed rule. Fed Regis 68:12157–12263.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety FDA. 2003b. FDA News. FDA Proposes Labeling and Manufacturing Standards for All Dietary Supplements. Online. Available at http://www.fda.gov/bbs/topics/NEWS/2003/NEW00876.html. Accessed July 2, 2003. FDA. 2003c. Safety reporting requirements for human drug and biological products. Proposed rule. Fed Regis 68:12406–12497. Finney DJ. 1971. Statistical aspects of monitoring for dangers in drug therapy. Methods Inf Med 10:1–8. Fletcher AP. 1991. Spontaneous adverse drug reaction reporting vs event monitoring: A comparison. J R Soc Med 84:341–344. GAO (General Accounting Office). 1992. Cross Design Synthesis: A New Strategy for Medical Effectiveness Research. GAO/PEMD-92-18. Washington, DC: GAO. Gardner MJ, Machin D, Campbell MJ. 1986. Use of check lists in assessing the statistical content of medical studies. Br Med J 292:810–812. Goldman SA. 1996. The Clinical Impact of Adverse Event Reporting. A MedWatch Continuing Education Article. Food and Drug Administration. Available at http://www.fda.gov/medwatch/articles.htm. Accessed February 25, 2003. Goldman SA. 1998. Limitations and strengths of spontaneous reports data. Clin Ther 20:C40–C44. Goldman SA, Kennedy DL, Lieberman R, eds. 1995. Clinical Therapeutics and the Recognition of Drug-Induced Disease. A MedWatch Continuing Education Article. Food and Drug Administration. Available at http://www.fda.gov/medwatch/articles.htm. Accessed June 4, 2003. Griggs B. 1997. Green Pharmacy. The History and Evolution of Western Herbal Medicine. 2nd ed. Rochester, VT: Healing Arts Press. Hennekens CH, Buring JE, Mayrent SL. 1987. Epidemiology in Medicine. Boston: Little, Brown. Hill AB. 1971. Principles of Medical Statistics. 9th ed. New York: Oxford University Press. ICH (International Conference on Harmonisation). 1995. Guidance for Industry. E3. Structure and Content of Clinical Study Reports. Rockville, MD: Drug Information Branch, Center for Drug Evaluation and Research, FDA. Ioannidis JP, Lau J. 2001. Completeness of safety reporting in randomized trials: An evaluation of 7 medical areas. J Am Med Assoc 285:437–443. Ioannidis JPA, Chew P, Lau J. 2002. Standardized retrieval of side effects data for metaanalysis of safety outcomes: A feasibility study in acute sinusitis. J Clin Epidemiol 55:619–626. Karch FE, Smith CL, Kerzner B, Mazzullo JM, Weintraub M, Lasagna L. 1976. Adverse drug reactions—A matter of opinion. Clin Pharmacol Ther 19:489–492. Kessler DA. 1993. Introducing MEDWatch: A new approach to reporting medication and device adverse effects and product problems. J Clin Res Drug Dev 7:183–190. Koch-Weser J, Sellers EM, Zacest R. 1977. The ambiguity of adverse drug reactions. Eur J Clin Pharmacol 11:75–78. Lewis JA. 1981. Post-marketing surveillance: How many patients? Trends Pharmacol Sci 2:93–94. Moher D, Schulz KF, Altman DG. 2001. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Med Res Methodol 1:2–8. OIG (Office of Inspector General). 2001. Adverse Event Reporting for Dietary Supplements. An Inadequate Safety Valve. Online. Food and Drug Administration. Available at http://oig.hhs.gov/oei/reports/oei-01-00-00180.pdf. Accessed February 26, 2003. Prance GT, Chadwick DJ, Marsh J, eds. 1994. Ethnobotany and the Search for New Drugs. Ciba Foundation Symposium 185. New York: John Wiley and Sons.

OCR for page 126
Dietary Supplements: A Framework for Evaluating Safety Rawlins MD. 1995. Pharmacovigilance: Paradise lost, regained or postponed? J R Coll Physicians Lond 29:41–49. Rossi AC, Knapp DE. 1984. Discovery of new adverse drug reactions. A review of the Food and Drug Administration’s spontaneous reporting system. J Am Med Assoc 252:1030–1033. Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SA, Howard BV, Johnson KC, Kotchen JM, Ockene J. 2002. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the Women’s Health Initiative randomized control trial. J Am Med Assoc 288:321–333. Rothman KJ, Greenland S. 1998. Modern Epidemiology. 2nd ed. Philadelphia, PA: Lippincott-Raven. Sachs RM, Bortnichak EA. 1986. An evaluation of spontaneous adverse drug reaction monitoring systems. Am J Med 81:49–55. Sackett DL, Haynes RB, Tugwell P. 1991. Deciding whether your treatment has done harm. In: Sackett DL, ed. Clinical Epidemiology: A Basic Science for Clinical Medicine. 2nd ed. Boston: Little Brown. Scott HD, Rosenbaum SE, Waters WJ, Colt AM, Andrews LG, Juergens JP, Faich GA. 1987. Rhode Island physicians’ recognition and reporting of adverse drug reactions. R I Med J 70:311–316. Skjeldestad FE, Amundsen T, Hoibraaten E. 2000. Reporting of adverse drug reactions to the Norwegian Medicines Control Agency—A discipline with few participants. Tidsskr Nor Laegeforen 120:336–338. Stolley PD. 1990. How to interpret studies of adverse drug reactions. Clin Pharmacol Ther 48:337–339. Strom BL, Tugwell P. 1990. Pharmacoepidemiology: Current status, prospects, and problems. Ann Intern Med 113:179–181. Tegtmeier M, Harnischfeger G. 1994. Methods for the reduction of thujone content in pharmaceutical preparations of Artemisia, Salvia and Thuja. Eur J Pharm Biopharm 40:337–340. Temple R. 1999. Meta-analysis and epidemiologic studies in drug development and postmarketing surveillance. J Am Med Assoc 281:841–844. Temple RJ, Jones JK, Crout JR. 1979. Adverse effects of newly marketed drugs. N Engl J Med 300:1046–1047. Walker A. 2000. The Relation between Voluntary Notification and Material Risk in Dietary Supplement Safety. FDA Commissioned Paper. FDA Docket 00N-1200(41). Wurglics M, Westerhoff K, Kaunzinger A, Wilke A, Baumeister A, Dressman J, Schubert-Zsilavecz M. 2001. Comparison of German St. John’s wort products according to hyperforin and total hypericin content. J Am Pharm Assoc 41:560–566.