6

Decision Making and Oversight of MRTP Studies: Findings and Recommendations

In the previous three chapters, the committee describes the evidence domains for potential modified risk tobacco products (MRTPs), including studies of health effects, addictive potential, and risk perception and communication. The committee discussed the governance of those studies in Chapter 2. The Food and Drug Administration (FDA) will have to integrate the evidence from those diverse domains when making regulatory decisions about MRTPs. Many of the same issues and concerns related to the governance and design of studies arise regardless of the evidence domain. The committee’s findings and recommendations, therefore, cut across the evidence domains and focus on the types of evidence and studies that the FDA should use in making regulatory decisions about MRTPs, the design of studies of MRTPs for the FDA’s decision making, and the governance of those studies. This chapter focuses on those cross-cutting issues, ways the FDA can integrate information from those studies as part of its regulatory decisions, and the committee’s overarching findings and recommendations.

This chapter begins with a brief summary of the evidence domains discussed in more detail in the previous chapters, a discussion of the scientific and ethical issues in studying MRTPs, and the governance issues that accompany such studies. The committee then discusses (1) the integration of all the evidence for the FDA’s regulatory decisions about these products, including mathematical modeling and simulation techniques, and (2) the comparative nature of the harm reduction claims that the FDA will be evaluating. The committee then presents its findings and recommendations.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 221
6 Decision Making and Oversight of MRTP Studies: Findings and Recommendations In the previous three chapters, the committee describes the evidence domains for potential modified risk tobacco products (MRTPs), includ- ing studies of health effects, addictive potential, and risk perception and communication. The committee discussed the governance of those studies in Chapter 2. The Food and Drug Administration (FDA) will have to inte- grate the evidence from those diverse domains when making regulatory decisions about MRTPs. Many of the same issues and concerns related to the governance and design of studies arise regardless of the evidence domain. The committee’s findings and recommendations, therefore, cut across the evidence domains and focus on the types of evidence and studies that the FDA should use in making regulatory decisions about MRTPs, the design of studies of MRTPs for the FDA’s decision making, and the governance of those studies. This chapter focuses on those cross- cutting issues, ways the FDA can integrate information from those studies as part of its regulatory decisions, and the committee’s overarching find - ings and recommendations. This chapter begins with a brief summary of the evidence domains discussed in more detail in the previous chapters, a discussion of the sci - entific and ethical issues in studying MRTPs, and the governance issues that accompany such studies. The committee then discusses (1) the inte - gration of all the evidence for the FDA’s regulatory decisions about these products, including mathematical modeling and simulation techniques, and (2) the comparative nature of the harm reduction claims that the FDA will be evaluating. The committee then presents its findings and recommendations. 221

OCR for page 221
222 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS ISSUES WITH EVIDENCE Evidence Domains The evidence to support the marketing authorization of an MRTP will come from studies of health effects, addictive potential, and risk percep - tion. In this section the committee provides a brief summary of the studies within those evidence domains. Health Effects Laboratory analysis of the performance and of the constituents of tobacco products will be the first step in the evaluation of any new product. These analyses involve standard methods of extraction, sample cleanup, analytic identification, and quantitation. There are important limitations to laboratory analysis of product performance and composition. Labora - tory analysis of constituents may not reflect constituent uptake under con- ditions of use. Smoking machines do not replicate human smoking condi- tions, and there is no proven way to replicate the many ways humans use tobacco. It is crucial, therefore, to describe the smoking regimen or other extraction methods employed. The second step in the evaluation of MRTPs will be preclinical studies of toxicity. These assays are essential in identifying particularly risky or toxic products that should not be tested in humans, as well as products that have reasonable potential to reduce risk and, therefore, should pro- ceed to clinical evaluation. Evaluation of products in vitro should precede in vivo assays. Although it is not possible to make laboratory animals use tobacco products the way humans do, and there are inherent interspecies differences that prevents meaningful extrapolation of human effects, it is still informative to observe the effect of tobacco products in live animal models. The number of animal studies required to characterize an MRTP preclinically could potentially be reduced by setting composition standards or limits or establishing standards for certain categories of MRTPs. Assays of toxicity in humans will also be essential, in particular assays of urinary mutagenicity and sister chromatid exchange in peripheral lymphocytes. Biomarkers of exposure measure human exposure to constituents of tobacco and could include the constituents themselves, their metabolites, or protein (or DNA) binding products of the constituents or their metabo- lites. These biomarkers have the potential to bypass many of the uncer- tainties in product composition analysis and provide a realistic and direct assessment of carcinogen and toxicant dose in an individual. Biomarkers of exposure should be validated before their use. When the FDA evaluates studies, it is important that it ensures the constituents of a product are accurately and precisely measured, that

OCR for page 221
223 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES exposure methods are appropriate, and that any biomarkers of exposure have been validated. Experimental designs, in particular randomized controlled trials (RCTs), provide data that can support the strong inferences about the effect of an MRTP on human health relative to conventional tobacco prod- ucts. The use of appropriately designed clinical trials will be important to establish whether the use of the MRTP reduces exposure to toxicants or induces positive changes in surrogate markers. An RCT is an effective means of examining acceptability and use of the MRTP, the ability of the MRTP to increase cessation in users of conventional tobacco products, and the likelihood that MRTP availability will lead to dual use. RCT methods can also produce evidence on whether and how much individuals use an MRTP after they have used it to help them quit conventional prod- ucts, changes in perception of the MRTP with its continued use, and the MRTP’s ability to suppress tobacco withdrawal symptoms. It is important to recognize that no single RCT can address all of these areas, and each study should have a focused objective with a primary endpoint. Postmarket studies of marketed MRTPs will be critical to evaluating the effect of the MRTP on both individuals and on the public’s health. In particular, the prospective cohort design will be an essential tool to vali - dating anticipated or claimed effects of marketed MRTPs. These studies can assess baseline tobacco and MRTP exposures; summarize product use as the study is ongoing, including any changes in product use habits; document and verify outcomes as they occur; and evaluate a wide variety of outcomes, including both intermediate and clinical outcomes. In addition, other study designs will be necessary to provide evidence on the public health effects of MRTPs, including retrospective cohort studies, case-control studies, crossover or case-crossover designs, and comparative effectiveness research methods. Different study designs will be necessary depending on the circumstances and the research question. Addictive Potential Evaluation of the likelihood of initiation, maintenance, and persis- tence of use in both conventional tobacco users and nonusers is critical to estimating the public health effect of marketing MRTPs. Specifically, evaluation of the MRTP’s ability to promote initiation and continuation of its regular use, switching to its use and cessation of the consumption of more harmful products, dual use, and to promote relapse back to more harmful tobacco use are all essential. All of these outcomes are related to the reinforcing value of the MRTP (that is, how rewarding it is). There is a continuum of reinforcement value. In theory, the MRTP should be somewhat more reinforcing than nicotine replacement thera -

OCR for page 221
224 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS pies but perhaps less reinforcing than conventional cigarettes. Ideally, an MRTP would be sufficiently reinforcing so as to attract smokers away from conventional cigarettes but not encourage the widespread depen - dent use of the product by individuals who were previously nonusers or who would have quit smoking. Data from all relevant measures should be integrated, taking into account not only the effects of the MRTP on important health outcomes, but also the prevalence of use and projected public health impact. Evaluation of the abuse and addiction potential of a product can be accomplished with a variety of experimental designs and in a variety of contexts. Those include subjective evaluations in laboratory contexts, acute self-administration studies in laboratory contexts, use in extended residence facilities, and natural environment contexts where long-term use can be studied in real-world circumstances via RCTs, cross-sectional survey studies, and longitudinal cohort studies. Evaluation of reinforcement value in a laboratory setting is particu- larly important because the results of these studies reliably correspond to an agent’s addictive potential in real-world use. One standard with regard to human abuse liability drug testing are acute dose-effect com - parison studies, because of the correspondence between subjective ratings of drug effects and real-world abuse potential. Behavioral economic self- administration studies will also be important to evaluating the reinforce- ment potency of a product. The usefulness of all studies in forecasting the risk for initiation and abuse of a product depends on study design factors. Important design considerations include the size of the sample, the nature of the sample (whether the sample includes heavy smokers or light smokers, smokers who want to quit, and nonsmokers), the character- ization of the sample (age, sex, gender, ethnicity, educational attainment, socioeconomic status, etc.), and the nature of the comparison product. Risk Perception and Communication Judgments about risk, otherwise known as risk perceptions, are a fundamental element to most theoretical models of health behavior and behavioral decision making. In general, those models argue that indi - viduals’ perceptions about the value and likelihood of behavior-related positive and negative consequences and their vulnerability to those con - sequences play a key role in behavioral choices. As such, understanding individuals’ perceptions of tobacco-related products, including MRTPs, whether such perceptions change over time, and whether such percep- tions play a role in tobacco use behavior, is critical. It will be important to identify consumers’ perceptions of disease risk, likelihood of addiction, likelihood of reducing or increasing others’ exposure to potentially haz -

OCR for page 221
225 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES ardous compounds, and perceptions of risk compared to other products already on the market. It is also important to assess intentions of using the product. It is essential that industry carefully crafts messages about risks and benefits of any MRTP and demonstrates through rigorous testing that people correctly understand and interpret the risks. Studies evaluating risk perceptions and risk communication must be performed both before the marketing of an MRTP and after the MRTP has been marketed. Premarket research will play an essential role in devel- oping the messages that the tobacco industry can use to communicate information about MRTPs to consumers. This research will determine consumers’ ability to accurately understand messages that communicate information about the risks, benefits, and conditions of use pertaining to the MRTP itself and compared to existing tobacco products. Studies should also test how these messages influence consumers’ perceptions of the risks, benefits, and likelihood of addiction related to the MRTP. The first stage of premarket research will involve formative work using focus groups. The second stage should include discussions with groups of similar individuals to assess how the messages that were developed in the first stage are received by consumers. Finally, the effects of these mes - sages on consumer perceptions should be tested. It will be important to evaluate consumer understanding and to compare consumer perceptions of the MRTP to conventional products. After the product is released on the market, it is vital to continue monitoring consumer perceptions and behavior related to that product. Conducting nationally representative cohort-sequential longitudinal surveys will be essential. Cross-Cutting Issues with Studies of MRTPs When evaluating the studies of MRTPs, there are a number of con- siderations that are relevant regardless of whether a study is looking at health effects, addictive potential, or risk perception. Those cross-cutting issues, which include issues related to study design and governance, are discussed in this section. Study Design Issues that could affect the FDA’s evaluation of a study on MRTPs include the generalizability of the study and how well the study is conducted. Studies should be designed appropriately to create an evidence base that can support a finding of public health benefit. The ultimate goal of studying the effect of MRTPs on human health and behavior is to be able to accurately predict the public health effects of allowing an MRTP to be

OCR for page 221
226 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS marketed. In other words, the ultimate goal of scientific studies it to pro - duce generalizable data. The “generalizability” of data, or the reliability of predictions that can be made about the real world based on scientific observations, will depend on the design of the studies. The FDA should carefully evaluate the size and nature of the sample to assess the generalizability of the study data. Sample sizes should be carefully determined and tailored to the study design and the effects studied. Statistically underpowered studies cannot support inferences or projections about the effects of a product. The nature of the study sample is critical to the usefulness of study results. Results from studies conducted in one population may not be applicable to other populations, because the characteristics that define the study population either are related to or cause the responses to the product. As such, it is important to study a wide range of populations. It is particularly important to include populations that have a high risk of using tobacco and populations that will be affected by the marketing of the product. Study designs must also carefully consider the degree of control imposed on experimental designs. Internal and external validity should be balanced not only within studies but also across studies of the same product. Highly controlled experimental designs can eliminate many vari- ables and confounders and support strong inferences, but simultaneously lose relevance to the FDA’s decisions because the conditions of product use do not reflect real-world circumstances and behaviors. Experimental designs that are less controlled can emulate circumstances that reflect real-world conditions and behaviors, and therefore may be more relevant in predicting real-world effects, but uncontrolled variables may confound meaningful associations or inferences. Regardless of the type of study design, the planning and conduct of the study should meet good research practice standards. As discussed earlier in this report, there are minimum standards for studies in other settings, such as studies of pharmaceuticals. Consensus statements such as Consolidated Standards of Reporting Trials (CONSORT) for clinical trials, Strengthening the Reporting of Observational Studies in Epidemi - ology (STROBE) for observational studies, publication criteria from the International Council of Medical Journal Editors, and reporting criteria of the International Conference on Harmonization have been implemented to ensure that the design, conduct, and reporting of studies are consistent with the state-of-the-art scientific standards. Studies of MRTPs should meet those or similar criteria to help ensure the overall quality and integ- rity of the studies. Certain types of studies will have specific criteria to ensure quality research. Some of those specific criteria were discussed in the preceding chapters.

OCR for page 221
227 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES Governance Research on MRTPs will require some oversight or governance to ensure the research is free from bias and conflicts of interest and that appropriate controls are in place for human subject research. Such gover- nance will also ensure the disclosure of data to ensure transparency and instill confidence in the research findings. The tobacco industry has a history of hiding and misrepresent- ing information about the risks of tobacco products (Cummings, 2003; Cummings et al., 2002, 2007).1 This history has lead to profound public distrust in both the tobacco industry and in the research it sponsors, and the absence of governance in the tobacco industry has created an iso- lated industry that lacks not only the expertise to produce the necessary range of credible and reliable data, but also the trustworthiness to acquire external expertise and avenues to disseminate acquired data (Ashley and Cohen, 2003; Harris Interactive, 2003, 2010; NCI, 2008). The production of reliable and credible data depends upon building rigor, oversight, and independence into the entire research process. Data problems often cannot be detected after study completion, and therefore integrity and accountability need to be built into the research throughout the study’s execution. There is neither an established set of regulatory practices for the review of MRTPs nor an established set of federal research standards for the design, conduct, analysis, monitoring, and completion of studies in support of MRTPs. There are also a number of ethical issues associated with conducting human subjects research involving MRTPs. The first issue is the risk of conducting clinical trials of MRTPs or other tobacco products in popu - lations with a high risk for tobacco initiation and addiction, including but not limited to adolescents, certain ethnic minorities, and individuals with mental health disorders. Randomization of participants to a product known to be potentially addictive and hazardous is ethically problematic. The second issue is the risk of improperly disclosing the substance abuse of a minor to the minor’s parents or guardians in the process of obtaining parental consent for research. While the assent of minors is always neces- sary, investigators should also be aware of circumstances where waiver of parental consent is warranted because obtaining parental consent will violate the confidence of the study participant. The third issue is the inclusion of individuals from high-risk groups with reduced decision- making capacity. Some populations at a high risk for tobacco use, such as 1 The history of research conducted, funded, or supported by the tobacco industry is not raised to be retributive or punitive, but simply to acknowledge that past actions reflect on the credibility of the industry’s current research, which may pose a problem for regulators, particularly in the contentious area of MRTPs.

OCR for page 221
228 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS adolescents and populations with mental health issues, may have a higher prevalence of individuals with reduced decision-making capacity. When investigators are conducting research involving these high-risk groups, they should be particularly cautious about the inclusion of individuals who lack the capacity to provide meaningful consent. The tobacco industry is currently limited in its ability to produce credible and comprehensive data. The challenges created by that lack of credibility are augmented by the fact that it is inevitable that product sponsors will need to collect extensive data on the effect of products in populations vulnerable to high use rates. Because of those challenges, at least part of the research base in support of an MRTP may need to be generated by researchers and organizations independent of the sponsors of the MRTP in question (Rees et al., 2009a, 2009b). The creation of a third party or third parties for the conduct and oversight of studies of MRTPs could help overcome some of the issues discussed above. The Health Effects Institute (HEI), a nonprofit corporation with approxi - mately one-half of its funding coming from the automobile industry and the other half coming from the federal government and other govern - ment sources, has successfully managed the boundary between industry and government, and between the research community in health effects and the research community in air quality (Keating, 2001). HEI, how - ever, does not fund projects in support of specific marketing applica - tions, but rather it funds projects that contribute to general knowledge. The Reagan-Udall Foundation (RUF) advises the FDA on modernizing regulatory science, and it conducts and oversees studies on regulatory science, particularly in the emerging fields of pharmacogenomics and genomic-based prediction of drug response and adverse event risk. The RUF has in place controls for bias and conflict of interest that are noteworthy. Public access to the totality of the data on MRTPs is critical to the credibility of MRTP research. Registering studies funded by the National Institutes of Health or used in drug approvals on the National Library of Medicine website Clinicaltrials.gov has greatly improved the transpar- ency of those studies. The FDA may have to balance the need for public access to key information with the need to protect proprietary information and promote innovation. Public availability of data provided in an MRTP application is discussed in the Family Smoking Prevention and Tobacco Control Act of 2009 (FSPTCA).2 2 Family Smoking Prevention and Tobacco Control Act of 2009, Public Law 111-31, 123 Stat. 1776 (June 22, 2009).

OCR for page 221
229 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES INTEGRATION OF EVIDENCE AND DECISION MAKING Table 6-1 presents the evidence domains and example considerations for using evidence from the different domains. A key challenge facing the FDA will be integrating the various domains and levels of evidence provided by sponsors in support of an MRTP application. A system- atic, explicit approach that weights outcomes in terms of their public health importance, identifies the measures and data most relevant to those outcomes, and combines the available evidence in a manner that is psychometrically sound, objective, and reproducible would be helpful. This effort could be informed by decision theory concepts and techniques such as expected utility, Bayesian, and improper linear model approaches. Mathematical modeling and simulation analysis, such as discussed earlier in Chapter 3, can also be used to predict population-level effects. It is clear that no single class of evidence (e.g., preclinical, RCTs, con - sumer perception, epidemiologic) in itself will be sufficient to support an MRTP application. Studies offered in support of an MRTP application should address all key research domains needed to prognosticate the product’s likely public health impact, including the following: • Product content (including but not limited to harmful and poten- tially harmful constituents), performance, and quality assurance regard - ing product consistency • Self-administration and subjective evaluation • Exposure assessment by state-of-the-art methods and measure- ments that focus on human exposure to harmful and potentially harmful constituents • Consumer and nonconsumer perceptions; and assessment of bio- markers of exposure, biomarkers of risk, and where feasible, disease outcomes The research should use designs that are properly powered, balance internal and external validity, and comprise multiple populations appro - priate to the experimental questions being addressed. Target populations of special relevance include (but are not limited to) users of tobacco products, both those interested and uninterested in quitting; nonsmokers; former smokers; beginning smokers; and adolescents. In addition, study samples must permit inferences to populations with significant smoking prevalence such as those low in socioeconomic status and educational attainment, and certain ethnic minorities. Beyond merely amassing evidence in support of the modified risk claims, higher-level processing of the evidence is needed to inform deci - sion making. It is probable that, depending on their construction, intended uses, and desired claims, different MRTP applications could require dif -

OCR for page 221
230 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS TABLE 6-1 Evidence Domains Relevant to an MRTP Application and Examples of Types of Findings Class of Evidence Examples of Types of Finding That May Be Required Preclinical • Assurance of manufacturing quality control • Significant and substantial reduction in toxicant and carcinogen content in product • Significant reduction in exposure to toxicants and carcinogens in limited human study • No significant evidence for offsetting increases in content of or exposure to other toxicants Clinical trial • Significant reduction in exposure to toxicants and carcinogens in relation to continued use of traditional product, preferably approaching nonsmoker levels • Significant rates of cessation of conventional tobacco product use, or significant decrease in the rates of conventional tobacco product use • Significant reduction in biomarkers or surrogates of disease Abuse potential • No more liable for abuse than currently marketed products • No significant evidence of attractiveness to nonusers of tobacco Epidemiology • Clear and consistent evidence of reduction in disease risk (e.g., cancer, cardiovascular disease, chronic obstructive pulmonary disease) or intermediate endpoint thereof • No significant evidence of offsetting increased risk for other diseases • No significant evidence of uptake among nonusers or relapse among former users (postmarketing) Consumer and • Evidence for accurate understanding of product claim nonconsumer • No significant evidence that consumers equate reduced perceptions exposure with reduced risk • No significant evidence of intention to use product among nonusers (especially adolescents) • No significant evidence of switching from MRTP to other tobacco product usage Populations at • No significant evidence of risk of initiation among nonusers high risk for (especially adolescents) tobacco use • Consistency of findings across relevant subpopulations of interest (e.g., low socioeconomic status, racial/ethnic minorities) Modeling and • Population predictions show reduction in smoking-related synthesis morbidity and mortality following the introduction of an MRTP with no significant evidence of uptake by nonusers (especially adolescents) NOTE: This table is not comprehensive and is not intended to be a guideline or framework for the evaluation of MRTP applications.

OCR for page 221
231 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES ferent balances of the various classes of evidence. As such, prescribing a universally applicable portfolio of evidence is difficult to envision. However, it remains possible to frame four broad scenarios for an MRTP application, as shown in the Table 6-2. A product sponsor may file an MRTP application for a product that is already on the market or for a new product (meaning one not available on the U.S. market). For each of these, a sponsor could request a Modi - fied Risk claim or a modified exposure claim under the Special Rule for Certain Products. Existing products for which sponsors wish to make a risk/exposure claim raise particular concerns because of conscious and unconscious associations built over time with branding and other marketing mes- sages. This is particularly true for MRTPs that are cobranded with other tobacco products. In the case of an existing product wishing to make a claim of modified risk, it would not be unreasonable of the FDA to expect epidemiologic evidence (e.g., case-control or cohort studies of users and nonusers of the MRTP about which the claim is made, with clinical or validated surrogate endpoints) and to have this evidence weigh heavily in its decision making. Supporting evidence for such an application could include switching studies and RCTs that conclusively demonstrate sub - stantially reduced biomarkers of toxicant exposure or biomarkers of risk. It would be anticipated that preclinical evidence from human, animal, in vitro, or in vivo studies would play relatively minor roles (e.g., providing mechanistic context) in justifying a claim for a product that is already on the market. However, all MRTP products would benefit from having sup- portive preclinical studies completed prior to human studies, even if they are already on the market. Lastly, significant emphasis would need to be placed on extensive consumer and nonconsumer testing of the proposed advertising and marketing materials, product packaging, and design to determine (1) whether the typical consumer understands the message and is unlikely to be misled and (2) if the product is minimally attractive to nonusers of tobacco. In the case of a product not currently available to consumers, claims of reduced risk (which require epidemiologic evidence) would be diffi- cult to support. However, if the identical product were available in other TABLE 6-2 Four Broad Scenarios for an MRTP Application Modified Exposure Modified Risk (Special Rule) Product already on market A B Product not on market C D

OCR for page 221
236 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS The cross-validity of a model refers to whether the model’s results agree with those of other models developed for the same purpose. For example, the Cancer Intervention and Surveillance Modeling Network (CISNET) initiative funded by the National Cancer Institute involves the development and comparison of the results of alternative models for assessing the impact of technologies for the early detection of cancer (CISNET Breast Cancer Collaborators, 2006). Cross-validation can be time and resource intensive, because it requires the development of several alternative models and the comparison of their results. The predictive validity of a model refers to the model’s ability to accurately predict future outcomes. This aspect of model validity is par- ticularly important for the intended use of modeling in regulating MRTPs. For example, a model developed on the basis of currently available infor- mation can be linked to future empirical studies of the impact of an MRTP and validated partly or fully using data from those studies. Uncertainty Uncertainty accounting refers to the systematic examina- tion, assessment, and reporting of the uncertainty in model inputs and assumptions, estimates of model parameters, and summary measures of the results (Bilcke et al., 2011). There are three important sources of uncer- tainty in model-based analyses. First, there can be uncertainty about the structural assumptions of the model, such as assumptions about variables to include and causation and prediction pathways. The uncertainty about such assumptions is often difficult to quantify, but their impact on the conclusions of the analysis may be substantial (Bojke et al., 2009). Second, there can be uncertainty about model parameter estimates derived from available studies. Formal statistical measures for this uncertainty can be derived using meta-analysis methods. However, the assessment of how this uncertainty propagates through the modeling computations to the final results continues to be a challenge. Third, there can be variability among individuals in the population, which can be explained on the basis of characteristics of these individuals (systematic or explained variation) or is considered to be random. The impact of the various forms of uncertainty on the final results of the model is typically assessed by deterministic or probabilistic sensitivity analysis. However, because of the extent and variety of uncertainty in a modeling analysis, a realistic account of its impact on the results of the analysis continues to be challenging. For example, most sensitivity analy - ses in practice do not account for the multivariate structure of inputs and the correlations in this structure. Guidelines on how the results of sen-

OCR for page 221
237 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES sitivity analyses should be reported are currently in development5 (Bilcke et al., 2011; Stout et al., 2009). Selection of Comparison Products The amount of harm reduction claimed by an MRTP sponsor in an application is a critical issue in deciding whether to issue an order for the marketing of the MRTP. Harm reduction is inherently relative; a reduc- tion claim is by definition relative to a comparison product. Selection of an appropriate comparison product is essential for informed and accurate decision making. The FSPTCA recognizes this, giving the Secretary of the Department of Health and Human Services authority to require product sponsors to compare their product to a commercially marketed represen - tative product. The choice of appropriate comparison products will be driven by the type of MRTP being tested, the anticipated claim, and the study design. And, indeed, the comparison products may differ between different classes of evidence. However, two reference products come to the forefront in terms of integration and synthesis of evidence: leading brands and smoking cessation products. Leading Brands Those products that are most commonly used by consumers are likely to provide a good comparison for products that claim to demonstrate reduced health risk. “Leading brands” represent a set of products that accounts for a significant portion of the market and could capture sub - groups of interest (e.g., low socioeconomic status, who tend to use dis - count brands, and racial/ethnic minorities, who tend toward menthols). Using leading brands increases the likelihood that the findings will have broader applicability to the population, which is crucial given the pub- lic health standard against which MRTPs are evaluated. Using leading brands as a comparator also avoids potential mischief in comparing an MRTP to a product that is little used but may inflate the apparent risk reduction of the MRTP. In some cases, when desiring a reduced exposure or risk claim, the comparison product will be a product within the same product class (e.g., cigarette-like MRTP versus leading brand cigarette), and thus the comparison is relatively straightforward. In other cases, an MRTP may make a reduced risk/exposure claim across product classes (e.g., smokeless MRTP versus leading brand cigarettes). In this case, the product should also be compared to leading products in its own class 5 ISPOR-SMDM Modeling Good Research Practices Task Force Working Group Part 3, unpublished data, 2011.

OCR for page 221
238 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS (e.g., smokeless MRTP versus moist snuff). Following this example, a smokeless MRTP that succeeds in both within- and cross-class compari - sons against leading brands could reduce risk/exposure for both smokers and traditional smokeless users. However, it is also possible that a smoke- less MRTP does pose less risk or exposure than cigarettes but is no differ- ent than other smokeless products not seeking a claim. Smoking Cessation On the opposite end of the spectrum of exposure and risk reduction is the “gold standard” of smoking cessation (or tobacco cessation in the case of smokeless tobacco users). This provides an aspirational goal for risk and exposure for MRTPs—in principle, the closer risks and exposures from the MRTP are to cessation products, the more confident a regulator can be in the chances for net public health benefit. Note that the use of this comparison product is not the same as studying whether the MRTP acts as an aid to smoking cessation. Rather, the goal is to compare how the risk or exposure reduction attained with use of the MRTP compares to smoking cessation of similar duration. It is also important to consider that for some health conditions, such as acute cardiovascular outcomes and lung function decline, the benefits of cessation accrue more quickly than for cancer. FINDINGS AND RECOMMENDATIONS In the committee’s view, the fundamental problem that confronts the FDA is a shortage of credible and reliable evidence about the effects of MRTPs on both individual and public health. The history of deceptive behavior by the tobacco industry undermined the trust of the public as well as the public’s confidence in the industry’s ability to rigorously conduct studies that will generate the data needed to evaluate these products. Therefore, the committee’s recommendations are designed to articulate the minimum standards for producing credible and reliable evidence to demonstrate that the marketing of an MRTP is consistent with the protection of public health. The committee articulates a strategy for the production of scientific evidence by making recommendations in three areas: 1. Types of evidence and studies 2. Design and integration of studies 3. Governance of studies

OCR for page 221
239 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES Types of Evidence and Studies Finding 1: Types of Evidence. The public health standard articulated by the FSPTCA requires collection of scientific evidence from a wide range of disciplines and research domains. Although the committee respects the FDA’s independence and discretion in regulating MRTPs, the committee maintains there is a minimum range of research domains required to evaluate the effect of MRTPs on individuals and public health. Individual methods may change as the technology or state of the science may evolve, but the minimum standards for the domains of evidence will be relevant regardless of the state of the science in the future. Recommendation 1: The FDA should require that studies submitted in support of an MRTP application address all key research domains needed to forecast and monitor the product’s public health impact, including: • product composition and performance; • addiction potential and likelihood for initiation or persistence of use; • human exposure to harmful and potentially harmful constituents; • perceptions about the product’s effects and likelihood of addic- tion; and • effects of the product on human health and surrogates of human health. Finding 2: Phased Approach to New MRTPs. Many novel MRTPs are likely to be developed for marketing in the near future. There are inher- ent uncertainties and risks with new products that should be addressed. Risks should be minimized before new products are tested in humans. To address the risk of new products, a phased approach, similar to the New Drug Application framework for the regulation and control of new drugs, is appropriate for the evaluation of new MRTPs. A phased approach will help the FDA ensure that only products that are unlikely to be unsafe and have a reasonable expectation of reducing harm relative to conventional tobacco products will be used in human studies. Recommendation 2: The FDA should establish guidance that conveys an expected sequencing of studies, such that preclinical work is com- pleted and submitted to the FDA before clinical (human subjects) work commences, and that there is a reasonable expectation based on preclinical work that a reduction or lack of harm will be seen in humans.

OCR for page 221
240 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS Finding 3: Clinical Trial Studies. Although the use of randomized con- trolled trial methods will be constrained for a number of reasons (includ - ing the practical limitations of study cost, size, and follow-up, and ethical constraints on randomizing study participants to harmful exposures), they will continue to play an essential role in creating an evidence base on the public health effects of MRTPs. Randomized controlled trial methods can provide highly reliable data on the likelihood of addiction and initia - tion or cessation of product use. Also, these methods can provide reliable evidence on human exposure. Recommendation 3: The FDA should require randomized controlled trials in the following domains: • Exposure reduction • Self-administration of the MRTP • Effects on use of conventional tobacco products These randomized controlled trials should include multiple com- parison products (such as nicotine replacement products, conventional cigarettes or smokeless tobacco, placebo preparations, and alternative nicotine delivery systems). These trials should also assess the effect of the MRTP on human exposure and on human health and surrogates of human health. Finding 4: Requirement for Postmarket, Prospective Epidemiologic Studies. Postmarket studies of MRTPs will be critical to evaluating the effect of MRTPs on both individuals and the public’s health. In particular, the prospective cohort design will be an essential tool to validating antici- pated or claimed effects of marketed MRTPs. These studies have several important strengths: (1) biochemical tobacco and MRTP exposure can be assessed at baseline, offering “unbiased” exposure assessment before health outcomes occur; (2) there is less of a problem with retrospective recall of product use, beause this information can be summarized at the start of the study and followed prospectively; (3) changing product use habits can be monitored as the study progresses; (4) outcomes can be documented as they occur, and verification is more efficient; and (5) a wide variety of outcomes can be evaluated in the same study, particu- larly outcomes that are more common. Furthermore, cohort studies allow assessment of overall health status and outcomes. Recommendation 4: The FDA should require prospective epidemio- logic studies to commence upon issuance of a marketing order to confirm reduced exposure and reduced risk claims, and to examine

OCR for page 221
241 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES effects of MRTP availability on the population as a whole, includ - ing the likelihood of initiation and cessation. The FDA should issue guidance on the design, conduct, and analysis of such studies. Finding 5: Modeling of Public Health Outcomes. Mathematical model- ing and simulation analysis provides a complementary approach to the conduct of empirical studies that can be useful at each stage of the regulatory process for MRTPs. Model-based analyses can (1) synthesize the available information from empirical studies of MRTPs; (2) enable researchers and decision makers to explore complex interactions and systems that may be impractical to evaluate in empirical studies; (3) allow researchers and decisions makers to explore “what if” questions relevant to decision making, which would not be practical to assess in empirical studies; and (4) be used to make projections about the short- and long- term effects of the introduction of MRTPs. Recommendation 5: The FDA should issue guidance on the develop- ment and use of simulation and modeling approaches to predict pub- lic health impact through the systematic integration of information about relevant assumptions and influences. Such approaches should be tested for robustness with regard to results and assumptions, they should be public and transparent, and they should be validated against postmarketing epidemiologic research. Design and Integration of Studies Finding 6: Standards for Sampling in MRTP Studies. To have regula- tory usefulness, studies of MRTPs must be generalizable to the overall population of interest and to specific populations, including populations at high risk for tobacco use. Failure to include relevant populations in studies will result in incomplete evidence on the effect of an MRTP on the public’s health and, therefore, will be inadequate to support regulatory decisions about the marketing of MRTPs. Recommendation 6: The FDA should require studies to include pop- ulations of special relevance, including (but not limited to) • users of tobacco products, including users who are and are not interested in quitting; • in certain circumstances, nonusers of tobacco products; • former smokers; • beginning smokers; • adolescents; and

OCR for page 221
242 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS • populations at a high risk for tobacco use, including, but not limited to, those low in socioeconomic status and educational attain - ment, and certain ethnic minorities. Finding 7: Quality of Studies. The usefulness of a study to inform a reg- ulatory decision hinges on the quality and appropriateness of the design. In many cases, complementary studies might be needed to provide a breadth of evidence for an informed regulatory decision with appropriate control of confounders and internal and external validity. Recommendation 7: For all studies of the effects of MRTPs on human health and behavior, the FDA should require a range of designs that are properly powered, balance internal and external validity, and comprise multiple populations appropriate to the experimental ques- tions being addressed. Finding 8: Standards for Good Research Practice. A significant amount of guidance on minimum standards for scientific studies directly relevant to the evaluation of MRTPs has already been developed. Guidelines for formatting, design, conduct, and reporting of science are articulated in consensus statements, such as the Consolidated Standards of Reporting Trials (CONSORT) reporting criteria for clinical trials, the Strengthen - ing the Reporting of Observational Studies in Epidemiology (STROBE) statement for observational studies, the publication criteria of the Inter- national Council of Medical Journal Editors, and the reporting criteria of the International Conference on Harmonization. These existing guidelines represent robust standards for the conduct of science across many of the research domains relevant to the evaluation of MRTPs. Recommendation 8: The FDA should issue guidance to the industry regarding the format, design, conduct, and reporting of studies in support of MRTP applications that is based upon current generally accepted principles for scientific investigation. Finding 9: Standards for Integration of Evidence. Regulatory decisions regarding MRTPs will be based on a wide range and variety of scientific evidence, and the integration of scientific evidence will play a pivotal role in that decision making. The assessment of MRTPs will typically require the evaluation and integration of evidence on risks and ben - efits across multiple diverse outcomes, such as measures of toxicity, bio - markers, addictiveness, and disease endpoints. Modeling and simulation approaches are relevant to estimating public health effects of tobacco and, therefore, the FDA will likely engage in various methods of data

OCR for page 221
243 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES integration, synthesis, and analysis, including, but not limited to, simula - tion and modeling. It is critical that these approaches are transparent and reproducible. Recommendation 9: The FDA should develop and use an approach to data integration that is explicit and transparent with regard to the importance of the different outcomes, that uses optimal available evidence, and that employs objective and reproducible methods for data integration. Governance of Studies Finding 10: Independent Oversight and Conduct of Studies. It has been established in public records and as a matter of law that the tobacco industry has engaged in illegal and improper practices, including the destruction and manipulation of scientific data. As a result, the tobacco industry is profoundly isolated from the mainstream scientific commu- nity. Many major universities have policies against acceptance of tobacco funding, and many high-impact scientific and medical journals will not accept tobacco industry-supported manuscripts. The consequence of this isolation is a lack of the expertise and the resources necessary to produce high-quality science across the range of disciplines to support an applica - tion to market an MRTP. Use of a trusted third party, particularly for prod- ucts developed by the tobacco industry, could provide an avenue for the production of credible evidence needed by the FDA to evaluate tobacco products. Ultimately, such a research structure could encourage and sup- port the production and dissemination of credible and reliable evidence about the effects of tobacco products on the public’s health. Recommendation 10: MRTP sponsors should consider use of inde- pendent third parties to undertake one or more key functions, including the design and conduct of research, the oversight of spe- cific studies, and the distribution of sponsor funds for research. Such independent third parties should be approved by the FDA in advance of the research. Finding 11: Public Disclosure of Research. Public availability of data not only builds credibility and public trust, but also benefits the public because it allows for independent analysis of study methods and data. The model of Clinicaltrials.gov is particularly compelling and relevant, and a similar model of public accounting and open disclosure should be expected of the tobacco industry.

OCR for page 221
244 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS Recommendation 11: The FDA should require all MRTP sponsors to place all data generated in the development and marketing of the MRTP in a public repository selected by the FDA. Finding 12: Proper Conduct of Research. Standards for the conduct of science and the protection of human research participants have been established for biomedical research enterprises not only in academics but also in commercial research. The FDA has the tools to ensure studies adhere to established standards in the drug development framework, which can be applied to the development of MRTPs. Those standards not only protect human participants, but also build credibility into any data that are provided to the FDA, particularly by the tobacco industry. Institutional credibility and trustworthiness is particularly relevant in this context, given the history of unethical and illegal practices of the tobacco industry. Recommendation 12: The FDA should require studies offered in support of an MRTP application to adhere to established standards and principles of good research governance, including appropri- ately qualified investigators, transparency, independent institutional review board or ethical review, and adherence to the Common Rule (21 CFR parts 50 and 56). REFERENCES Ashley, M. J., and J. E. Cohen. 2003. What the public thinks about the tobacco industry and its products. Tobacco Control 12(4):396-400. Bilcke, J., P. Beutels, M. Brisson, and M. Jit. 2011. Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: A practical guide. Medical Deci- sion Making 31(4):675-692. Bojke, L., K. Claxton, M. Sculpher, and S. Palmer. 2009. Characterizing structural uncer- tainty in decision analytic models: A review and application of methods. Value Health 12(5):739-749. CISNET Breast Cancer Collaborators. 2006. The impact of mammography and adjuvant therapy on U.S. breast cancer mortality (1975-2000): Collective results from the cancer intervention and surveillance modeling network. JNCI Monographs 2006(36):1-126. Cummings, K. M. 2003. A promise is a promise. Tobacco Control 12(2):117-118. Cummings, K. M., C. Morley, and A. Hyland. 2002. Failed promises of the cigarette industry and its effect on consumer misperceptions about the health risks of smoking. Tobacco Control 11(Suppl. 1):i110-i117. Cummings, K. M., A. Brown, and R. O’Connor. 2007. The cigarette controversy. Cancer Epi- demiology, Biomarkers & Prevention 16(6):1070-1076. Glickman, S. W., J. G. McHutchison, E. D. Peterson, C. B. Cairns, R. A. Harrington, R. M. Califf, and K. A. Schulman. 2009. Ethical and scientific implications of the globalization of clinical research. NEJM 360(8):816-823.

OCR for page 221
245 DECISION MAKING AND OVERSIGHT OF MRTP STUDIES Harris Interactive. 2003. Attitudes to government regulation vary greatly for different indus - tries; The Harris Poll #19. http://www.harrisinteractive.com/vault/Harris-Interactive- Poll-Research-Attitudes-to-Government-Regulation-Vary-Greatly-for-Different- Industries-2003-04.pdf (accessed December 5, 2011). Harris Interactive. 2010. Oil, pharmaceutical, health insurance, and tobacco top the list of industries that people think should be more regulated; The Harris Poll #149. http://www.harrisinteractive. com/vault/HI-Harris-Poll-Industry-Regulation-2010-12-02.pdf (accessed December 5, 2010). Keating, T. J. 2001. Lessons from the recent history of the Health Effects Institute. Science, Technology, & Human Values 26(4):409-430. Mendez, D., S. P. Marcus, P. I. Clark, S. J. Leischow, and T. F. Pechacek. 2008. Tobacco mod - elers conference proceedings. In Tobacco modelers conference. Ann Arbor: University of Michigan. NCI (National Cancer Institute). 2008. The role of the media in promoting and reducing tobacco use. Edited by R. M. Davis, NCI tobacco control monograph series. Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health, National Cancer Institute. NRC (National Research Council). 1991. Improving information for social policy decisions: The uses of microsimulation modeling. Vol. 1. Washington, DC: National Academy Press. Rees, V. W., J. M. Kreslake, K. M. Cummings, R. J. O’Connor, D. K. Hatsukami, M. Parascandola, P. G. Shields, and G. N. Connolly. 2009a. Assessing consumer responses to potential reduced-exposure tobacco products: A review of tobacco industry and independent research methods. Cancer Epidemiology, Biomarkers & Prevention 18(12):3225-3240. Rees, V. W., J. M. Kreslake, R. J. O’Connor, K. M. Cummings, M. Parascandola, D. Hatsukami, P. G. Shields, and G. N. Connolly. 2009b. Methods used in internal industry clinical trials to assess tobacco risk reduction. Cancer Epidemiology, Biomarkers & Prevention 18(12):3196-3208. Rutter, C. M., A. M. Zaslavsky, and E. J. Feuer. 2011. Dynamic microsimulation models for health outcomes. Medical Decision Making 31(1):10-18. Sonnenberg, F. A., and J. R. Beck. 1993. Markov models in medical decision making: A practi- cal guide. Medical Decision Making 13(4):322-338. Stout, N. K., A. B. Knudsen, C. Y. J. Kong, P. M. McMahon, and G. S. Gazelle. 2009. Calibra - tion methods used in cancer simulation models and suggested reporting guidelines. Pharmacoeconomics 27(7):533-545. TPSAC (Tobacco Products Scientific Advisory Committee). 2011. Menthol cigarettes and public health: Review of the scientific evidence and recommendations. Tobacco Products Scientific Advisory Committee. Vanni, T., J. Karnon, J. Madan, R. G. White, W. J. Edmunds, A. M. Foss, and R. Legood. 2011. Calibrating models in economic evaluation: A seven-step approach. Pharmacoeconomics 29(1):35-49. Weinstein, M. C., B. O’Brien, J. Hornberger, J. Jackson, M. Johannesson, C. McCabe, and B. R. Luce. 2003. Principles of good practice for decision analytic modeling in health- care evaluation: Report of the ispor task force on good research practices—modeling studies. Value Health 6(1):9-17.

OCR for page 221