Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 221
6
Decision Making and
Oversight of MRTP Studies:
Findings and Recommendations
In the previous three chapters, the committee describes the evidence
domains for potential modified risk tobacco products (MRTPs), includ-
ing studies of health effects, addictive potential, and risk perception and
communication. The committee discussed the governance of those studies
in Chapter 2. The Food and Drug Administration (FDA) will have to inte-
grate the evidence from those diverse domains when making regulatory
decisions about MRTPs. Many of the same issues and concerns related
to the governance and design of studies arise regardless of the evidence
domain. The committee’s findings and recommendations, therefore, cut
across the evidence domains and focus on the types of evidence and
studies that the FDA should use in making regulatory decisions about
MRTPs, the design of studies of MRTPs for the FDA’s decision making,
and the governance of those studies. This chapter focuses on those cross-
cutting issues, ways the FDA can integrate information from those studies
as part of its regulatory decisions, and the committee’s overarching find -
ings and recommendations.
This chapter begins with a brief summary of the evidence domains
discussed in more detail in the previous chapters, a discussion of the sci -
entific and ethical issues in studying MRTPs, and the governance issues
that accompany such studies. The committee then discusses (1) the inte -
gration of all the evidence for the FDA’s regulatory decisions about these
products, including mathematical modeling and simulation techniques,
and (2) the comparative nature of the harm reduction claims that the
FDA will be evaluating. The committee then presents its findings and
recommendations.
221
OCR for page 222
222 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
ISSUES WITH EVIDENCE
Evidence Domains
The evidence to support the marketing authorization of an MRTP will
come from studies of health effects, addictive potential, and risk percep -
tion. In this section the committee provides a brief summary of the studies
within those evidence domains.
Health Effects
Laboratory analysis of the performance and of the constituents of
tobacco products will be the first step in the evaluation of any new product.
These analyses involve standard methods of extraction, sample cleanup,
analytic identification, and quantitation. There are important limitations
to laboratory analysis of product performance and composition. Labora -
tory analysis of constituents may not reflect constituent uptake under con-
ditions of use. Smoking machines do not replicate human smoking condi-
tions, and there is no proven way to replicate the many ways humans use
tobacco. It is crucial, therefore, to describe the smoking regimen or other
extraction methods employed.
The second step in the evaluation of MRTPs will be preclinical studies
of toxicity. These assays are essential in identifying particularly risky or
toxic products that should not be tested in humans, as well as products
that have reasonable potential to reduce risk and, therefore, should pro-
ceed to clinical evaluation. Evaluation of products in vitro should precede
in vivo assays. Although it is not possible to make laboratory animals use
tobacco products the way humans do, and there are inherent interspecies
differences that prevents meaningful extrapolation of human effects, it is
still informative to observe the effect of tobacco products in live animal
models. The number of animal studies required to characterize an MRTP
preclinically could potentially be reduced by setting composition standards
or limits or establishing standards for certain categories of MRTPs. Assays
of toxicity in humans will also be essential, in particular assays of urinary
mutagenicity and sister chromatid exchange in peripheral lymphocytes.
Biomarkers of exposure measure human exposure to constituents of
tobacco and could include the constituents themselves, their metabolites,
or protein (or DNA) binding products of the constituents or their metabo-
lites. These biomarkers have the potential to bypass many of the uncer-
tainties in product composition analysis and provide a realistic and direct
assessment of carcinogen and toxicant dose in an individual. Biomarkers
of exposure should be validated before their use.
When the FDA evaluates studies, it is important that it ensures the
constituents of a product are accurately and precisely measured, that
OCR for page 223
223
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
exposure methods are appropriate, and that any biomarkers of exposure
have been validated.
Experimental designs, in particular randomized controlled trials
(RCTs), provide data that can support the strong inferences about the
effect of an MRTP on human health relative to conventional tobacco prod-
ucts. The use of appropriately designed clinical trials will be important
to establish whether the use of the MRTP reduces exposure to toxicants
or induces positive changes in surrogate markers. An RCT is an effective
means of examining acceptability and use of the MRTP, the ability of the
MRTP to increase cessation in users of conventional tobacco products, and
the likelihood that MRTP availability will lead to dual use. RCT methods
can also produce evidence on whether and how much individuals use
an MRTP after they have used it to help them quit conventional prod-
ucts, changes in perception of the MRTP with its continued use, and the
MRTP’s ability to suppress tobacco withdrawal symptoms. It is important
to recognize that no single RCT can address all of these areas, and each
study should have a focused objective with a primary endpoint.
Postmarket studies of marketed MRTPs will be critical to evaluating
the effect of the MRTP on both individuals and on the public’s health. In
particular, the prospective cohort design will be an essential tool to vali -
dating anticipated or claimed effects of marketed MRTPs. These studies
can assess baseline tobacco and MRTP exposures; summarize product
use as the study is ongoing, including any changes in product use habits;
document and verify outcomes as they occur; and evaluate a wide variety
of outcomes, including both intermediate and clinical outcomes.
In addition, other study designs will be necessary to provide evidence
on the public health effects of MRTPs, including retrospective cohort
studies, case-control studies, crossover or case-crossover designs, and
comparative effectiveness research methods. Different study designs will
be necessary depending on the circumstances and the research question.
Addictive Potential
Evaluation of the likelihood of initiation, maintenance, and persis-
tence of use in both conventional tobacco users and nonusers is critical
to estimating the public health effect of marketing MRTPs. Specifically,
evaluation of the MRTP’s ability to promote initiation and continuation
of its regular use, switching to its use and cessation of the consumption
of more harmful products, dual use, and to promote relapse back to more
harmful tobacco use are all essential. All of these outcomes are related to
the reinforcing value of the MRTP (that is, how rewarding it is).
There is a continuum of reinforcement value. In theory, the MRTP
should be somewhat more reinforcing than nicotine replacement thera -
OCR for page 224
224 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
pies but perhaps less reinforcing than conventional cigarettes. Ideally, an
MRTP would be sufficiently reinforcing so as to attract smokers away
from conventional cigarettes but not encourage the widespread depen -
dent use of the product by individuals who were previously nonusers or
who would have quit smoking. Data from all relevant measures should
be integrated, taking into account not only the effects of the MRTP on
important health outcomes, but also the prevalence of use and projected
public health impact.
Evaluation of the abuse and addiction potential of a product can be
accomplished with a variety of experimental designs and in a variety
of contexts. Those include subjective evaluations in laboratory contexts,
acute self-administration studies in laboratory contexts, use in extended
residence facilities, and natural environment contexts where long-term
use can be studied in real-world circumstances via RCTs, cross-sectional
survey studies, and longitudinal cohort studies.
Evaluation of reinforcement value in a laboratory setting is particu-
larly important because the results of these studies reliably correspond
to an agent’s addictive potential in real-world use. One standard with
regard to human abuse liability drug testing are acute dose-effect com -
parison studies, because of the correspondence between subjective ratings
of drug effects and real-world abuse potential. Behavioral economic self-
administration studies will also be important to evaluating the reinforce-
ment potency of a product. The usefulness of all studies in forecasting
the risk for initiation and abuse of a product depends on study design
factors. Important design considerations include the size of the sample,
the nature of the sample (whether the sample includes heavy smokers or
light smokers, smokers who want to quit, and nonsmokers), the character-
ization of the sample (age, sex, gender, ethnicity, educational attainment,
socioeconomic status, etc.), and the nature of the comparison product.
Risk Perception and Communication
Judgments about risk, otherwise known as risk perceptions, are a
fundamental element to most theoretical models of health behavior and
behavioral decision making. In general, those models argue that indi -
viduals’ perceptions about the value and likelihood of behavior-related
positive and negative consequences and their vulnerability to those con -
sequences play a key role in behavioral choices. As such, understanding
individuals’ perceptions of tobacco-related products, including MRTPs,
whether such perceptions change over time, and whether such percep-
tions play a role in tobacco use behavior, is critical. It will be important to
identify consumers’ perceptions of disease risk, likelihood of addiction,
likelihood of reducing or increasing others’ exposure to potentially haz -
OCR for page 225
225
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
ardous compounds, and perceptions of risk compared to other products
already on the market. It is also important to assess intentions of using the
product. It is essential that industry carefully crafts messages about risks
and benefits of any MRTP and demonstrates through rigorous testing that
people correctly understand and interpret the risks.
Studies evaluating risk perceptions and risk communication must be
performed both before the marketing of an MRTP and after the MRTP has
been marketed. Premarket research will play an essential role in devel-
oping the messages that the tobacco industry can use to communicate
information about MRTPs to consumers. This research will determine
consumers’ ability to accurately understand messages that communicate
information about the risks, benefits, and conditions of use pertaining
to the MRTP itself and compared to existing tobacco products. Studies
should also test how these messages influence consumers’ perceptions
of the risks, benefits, and likelihood of addiction related to the MRTP.
The first stage of premarket research will involve formative work using
focus groups. The second stage should include discussions with groups
of similar individuals to assess how the messages that were developed in
the first stage are received by consumers. Finally, the effects of these mes -
sages on consumer perceptions should be tested. It will be important to
evaluate consumer understanding and to compare consumer perceptions
of the MRTP to conventional products. After the product is released on
the market, it is vital to continue monitoring consumer perceptions and
behavior related to that product. Conducting nationally representative
cohort-sequential longitudinal surveys will be essential.
Cross-Cutting Issues with Studies of MRTPs
When evaluating the studies of MRTPs, there are a number of con-
siderations that are relevant regardless of whether a study is looking at
health effects, addictive potential, or risk perception. Those cross-cutting
issues, which include issues related to study design and governance, are
discussed in this section.
Study Design
Issues that could affect the FDA’s evaluation of a study on MRTPs
include the generalizability of the study and how well the study is
conducted.
Studies should be designed appropriately to create an evidence base
that can support a finding of public health benefit. The ultimate goal of
studying the effect of MRTPs on human health and behavior is to be able
to accurately predict the public health effects of allowing an MRTP to be
OCR for page 226
226 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
marketed. In other words, the ultimate goal of scientific studies it to pro -
duce generalizable data. The “generalizability” of data, or the reliability
of predictions that can be made about the real world based on scientific
observations, will depend on the design of the studies.
The FDA should carefully evaluate the size and nature of the sample
to assess the generalizability of the study data. Sample sizes should be
carefully determined and tailored to the study design and the effects
studied. Statistically underpowered studies cannot support inferences
or projections about the effects of a product. The nature of the study
sample is critical to the usefulness of study results. Results from studies
conducted in one population may not be applicable to other populations,
because the characteristics that define the study population either are
related to or cause the responses to the product. As such, it is important to
study a wide range of populations. It is particularly important to include
populations that have a high risk of using tobacco and populations that
will be affected by the marketing of the product.
Study designs must also carefully consider the degree of control
imposed on experimental designs. Internal and external validity should
be balanced not only within studies but also across studies of the same
product. Highly controlled experimental designs can eliminate many vari-
ables and confounders and support strong inferences, but simultaneously
lose relevance to the FDA’s decisions because the conditions of product
use do not reflect real-world circumstances and behaviors. Experimental
designs that are less controlled can emulate circumstances that reflect
real-world conditions and behaviors, and therefore may be more relevant
in predicting real-world effects, but uncontrolled variables may confound
meaningful associations or inferences.
Regardless of the type of study design, the planning and conduct of
the study should meet good research practice standards. As discussed
earlier in this report, there are minimum standards for studies in other
settings, such as studies of pharmaceuticals. Consensus statements such
as Consolidated Standards of Reporting Trials (CONSORT) for clinical
trials, Strengthening the Reporting of Observational Studies in Epidemi -
ology (STROBE) for observational studies, publication criteria from the
International Council of Medical Journal Editors, and reporting criteria of
the International Conference on Harmonization have been implemented
to ensure that the design, conduct, and reporting of studies are consistent
with the state-of-the-art scientific standards. Studies of MRTPs should
meet those or similar criteria to help ensure the overall quality and integ-
rity of the studies. Certain types of studies will have specific criteria to
ensure quality research. Some of those specific criteria were discussed in
the preceding chapters.
OCR for page 227
227
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
Governance
Research on MRTPs will require some oversight or governance to
ensure the research is free from bias and conflicts of interest and that
appropriate controls are in place for human subject research. Such gover-
nance will also ensure the disclosure of data to ensure transparency and
instill confidence in the research findings.
The tobacco industry has a history of hiding and misrepresent-
ing information about the risks of tobacco products (Cummings, 2003;
Cummings et al., 2002, 2007).1 This history has lead to profound public
distrust in both the tobacco industry and in the research it sponsors, and
the absence of governance in the tobacco industry has created an iso-
lated industry that lacks not only the expertise to produce the necessary
range of credible and reliable data, but also the trustworthiness to acquire
external expertise and avenues to disseminate acquired data (Ashley and
Cohen, 2003; Harris Interactive, 2003, 2010; NCI, 2008). The production
of reliable and credible data depends upon building rigor, oversight,
and independence into the entire research process. Data problems often
cannot be detected after study completion, and therefore integrity and
accountability need to be built into the research throughout the study’s
execution. There is neither an established set of regulatory practices for
the review of MRTPs nor an established set of federal research standards
for the design, conduct, analysis, monitoring, and completion of studies
in support of MRTPs.
There are also a number of ethical issues associated with conducting
human subjects research involving MRTPs. The first issue is the risk of
conducting clinical trials of MRTPs or other tobacco products in popu -
lations with a high risk for tobacco initiation and addiction, including
but not limited to adolescents, certain ethnic minorities, and individuals
with mental health disorders. Randomization of participants to a product
known to be potentially addictive and hazardous is ethically problematic.
The second issue is the risk of improperly disclosing the substance abuse
of a minor to the minor’s parents or guardians in the process of obtaining
parental consent for research. While the assent of minors is always neces-
sary, investigators should also be aware of circumstances where waiver
of parental consent is warranted because obtaining parental consent will
violate the confidence of the study participant. The third issue is the
inclusion of individuals from high-risk groups with reduced decision-
making capacity. Some populations at a high risk for tobacco use, such as
1 The history of research conducted, funded, or supported by the tobacco industry is not
raised to be retributive or punitive, but simply to acknowledge that past actions reflect on
the credibility of the industry’s current research, which may pose a problem for regulators,
particularly in the contentious area of MRTPs.
OCR for page 228
228 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
adolescents and populations with mental health issues, may have a higher
prevalence of individuals with reduced decision-making capacity. When
investigators are conducting research involving these high-risk groups,
they should be particularly cautious about the inclusion of individuals
who lack the capacity to provide meaningful consent.
The tobacco industry is currently limited in its ability to produce
credible and comprehensive data. The challenges created by that lack
of credibility are augmented by the fact that it is inevitable that product
sponsors will need to collect extensive data on the effect of products in
populations vulnerable to high use rates. Because of those challenges,
at least part of the research base in support of an MRTP may need to be
generated by researchers and organizations independent of the sponsors
of the MRTP in question (Rees et al., 2009a, 2009b). The creation of a
third party or third parties for the conduct and oversight of studies of
MRTPs could help overcome some of the issues discussed above. The
Health Effects Institute (HEI), a nonprofit corporation with approxi -
mately one-half of its funding coming from the automobile industry and
the other half coming from the federal government and other govern -
ment sources, has successfully managed the boundary between industry
and government, and between the research community in health effects
and the research community in air quality (Keating, 2001). HEI, how -
ever, does not fund projects in support of specific marketing applica -
tions, but rather it funds projects that contribute to general knowledge.
The Reagan-Udall Foundation (RUF) advises the FDA on modernizing
regulatory science, and it conducts and oversees studies on regulatory
science, particularly in the emerging fields of pharmacogenomics and
genomic-based prediction of drug response and adverse event risk.
The RUF has in place controls for bias and conflict of interest that are
noteworthy.
Public access to the totality of the data on MRTPs is critical to the
credibility of MRTP research. Registering studies funded by the National
Institutes of Health or used in drug approvals on the National Library of
Medicine website Clinicaltrials.gov has greatly improved the transpar-
ency of those studies. The FDA may have to balance the need for public
access to key information with the need to protect proprietary information
and promote innovation. Public availability of data provided in an MRTP
application is discussed in the Family Smoking Prevention and Tobacco
Control Act of 2009 (FSPTCA).2
2 Family Smoking Prevention and Tobacco Control Act of 2009, Public Law 111-31, 123 Stat.
1776 (June 22, 2009).
OCR for page 229
229
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
INTEGRATION OF EVIDENCE AND DECISION MAKING
Table 6-1 presents the evidence domains and example considerations
for using evidence from the different domains. A key challenge facing
the FDA will be integrating the various domains and levels of evidence
provided by sponsors in support of an MRTP application. A system-
atic, explicit approach that weights outcomes in terms of their public
health importance, identifies the measures and data most relevant to
those outcomes, and combines the available evidence in a manner that is
psychometrically sound, objective, and reproducible would be helpful.
This effort could be informed by decision theory concepts and techniques
such as expected utility, Bayesian, and improper linear model approaches.
Mathematical modeling and simulation analysis, such as discussed earlier
in Chapter 3, can also be used to predict population-level effects.
It is clear that no single class of evidence (e.g., preclinical, RCTs, con -
sumer perception, epidemiologic) in itself will be sufficient to support
an MRTP application. Studies offered in support of an MRTP application
should address all key research domains needed to prognosticate the
product’s likely public health impact, including the following:
• Product content (including but not limited to harmful and poten-
tially harmful constituents), performance, and quality assurance regard -
ing product consistency
• Self-administration and subjective evaluation
• Exposure assessment by state-of-the-art methods and measure-
ments that focus on human exposure to harmful and potentially harmful
constituents
• Consumer and nonconsumer perceptions; and assessment of bio-
markers of exposure, biomarkers of risk, and where feasible, disease
outcomes
The research should use designs that are properly powered, balance
internal and external validity, and comprise multiple populations appro -
priate to the experimental questions being addressed. Target populations
of special relevance include (but are not limited to) users of tobacco
products, both those interested and uninterested in quitting; nonsmokers;
former smokers; beginning smokers; and adolescents. In addition, study
samples must permit inferences to populations with significant smoking
prevalence such as those low in socioeconomic status and educational
attainment, and certain ethnic minorities.
Beyond merely amassing evidence in support of the modified risk
claims, higher-level processing of the evidence is needed to inform deci -
sion making. It is probable that, depending on their construction, intended
uses, and desired claims, different MRTP applications could require dif -
OCR for page 230
230 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
TABLE 6-1 Evidence Domains Relevant to an MRTP Application
and Examples of Types of Findings
Class of Evidence Examples of Types of Finding That May Be Required
Preclinical • Assurance of manufacturing quality control
• Significant and substantial reduction in toxicant and carcinogen
content in product
• Significant reduction in exposure to toxicants and carcinogens
in limited human study
• No significant evidence for offsetting increases in content of or
exposure to other toxicants
Clinical trial • Significant reduction in exposure to toxicants and carcinogens
in relation to continued use of traditional product, preferably
approaching nonsmoker levels
• Significant rates of cessation of conventional tobacco product
use, or significant decrease in the rates of conventional tobacco
product use
• Significant reduction in biomarkers or surrogates of disease
Abuse potential • No more liable for abuse than currently marketed products
• No significant evidence of attractiveness to nonusers of tobacco
Epidemiology • Clear and consistent evidence of reduction in disease risk (e.g.,
cancer, cardiovascular disease, chronic obstructive pulmonary
disease) or intermediate endpoint thereof
• No significant evidence of offsetting increased risk for other
diseases
• No significant evidence of uptake among nonusers or relapse
among former users (postmarketing)
Consumer and • Evidence for accurate understanding of product claim
nonconsumer • No significant evidence that consumers equate reduced
perceptions exposure with reduced risk
• No significant evidence of intention to use product among
nonusers (especially adolescents)
• No significant evidence of switching from MRTP to other
tobacco product usage
Populations at • No significant evidence of risk of initiation among nonusers
high risk for (especially adolescents)
tobacco use • Consistency of findings across relevant subpopulations of
interest (e.g., low socioeconomic status, racial/ethnic minorities)
Modeling and • Population predictions show reduction in smoking-related
synthesis morbidity and mortality following the introduction of an MRTP
with no significant evidence of uptake by nonusers (especially
adolescents)
NOTE: This table is not comprehensive and is not intended to be a guideline or framework
for the evaluation of MRTP applications.
OCR for page 231
231
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
ferent balances of the various classes of evidence. As such, prescribing
a universally applicable portfolio of evidence is difficult to envision.
However, it remains possible to frame four broad scenarios for an MRTP
application, as shown in the Table 6-2.
A product sponsor may file an MRTP application for a product that
is already on the market or for a new product (meaning one not available
on the U.S. market). For each of these, a sponsor could request a Modi -
fied Risk claim or a modified exposure claim under the Special Rule for
Certain Products.
Existing products for which sponsors wish to make a risk/exposure
claim raise particular concerns because of conscious and unconscious
associations built over time with branding and other marketing mes-
sages. This is particularly true for MRTPs that are cobranded with other
tobacco products. In the case of an existing product wishing to make a
claim of modified risk, it would not be unreasonable of the FDA to expect
epidemiologic evidence (e.g., case-control or cohort studies of users and
nonusers of the MRTP about which the claim is made, with clinical or
validated surrogate endpoints) and to have this evidence weigh heavily
in its decision making. Supporting evidence for such an application could
include switching studies and RCTs that conclusively demonstrate sub -
stantially reduced biomarkers of toxicant exposure or biomarkers of risk.
It would be anticipated that preclinical evidence from human, animal, in
vitro, or in vivo studies would play relatively minor roles (e.g., providing
mechanistic context) in justifying a claim for a product that is already on
the market. However, all MRTP products would benefit from having sup-
portive preclinical studies completed prior to human studies, even if they
are already on the market. Lastly, significant emphasis would need to be
placed on extensive consumer and nonconsumer testing of the proposed
advertising and marketing materials, product packaging, and design to
determine (1) whether the typical consumer understands the message and
is unlikely to be misled and (2) if the product is minimally attractive to
nonusers of tobacco.
In the case of a product not currently available to consumers, claims
of reduced risk (which require epidemiologic evidence) would be diffi-
cult to support. However, if the identical product were available in other
TABLE 6-2 Four Broad Scenarios for an MRTP Application
Modified Exposure
Modified Risk (Special Rule)
Product already on market A B
Product not on market C D
OCR for page 236
236 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
The cross-validity of a model refers to whether the model’s results
agree with those of other models developed for the same purpose. For
example, the Cancer Intervention and Surveillance Modeling Network
(CISNET) initiative funded by the National Cancer Institute involves
the development and comparison of the results of alternative models
for assessing the impact of technologies for the early detection of cancer
(CISNET Breast Cancer Collaborators, 2006). Cross-validation can be time
and resource intensive, because it requires the development of several
alternative models and the comparison of their results.
The predictive validity of a model refers to the model’s ability to
accurately predict future outcomes. This aspect of model validity is par-
ticularly important for the intended use of modeling in regulating MRTPs.
For example, a model developed on the basis of currently available infor-
mation can be linked to future empirical studies of the impact of an MRTP
and validated partly or fully using data from those studies.
Uncertainty Uncertainty accounting refers to the systematic examina-
tion, assessment, and reporting of the uncertainty in model inputs and
assumptions, estimates of model parameters, and summary measures of
the results (Bilcke et al., 2011). There are three important sources of uncer-
tainty in model-based analyses. First, there can be uncertainty about the
structural assumptions of the model, such as assumptions about variables
to include and causation and prediction pathways. The uncertainty about
such assumptions is often difficult to quantify, but their impact on the
conclusions of the analysis may be substantial (Bojke et al., 2009). Second,
there can be uncertainty about model parameter estimates derived from
available studies. Formal statistical measures for this uncertainty can be
derived using meta-analysis methods. However, the assessment of how
this uncertainty propagates through the modeling computations to the
final results continues to be a challenge. Third, there can be variability
among individuals in the population, which can be explained on the basis
of characteristics of these individuals (systematic or explained variation)
or is considered to be random.
The impact of the various forms of uncertainty on the final results of
the model is typically assessed by deterministic or probabilistic sensitivity
analysis. However, because of the extent and variety of uncertainty in a
modeling analysis, a realistic account of its impact on the results of the
analysis continues to be challenging. For example, most sensitivity analy -
ses in practice do not account for the multivariate structure of inputs and
the correlations in this structure. Guidelines on how the results of sen-
OCR for page 237
237
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
sitivity analyses should be reported are currently in development5 (Bilcke
et al., 2011; Stout et al., 2009).
Selection of Comparison Products
The amount of harm reduction claimed by an MRTP sponsor in an
application is a critical issue in deciding whether to issue an order for the
marketing of the MRTP. Harm reduction is inherently relative; a reduc-
tion claim is by definition relative to a comparison product. Selection of
an appropriate comparison product is essential for informed and accurate
decision making. The FSPTCA recognizes this, giving the Secretary of the
Department of Health and Human Services authority to require product
sponsors to compare their product to a commercially marketed represen -
tative product. The choice of appropriate comparison products will be
driven by the type of MRTP being tested, the anticipated claim, and the
study design. And, indeed, the comparison products may differ between
different classes of evidence. However, two reference products come to
the forefront in terms of integration and synthesis of evidence: leading
brands and smoking cessation products.
Leading Brands
Those products that are most commonly used by consumers are likely
to provide a good comparison for products that claim to demonstrate
reduced health risk. “Leading brands” represent a set of products that
accounts for a significant portion of the market and could capture sub -
groups of interest (e.g., low socioeconomic status, who tend to use dis -
count brands, and racial/ethnic minorities, who tend toward menthols).
Using leading brands increases the likelihood that the findings will have
broader applicability to the population, which is crucial given the pub-
lic health standard against which MRTPs are evaluated. Using leading
brands as a comparator also avoids potential mischief in comparing an
MRTP to a product that is little used but may inflate the apparent risk
reduction of the MRTP. In some cases, when desiring a reduced exposure
or risk claim, the comparison product will be a product within the same
product class (e.g., cigarette-like MRTP versus leading brand cigarette),
and thus the comparison is relatively straightforward. In other cases, an
MRTP may make a reduced risk/exposure claim across product classes
(e.g., smokeless MRTP versus leading brand cigarettes). In this case, the
product should also be compared to leading products in its own class
5 ISPOR-SMDM Modeling Good Research Practices Task Force Working Group Part 3,
unpublished data, 2011.
OCR for page 238
238 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
(e.g., smokeless MRTP versus moist snuff). Following this example, a
smokeless MRTP that succeeds in both within- and cross-class compari -
sons against leading brands could reduce risk/exposure for both smokers
and traditional smokeless users. However, it is also possible that a smoke-
less MRTP does pose less risk or exposure than cigarettes but is no differ-
ent than other smokeless products not seeking a claim.
Smoking Cessation
On the opposite end of the spectrum of exposure and risk reduction
is the “gold standard” of smoking cessation (or tobacco cessation in the
case of smokeless tobacco users). This provides an aspirational goal for
risk and exposure for MRTPs—in principle, the closer risks and exposures
from the MRTP are to cessation products, the more confident a regulator
can be in the chances for net public health benefit. Note that the use of
this comparison product is not the same as studying whether the MRTP
acts as an aid to smoking cessation. Rather, the goal is to compare how
the risk or exposure reduction attained with use of the MRTP compares
to smoking cessation of similar duration. It is also important to consider
that for some health conditions, such as acute cardiovascular outcomes
and lung function decline, the benefits of cessation accrue more quickly
than for cancer.
FINDINGS AND RECOMMENDATIONS
In the committee’s view, the fundamental problem that confronts the
FDA is a shortage of credible and reliable evidence about the effects of
MRTPs on both individual and public health. The history of deceptive
behavior by the tobacco industry undermined the trust of the public
as well as the public’s confidence in the industry’s ability to rigorously
conduct studies that will generate the data needed to evaluate these
products. Therefore, the committee’s recommendations are designed to
articulate the minimum standards for producing credible and reliable
evidence to demonstrate that the marketing of an MRTP is consistent
with the protection of public health. The committee articulates a strategy
for the production of scientific evidence by making recommendations in
three areas:
1. Types of evidence and studies
2. Design and integration of studies
3. Governance of studies
OCR for page 239
239
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
Types of Evidence and Studies
Finding 1: Types of Evidence. The public health standard articulated by
the FSPTCA requires collection of scientific evidence from a wide range
of disciplines and research domains. Although the committee respects the
FDA’s independence and discretion in regulating MRTPs, the committee
maintains there is a minimum range of research domains required to
evaluate the effect of MRTPs on individuals and public health. Individual
methods may change as the technology or state of the science may evolve,
but the minimum standards for the domains of evidence will be relevant
regardless of the state of the science in the future.
Recommendation 1: The FDA should require that studies submitted
in support of an MRTP application address all key research domains
needed to forecast and monitor the product’s public health impact,
including:
• product composition and performance;
• addiction potential and likelihood for initiation or persistence of
use;
• human exposure to harmful and potentially harmful constituents;
• perceptions about the product’s effects and likelihood of addic-
tion; and
• effects of the product on human health and surrogates of human
health.
Finding 2: Phased Approach to New MRTPs. Many novel MRTPs are
likely to be developed for marketing in the near future. There are inher-
ent uncertainties and risks with new products that should be addressed.
Risks should be minimized before new products are tested in humans. To
address the risk of new products, a phased approach, similar to the New
Drug Application framework for the regulation and control of new drugs,
is appropriate for the evaluation of new MRTPs. A phased approach will
help the FDA ensure that only products that are unlikely to be unsafe and
have a reasonable expectation of reducing harm relative to conventional
tobacco products will be used in human studies.
Recommendation 2: The FDA should establish guidance that conveys
an expected sequencing of studies, such that preclinical work is com-
pleted and submitted to the FDA before clinical (human subjects)
work commences, and that there is a reasonable expectation based
on preclinical work that a reduction or lack of harm will be seen in
humans.
OCR for page 240
240 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
Finding 3: Clinical Trial Studies. Although the use of randomized con-
trolled trial methods will be constrained for a number of reasons (includ -
ing the practical limitations of study cost, size, and follow-up, and ethical
constraints on randomizing study participants to harmful exposures),
they will continue to play an essential role in creating an evidence base on
the public health effects of MRTPs. Randomized controlled trial methods
can provide highly reliable data on the likelihood of addiction and initia -
tion or cessation of product use. Also, these methods can provide reliable
evidence on human exposure.
Recommendation 3: The FDA should require randomized controlled
trials in the following domains:
• Exposure reduction
• Self-administration of the MRTP
• Effects on use of conventional tobacco products
These randomized controlled trials should include multiple com-
parison products (such as nicotine replacement products, conventional
cigarettes or smokeless tobacco, placebo preparations, and alternative
nicotine delivery systems). These trials should also assess the effect of
the MRTP on human exposure and on human health and surrogates
of human health.
Finding 4: Requirement for Postmarket, Prospective Epidemiologic
Studies. Postmarket studies of MRTPs will be critical to evaluating the
effect of MRTPs on both individuals and the public’s health. In particular,
the prospective cohort design will be an essential tool to validating antici-
pated or claimed effects of marketed MRTPs. These studies have several
important strengths: (1) biochemical tobacco and MRTP exposure can
be assessed at baseline, offering “unbiased” exposure assessment before
health outcomes occur; (2) there is less of a problem with retrospective
recall of product use, beause this information can be summarized at the
start of the study and followed prospectively; (3) changing product use
habits can be monitored as the study progresses; (4) outcomes can be
documented as they occur, and verification is more efficient; and (5) a
wide variety of outcomes can be evaluated in the same study, particu-
larly outcomes that are more common. Furthermore, cohort studies allow
assessment of overall health status and outcomes.
Recommendation 4: The FDA should require prospective epidemio-
logic studies to commence upon issuance of a marketing order to
confirm reduced exposure and reduced risk claims, and to examine
OCR for page 241
241
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
effects of MRTP availability on the population as a whole, includ -
ing the likelihood of initiation and cessation. The FDA should issue
guidance on the design, conduct, and analysis of such studies.
Finding 5: Modeling of Public Health Outcomes. Mathematical model-
ing and simulation analysis provides a complementary approach to
the conduct of empirical studies that can be useful at each stage of the
regulatory process for MRTPs. Model-based analyses can (1) synthesize
the available information from empirical studies of MRTPs; (2) enable
researchers and decision makers to explore complex interactions and
systems that may be impractical to evaluate in empirical studies; (3) allow
researchers and decisions makers to explore “what if” questions relevant
to decision making, which would not be practical to assess in empirical
studies; and (4) be used to make projections about the short- and long-
term effects of the introduction of MRTPs.
Recommendation 5: The FDA should issue guidance on the develop-
ment and use of simulation and modeling approaches to predict pub-
lic health impact through the systematic integration of information
about relevant assumptions and influences. Such approaches should
be tested for robustness with regard to results and assumptions,
they should be public and transparent, and they should be validated
against postmarketing epidemiologic research.
Design and Integration of Studies
Finding 6: Standards for Sampling in MRTP Studies. To have regula-
tory usefulness, studies of MRTPs must be generalizable to the overall
population of interest and to specific populations, including populations
at high risk for tobacco use. Failure to include relevant populations in
studies will result in incomplete evidence on the effect of an MRTP on the
public’s health and, therefore, will be inadequate to support regulatory
decisions about the marketing of MRTPs.
Recommendation 6: The FDA should require studies to include pop-
ulations of special relevance, including (but not limited to)
• users of tobacco products, including users who are and are not
interested in quitting;
• in certain circumstances, nonusers of tobacco products;
• former smokers;
• beginning smokers;
• adolescents; and
OCR for page 242
242 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
• populations at a high risk for tobacco use, including, but not
limited to, those low in socioeconomic status and educational attain -
ment, and certain ethnic minorities.
Finding 7: Quality of Studies. The usefulness of a study to inform a reg-
ulatory decision hinges on the quality and appropriateness of the design.
In many cases, complementary studies might be needed to provide a
breadth of evidence for an informed regulatory decision with appropriate
control of confounders and internal and external validity.
Recommendation 7: For all studies of the effects of MRTPs on human
health and behavior, the FDA should require a range of designs that
are properly powered, balance internal and external validity, and
comprise multiple populations appropriate to the experimental ques-
tions being addressed.
Finding 8: Standards for Good Research Practice. A significant amount
of guidance on minimum standards for scientific studies directly relevant
to the evaluation of MRTPs has already been developed. Guidelines for
formatting, design, conduct, and reporting of science are articulated in
consensus statements, such as the Consolidated Standards of Reporting
Trials (CONSORT) reporting criteria for clinical trials, the Strengthen -
ing the Reporting of Observational Studies in Epidemiology (STROBE)
statement for observational studies, the publication criteria of the Inter-
national Council of Medical Journal Editors, and the reporting criteria of
the International Conference on Harmonization. These existing guidelines
represent robust standards for the conduct of science across many of the
research domains relevant to the evaluation of MRTPs.
Recommendation 8: The FDA should issue guidance to the industry
regarding the format, design, conduct, and reporting of studies in
support of MRTP applications that is based upon current generally
accepted principles for scientific investigation.
Finding 9: Standards for Integration of Evidence. Regulatory decisions
regarding MRTPs will be based on a wide range and variety of scientific
evidence, and the integration of scientific evidence will play a pivotal
role in that decision making. The assessment of MRTPs will typically
require the evaluation and integration of evidence on risks and ben -
efits across multiple diverse outcomes, such as measures of toxicity, bio -
markers, addictiveness, and disease endpoints. Modeling and simulation
approaches are relevant to estimating public health effects of tobacco
and, therefore, the FDA will likely engage in various methods of data
OCR for page 243
243
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
integration, synthesis, and analysis, including, but not limited to, simula -
tion and modeling. It is critical that these approaches are transparent and
reproducible.
Recommendation 9: The FDA should develop and use an approach
to data integration that is explicit and transparent with regard to the
importance of the different outcomes, that uses optimal available
evidence, and that employs objective and reproducible methods for
data integration.
Governance of Studies
Finding 10: Independent Oversight and Conduct of Studies. It has been
established in public records and as a matter of law that the tobacco
industry has engaged in illegal and improper practices, including the
destruction and manipulation of scientific data. As a result, the tobacco
industry is profoundly isolated from the mainstream scientific commu-
nity. Many major universities have policies against acceptance of tobacco
funding, and many high-impact scientific and medical journals will not
accept tobacco industry-supported manuscripts. The consequence of this
isolation is a lack of the expertise and the resources necessary to produce
high-quality science across the range of disciplines to support an applica -
tion to market an MRTP. Use of a trusted third party, particularly for prod-
ucts developed by the tobacco industry, could provide an avenue for the
production of credible evidence needed by the FDA to evaluate tobacco
products. Ultimately, such a research structure could encourage and sup-
port the production and dissemination of credible and reliable evidence
about the effects of tobacco products on the public’s health.
Recommendation 10: MRTP sponsors should consider use of inde-
pendent third parties to undertake one or more key functions,
including the design and conduct of research, the oversight of spe-
cific studies, and the distribution of sponsor funds for research.
Such independent third parties should be approved by the FDA in
advance of the research.
Finding 11: Public Disclosure of Research. Public availability of data
not only builds credibility and public trust, but also benefits the public
because it allows for independent analysis of study methods and data.
The model of Clinicaltrials.gov is particularly compelling and relevant,
and a similar model of public accounting and open disclosure should be
expected of the tobacco industry.
OCR for page 244
244 STUDIES ON MODIFIED RISK TOBACCO PRODUCTS
Recommendation 11: The FDA should require all MRTP sponsors
to place all data generated in the development and marketing of the
MRTP in a public repository selected by the FDA.
Finding 12: Proper Conduct of Research. Standards for the conduct
of science and the protection of human research participants have been
established for biomedical research enterprises not only in academics
but also in commercial research. The FDA has the tools to ensure studies
adhere to established standards in the drug development framework,
which can be applied to the development of MRTPs. Those standards
not only protect human participants, but also build credibility into any
data that are provided to the FDA, particularly by the tobacco industry.
Institutional credibility and trustworthiness is particularly relevant in this
context, given the history of unethical and illegal practices of the tobacco
industry.
Recommendation 12: The FDA should require studies offered in
support of an MRTP application to adhere to established standards
and principles of good research governance, including appropri-
ately qualified investigators, transparency, independent institutional
review board or ethical review, and adherence to the Common Rule
(21 CFR parts 50 and 56).
REFERENCES
Ashley, M. J., and J. E. Cohen. 2003. What the public thinks about the tobacco industry and
its products. Tobacco Control 12(4):396-400.
Bilcke, J., P. Beutels, M. Brisson, and M. Jit. 2011. Accounting for methodological, structural,
and parameter uncertainty in decision-analytic models: A practical guide. Medical Deci-
sion Making 31(4):675-692.
Bojke, L., K. Claxton, M. Sculpher, and S. Palmer. 2009. Characterizing structural uncer-
tainty in decision analytic models: A review and application of methods. Value Health
12(5):739-749.
CISNET Breast Cancer Collaborators. 2006. The impact of mammography and adjuvant
therapy on U.S. breast cancer mortality (1975-2000): Collective results from the cancer
intervention and surveillance modeling network. JNCI Monographs 2006(36):1-126.
Cummings, K. M. 2003. A promise is a promise. Tobacco Control 12(2):117-118.
Cummings, K. M., C. Morley, and A. Hyland. 2002. Failed promises of the cigarette industry
and its effect on consumer misperceptions about the health risks of smoking. Tobacco
Control 11(Suppl. 1):i110-i117.
Cummings, K. M., A. Brown, and R. O’Connor. 2007. The cigarette controversy. Cancer Epi-
demiology, Biomarkers & Prevention 16(6):1070-1076.
Glickman, S. W., J. G. McHutchison, E. D. Peterson, C. B. Cairns, R. A. Harrington, R. M.
Califf, and K. A. Schulman. 2009. Ethical and scientific implications of the globalization
of clinical research. NEJM 360(8):816-823.
OCR for page 245
245
DECISION MAKING AND OVERSIGHT OF MRTP STUDIES
Harris Interactive. 2003. Attitudes to government regulation vary greatly for different indus -
tries; The Harris Poll #19. http://www.harrisinteractive.com/vault/Harris-Interactive-
Poll-Research-Attitudes-to-Government-Regulation-Vary-Greatly-for-Different-
Industries-2003-04.pdf (accessed December 5, 2011).
Harris Interactive. 2010. Oil, pharmaceutical, health insurance, and tobacco top the list of industries
that people think should be more regulated; The Harris Poll #149. http://www.harrisinteractive.
com/vault/HI-Harris-Poll-Industry-Regulation-2010-12-02.pdf (accessed December 5,
2010).
Keating, T. J. 2001. Lessons from the recent history of the Health Effects Institute. Science,
Technology, & Human Values 26(4):409-430.
Mendez, D., S. P. Marcus, P. I. Clark, S. J. Leischow, and T. F. Pechacek. 2008. Tobacco mod -
elers conference proceedings. In Tobacco modelers conference. Ann Arbor: University of
Michigan.
NCI (National Cancer Institute). 2008. The role of the media in promoting and reducing tobacco
use. Edited by R. M. Davis, NCI tobacco control monograph series. Bethesda, MD: U.S.
Department of Health and Human Services, National Institutes of Health, National
Cancer Institute.
NRC (National Research Council). 1991. Improving information for social policy decisions: The
uses of microsimulation modeling. Vol. 1. Washington, DC: National Academy Press.
Rees, V. W., J. M. Kreslake, K. M. Cummings, R. J. O’Connor, D. K. Hatsukami, M. Parascandola,
P. G. Shields, and G. N. Connolly. 2009a. Assessing consumer responses to potential
reduced-exposure tobacco products: A review of tobacco industry and independent
research methods. Cancer Epidemiology, Biomarkers & Prevention 18(12):3225-3240.
Rees, V. W., J. M. Kreslake, R. J. O’Connor, K. M. Cummings, M. Parascandola, D. Hatsukami,
P. G. Shields, and G. N. Connolly. 2009b. Methods used in internal industry clinical
trials to assess tobacco risk reduction. Cancer Epidemiology, Biomarkers & Prevention
18(12):3196-3208.
Rutter, C. M., A. M. Zaslavsky, and E. J. Feuer. 2011. Dynamic microsimulation models for
health outcomes. Medical Decision Making 31(1):10-18.
Sonnenberg, F. A., and J. R. Beck. 1993. Markov models in medical decision making: A practi-
cal guide. Medical Decision Making 13(4):322-338.
Stout, N. K., A. B. Knudsen, C. Y. J. Kong, P. M. McMahon, and G. S. Gazelle. 2009. Calibra -
tion methods used in cancer simulation models and suggested reporting guidelines.
Pharmacoeconomics 27(7):533-545.
TPSAC (Tobacco Products Scientific Advisory Committee). 2011. Menthol cigarettes and public
health: Review of the scientific evidence and recommendations. Tobacco Products Scientific
Advisory Committee.
Vanni, T., J. Karnon, J. Madan, R. G. White, W. J. Edmunds, A. M. Foss, and R. Legood. 2011.
Calibrating models in economic evaluation: A seven-step approach. Pharmacoeconomics
29(1):35-49.
Weinstein, M. C., B. O’Brien, J. Hornberger, J. Jackson, M. Johannesson, C. McCabe, and
B. R. Luce. 2003. Principles of good practice for decision analytic modeling in health-
care evaluation: Report of the ispor task force on good research practices—modeling
studies. Value Health 6(1):9-17.
OCR for page 246
