Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 139
6
Essential Priorities for a
Robust CER Enterprise
abstract: This chapter presents the committee’s findings and recommenda-
tions for ensuring effective implementation of comparative effectiveness
research (CER) and its translation into health care delivery. A short-term
priority research agenda alone will not fulfill the potential of CER to im-
prove the health of Americans and the quality of health care in the United
States. The committee strongly recommends that Congress and the Secre-
tary of Health and Human Services act to establish a sustainable strategy
to coordinate government CER activity. The organizational and scientific
challenges of CER are immense and the case for a strong, coordinating
authority is compelling. Effective implementation of the CER agenda will
involve collaboration among multiple government agencies and numer-
ous professional disciplines and areas of expertise. The relevant areas of
research encompass the complete continuum of health care services, all
age groups, numerous disease conditions and health technologies, diverse
health care settings, and the organization of health care delivery itself. No
single U.S. research agency or organization possesses the breadth of exper-
tise necessary to address this considerable scientific challenge. Four CER
program priorities merit high-level attention and coordination: () mean-
ingful participation of consumers, patients, and caregivers; () building of
robust data and information systems as well as research and innovation
in the methods of CER research; () development and support of a highly
skilled CER workforce; and () vigorous support of research and efforts
to translate CER knowledge into everyday clinical practice.
Early in its deliberations, the Institute of Medicine (IOM) committee
agreed that the nation’s investment in comparative effectiveness research
OCR for page 139
0 INITIAL NATIONAL PRIORITIES FOR CER
(CER) should go beyond recommending the individual CER topics sug-
gested in the previous chapter. A short-term priority research agenda alone
will not fulfill the potential of CER to improve the health of Americans and
the quality of health care in the United States. The most important priority
of all should be the building of a broad and supportive infrastructure to
carry out a sustainable national CER strategy. Congress and the Secretary
of Health and Human Services (HHS) must take concerted steps to establish
a robust CER enterprise. This chapter refers to this effort, its coordination,
and its recommended tasks as the “CER Program.”
Rather than develop a comprehensive blueprint, the committee fo-
cused on four essential program priorities that the HHS Secretary should
embrace: (1) meaningful consumer, patient, and caregiver participation (in
addition to other stakeholders); (2) investing in building robust data and
information systems and in strengthening the research infrastructure for
conducting new prospective CER studies; (3) investing in development,
deployment, and support of a highly skilled CER workforce; and (4) sup-
porting a vigorous translational effort to help bring CER knowledge into
everyday clinical decision making.
The objective of this chapter is to describe these essential priorities. It
begins with a section outlining the imperative for effective coordination
of CER activities. The rest of the chapter reviews the other four program
priorities listed above. See Chapter 4 for the committee’s recommendations
for CER priority setting.
THE IMPERATIvE FOR EFFECTIvE COORDINATION
OF THE CER ENTERPRISE
The committee strongly agreed that Congress should direct the HHS
Secretary to implement a sustainable strategy to coordinate government
CER activity including the Agency for Healthcare Research and Quality
(AHRQ), Centers for Disease Control and Prevention, Centers for Medicare
& Medicaid Services (CMS), National Institutes of Health (NIH), Food and
Drug Administration (FDA), Department of Defense (DOD), and Veterans
Administration.
Recommendation 5: The HHS Secretary should establish a mechanism—
such as a coordinating advisory body—with the mandate to strategize,
organize, monitor, evaluate, and report on the implementation and im-
pact of the CER Program.
OCR for page 139
ESSENTIAL PRIORITIES
Organizational Challenges
Broad Research Scope
The organizational and scientific challenges of CER are immense and
the case for a strong, coordinating authority is compelling. This report’s
recommended research topics1 encompass not only the complete continuum
of health care services (prevention, early detection, diagnosis, treatment,
rehabilitation, palliation, end-of-life care), but also the effective organiza-
tion of health care delivery itself. The high priority CER topics also span
all age groups, from infants to adolescents to young, middle age, and older
adults; numerous disease conditions; health care technologies (drugs, imag-
ing, surgery, devices); and diverse health care settings. And, the investigative
means include an array of complex methods including randomized clinical
trials and observational studies.
No current research organization in the United States possesses the
breadth of expertise necessary to address this considerable scientific chal-
lenge. High-level coordination and funding authority across this broad
front is of paramount importance.
Scientific Rigor
An essential component of CER is the study of representative popula-
tions in real-world clinical settings. This demands a wide array of study de-
signs including systematic reviews and meta-analysis, observational analytic
methods, modeling, clinical trials, and others. The field must set uniform
quality standards at each phase (i.e., priority setting, design and analysis of
observational and experimental studies, interpretation and dissemination)
and maintain a highly skilled workforce.
Objectivity
Objectivity will be central to the public’s trust and confidence in the
integrity of the CER Program. Conflict of interest and bias in clinical
research—published in even the most respected medical journals—is well-
documented (IOM, 2009b). Selective reporting or publication bias is com-
mon. Positive findings are more likely to be published than negative results
(Chan et al., 2004; Dickersin, 2005; Dickersin and Min, 1993; Rising et
al., 2008; Turner et al., 2008). In addition, there have been significant in-
stances in which leading journals have not sufficiently enforced disclosure
1 See Chapter 5.
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
requirements for authors and reviewers (Schwartz et al., 2008; Weinfurt
et al., 2008).
CER is as vulnerable to bias and conflict of interest as any other area
of medical research. The ultimate value of the CER enterprise will rest, in
part, on vigilant attention to these issues. A 2009 IOM report, Conflict
of Interest in Medical Research, Education, and Practice, recommends
principles to inform the design of policies to identify, limit, and manage
conflicts of interest in health care research. The committee urges that the
CER Program be constituted and managed in accordance with the recom-
mendations of this report.
Public-Private Collaboration
The U.S. health care system has substantial resources—both public
and private—to contribute to the CER effort. These resources include the
private health care organizations that provide care for potential enrollees
in CER studies. At present, experience in developing and maintaining such
collaborative relationships is very limited within the federal government. To
lead this coordinated effort, the committee agreed, will require an organiza-
tion that is highly, preferably solely, focused on achieving the goals of CER.
A national program of CER must engage the public—including all of the
stakeholders—at all levels of its organization if it is to fulfill its potential to
improve health care outcomes and to reduce unnecessary health care costs,
which are both urgent needs.
Sustainability
The CER Program needs sustained and predictable funding beyond the
American Recovery and Reinvestment Act (ARRA) of 2009 (P.L. 111-5)
to achieve its objectives. To ensure research activities that truly embrace
the definition of CER, the ARRA funds—and subsequent funding to sup-
port CER—should flow through a CER coordinating authority directly to
grantees, through federal agencies, or both.
MEANINGFuL CONSuMER, PATIENT, AND
CAREGIvER ENGAGEMENT2
In Chapter 4, the committee urged that consumers, patients, and care-
givers be active participants in setting research priorities. In this chapter,
the committee recommends that consumers also be integrally involved in
2 In this chapter, the term “consumer” is used to represent not only consumers, but also
patients and their families and caregivers.
OCR for page 139
ESSENTIAL PRIORITIES
the governance of the CER Program, the framing of research questions
and research protocols, peer review of systematic reviews and monitoring
of trials, and interpreting and disseminating the results of CER studies
to ensure that new knowledge improves everyday clinical practice. CER
will not achieve its basic objectives unless it embraces—and acts upon—a
patient-centered mindset.
Centering on the patient is fundamental to high-quality health care
(IOM, 2001). Patient-centered health care demands that CER be developed
and applied with respect to each patient’s unique needs, beliefs, and values.
There is strong evidence that many consumers want to be involved in deci-
sion making about their care (President’s Commission for the Study of Ethi-
cal Problems in Medicine and Biomedical and Behavioral Research, 1982).
Many—but not all—patients expect to make their own decisions about
diagnosis and treatments and look to their health providers for support in
interpreting and assessing the available information (Deber et al., 1996;
Degner and Russell, 1988; Guadagnoli and Ward, 1998; Mansell et al.,
2000; Mazur and Hickam, 1997). Yet, even the most sophisticated health
care consumers often struggle to find the information that is relevant to
their specific health-related questions and particular clinical circumstances
(IOM, 2008).
Recommendation 6: The CER Program should fully involve consum-
ers, patients, and caregivers in key aspects of CER, including strategic
planning, priority setting, research proposal development, peer review,
and dissemination.
• The CER Program should develop strategies to reach out to, engage,
support, educate, and, as necessary, prepare consumers, patients,
and caregivers for leadership roles in these activities.
• The CER Program should also encourage broad participation in
CER in order to create a representative evidence base that could help
identify health disparities and inform decisions by patients in special
population groups.
Consumers’ Role in Informing and Framing the Research
Experts and consumers often have different perspectives on the ques-
tions that research should answer. Clinicians and patients do not always
consider the same factors when weighing the tradeoffs posed by important
health care alternatives (Entwistle et al., 1998). To ensure that the fruits of
CER support consumers’ health care decision making, the CER Program
should focus on the questions and perspectives of patients as well as their
health care providers.
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
Many researchers, acknowledging the importance of consumers’ con-
tribution to framing research questions, advocate for decision makers to
participate directly in formulating research questions (IOM, 2008). Con-
sumers can inform investigators’ decisions about which patient outcomes to
measure, patient populations to study (including important subgroups and
relevant comorbidities), and interventions to compare, among other issues
(Andejeski et al., 2002a,b; Hubbard et al., 2008; Saunders et al., 2008).
Diabetes researchers, for example, have reported that involving patients
is particularly helpful at keeping their research relevant and applicable to
real-world settings (Lindenmeyer et al., 2007). Researchers responding to
a survey on consumer involvement in randomized controlled studies in
the United Kingdom reported that involving consumers helped to refine
research questions and make the trials more relevant to patients’ needs
(Hanley et al., 2001). Nevertheless, opportunities for public input into
clinical research remain rare.
In breast cancer, the involvement of consumers at all levels of decision
making at the DOD Breast Cancer Research Program including vision
setting, and peer and programmatic review, has proven valuable to the
research process, resulted in an educated and engaged consumer force, and
influenced clinical research beyond the DOD research program and beyond
breast cancer (IOM, 1997).
Cultivating Consumers’, Patients’, and Caregivers’ Participation in CER
Community-based participatory research refers to research that in-
volves community members or recipients of interventions in all phases of
the research process, starting with a research topic of importance to the
community. Numerous researchers have advocated that community-based
participatory research is key to improving the relevance of clinical research,
especially research on health care disparities (Faridi et al., 2007; Fretheim
et al., 2006; Jones and Wells, 2007; Minkler et al., 2003; Omenn, 1999;
Zerhouni, 2005). Nevertheless, consumer participation in research is not
the norm and there is no agreed-upon model for conducting community-
based research effectively (Johnson et al., 2008). Researchers do not know
how to meaningfully engage consumers in their work or to whom to turn
for advice on a consumer representative. If scientists choose the consumer
representatives, the representatives may not represent the consumer view-
point. In addition, independent consumer groups face numerous challenges
when their members want to contribute to research. The CER Program
should identify best practices for consumer involvement and set standards
for the key competencies required for consumer participation in CER.
Consumers will likely need appropriate information and education
about CER to contribute meaningfully (Hubbard et al., 2008; Saunders
OCR for page 139
ESSENTIAL PRIORITIES
et al., 2008). The CER Program must reach out first to engage consumers
and then to support, educate, and, as necessary, prepare them for their
roles. Several programs have already been developed for this purpose.
For example, a special initiative of the U.S. Cochrane Center, Consumers
United for Evidence-based Healthcare, has developed a web-based course
to help consumers understand the fundamentals of evidence-based health
care (United States Cochrane Center Consumers United for Evidence-based
Healthcare, 2009). Other programs include the National Breast Cancer
Coalition’s Project LEAD Institute, a science education program for breast
cancer advocates, and Quality Care Project LEAD; the DOD Breast Cancer
Research Program’s peer review process, which involves consumer advo-
cates; the FDA’s Office of Special Health Initiatives, which trains patient
representatives to participate on Advisory Panels (FDA, 2009); and the
United Kingdom’s national advisory group, which educates the public about
involvement in research (INVOLVE, 2009).3
To achieve meaningful consumer participation, the CER program
should incorporate the following:
• Substantial consumer representation in program governance
• Focus groups, forums, and citizen juries. Public meetings should be
well-publicized and held at convenient times and locations
• Well-publicized web-based surveys (see Chapter 4 for how the com-
mittee solicited public nominations of CER topics)
• Educational programs offered through public symposia and semi-
nars. Active consumer participants should have formal training
opportunities and be compensated for their time
Public Trust
Some members of the public have voiced concerns that CER research
may lead to health care rationing and inappropriate limits on patients’
treatment choices (Meier, 2009). Engaging consumers in CER, and building
the case for the value of CER, could help improve the public’s trust in the
U.S. research enterprise, because the communication is expanded to be in-
clusive, rather than exclusive, among the key decision makers (IOM, 2002).
In fact, consumers may have the most credibility in conveying information
about CER back to the general public and help in explaining health and
health care delivery (Oliver et al., 2008).
A public that is more informed about the processes and value of CER
is likely to have greater enthusiasm and confidence in both the research
3 See http://www.stopbreastcancer.org/index.php?option=com_content&task=view&id=395
for further information on Project LEAD.
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
and the research community (Academy of Medical Sciences, 2006) and
may be more likely to participate in CER, either actively or passively as
research subjects. The CER Program should, therefore, work to lower bar-
riers to active public participation in planning research, such as the lack of
adequate financial support to allow for travel and to compensate for the
time required to participate (Staniszewska et al., 2007).
RObuST DATA AND INFORMATION SySTEMS
As noted earlier and described in greater detail in Chapter 2, CER
comprises a broad spectrum of established and emerging research methods
including systematic reviews of existing evidence, observational research,
and experimental studies such as clinical trials (each described in this sec-
tion). A critical first step in launching a comparative effectiveness study is
to identify the most appropriate design for the type of research question
being asked (IOM, 2008). Every study design has limitations and no single
method is ideal for addressing questions of comparative effectiveness. Each
study should be well-designed to ensure scientific rigor and minimal bias.
Systematic Reviews
Systematic reviews address a specific research question by identifying,
selecting, assessing, and summarizing the existing body of evidence. Indi-
vidual research studies often do not provide definitive answers to clinical
effectiveness questions (IOM, 2008). If conducted properly, a systematic
review should make obvious the gap between what is known about the
effectiveness of an intervention and what clinicians and patients want to
know. Thus, systematic reviews provide a central link between research
evidence and clinical decision making. If the systematic review is both
scientific and transparent, researchers and decision makers should be able
to interpret the evidence, to know what is not known, and to describe the
extent to which the evidence is applicable to clinical practice and particular
patients. As such, systematic reviews are integral to framing research ques-
tions for future study regarding comparative effectiveness.
To date, the quality of systematic reviews has been variable and some
published reviews have been unreliable. Criticisms include a confusing
array of schemes for grading evidence in the literature, hierarchies of evi-
dence that may not account for the true quality of studies, no disclosure of
potential bias or conflict of interest, and a failure to use existing standards
for reporting methods and results in systematic reviews. In Knowing What
Works in Health Care, the IOM recommended that the HHS Secretary only
fund systematic reviews that commit to and consistently meet evidence-
OCR for page 139
ESSENTIAL PRIORITIES
based, methodologic standards (IOM 2008).4 This principle should also be
followed in HHS funding of CER studies.
Clinical Trials
Fundamental questions of comparative effectiveness often require head-
to-head comparisons of alternative interventions using randomized assign-
ment to the interventions to be compared. Randomized controlled trials
(RCTs) are the gold standard for determining effectiveness because they
minimize selection bias, that is, the likelihood that study participants will
be given a treatment related to their prognosis such as comorbidities. RCTs
have answered many important comparative effectiveness questions. The
ALLHAT trial, for example, compared the benefits and harms of different
forms of antihypertensive therapies (Furberg et al., 2002).
Clinical trials, however, cannot address many comparative effectiveness
questions because of cost, ethical considerations, or other issues. RCTs
are expensive because they involve careful follow-up of study participants
as well as multiple clinical centers and investigators and centralized data
coordination. Ethical considerations preclude trials of many types of in-
terventions. For example, a randomized study comparing prophylactic
mastectomy to “watchful waiting” in women positive for BRCA1 is very
unlikely.
Smaller scale trials with small study populations conducted at a single
site are often not representative of real-world clinical settings. This is not
to say that small single center studies should never be done. For example,
entirely new research questions should be addressed using observational
studies to begin with, progressing to small scale trials and finally testing in
the context of large scale trials and real-world settings.
Studies of intervention effectiveness and prognosis often require years
of follow-up (e.g., interventions for chronic diseases and interventions in
children). As a result, such research is subject to high drop-out rates and
missing data. The findings must be interpreted cautiously. Moreover, as time
goes by, the technology being studied may change, its use may improve, or,
in the case of medications, the indications for use may broaden (Kent and
Hayward, 2007).
Registration trials conducted for the purpose of FDA approval are
unlikely to detect uncommon adverse effects because they typically involve
relatively few subjects and often address short-term outcomes. The study
4 The Medicare Improvements for Patients and Providers Act of 2008 (P.L. 110-275 Sec.
304) directed the HHS Secretary to contract with the IOM for the purpose of identifying such
standards and reporting the results of this effort to Congress. This study is scheduled to begin
the summer of 2009.
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
population for FDA pre-approval and marketing trials is often younger
and healthier than the target population of the health care intervention.
Comparator interventions in these trials may not reflect the comparisons of
interest to clinicians and patients because the comparator is often a placebo
or an atypical dose of a competing drug.
Increasingly, trialists are applying methods to adapt clinical trials to
real-world conditions. Methods to recruit and follow patients efficiently
and to adapt trial designs to accommodate changing technologies are being
incorporated in the design of randomized trials (Berry, 2003; Godwin et al.,
2003). These methodologies should be refined and applied to meet the need
for stronger, more applicable comparative effectiveness trials.
Observational Research
Observational studies can address gaps in the evidence when a random-
ized trial design is not practical. Observational research includes prospec-
tive and retrospective cohort studies, case-control studies, and case series
analyses. In observational studies, the researcher does not intervene in pa-
tient care but observes the process of patient care and its outcomes as they
occur in everyday life. Well-characterized cohort studies are particularly
useful. In the Women’s Health Initiative, for example, this method was
used to identify predictors of disease, medication-related outcomes, and
factors associated with health disparities in women ages 50 to 79 years old
(National Heart, Lung, and Blood Institute, 2009). Case-control studies
are useful for identifying risk factors for rare events such as deep venous
thrombosis during long-distance travel or harm from interventions (Aryal
and Al-khaffaf, 2006).
Observational studies are typically most appropriate for answering
questions related to prognosis, diagnostic accuracy, incidence, prevalence,
and etiology (Chou and Helfand, 2005; Tatsioni et al., 2005). They have
the potential to address gaps in randomized trial evidence by including
larger, more representative populations to identify rare or long-term adverse
effects.
Observational studies that link process of care datasets (such as admin-
istrative claims data) to outcomes datasets (such as national death indexes)
provide excellent opportunities to study both health services utilization and
health outcomes, as discussed in the next section.
Despite their potential advantages, however, observational studies are
more subject to bias than randomized trials, and the decision to rely on
data from observational studies must be weighed against the possibility of
misleading results. The main form of bias (selection bias) occurs when the
factors causing a person to experience the intervention are associated with
the patient’s prognosis.
OCR for page 139
ESSENTIAL PRIORITIES
Innovation Is Needed in CER Methods
Recommendation 7: The CER Program should devote sufficient re-
sources to research and innovation in the methods of CER, including
the development of methodological guidance for CER study design
such as the appropriate use of observational data and more informa-
tive, practical, and efficient clinical trials.
There is a significant need for new and better research methods for
studying comparative effectiveness (IOM, 2007; McClellan and Benner,
2009; Rawlins, 2008; Tunis, 2009). Current study designs, both experimen-
tal and nonexperimental, must be further refined if CER is to be scientifically
valid, efficient, and credible. In systematic reviews, for example, research is
needed on how to identify and use evidence from observational studies on
intervention effectiveness, and also on how to assess a heterogeneous body
of evidence (IOM, 2008). New analytic techniques are needed to evaluate
the effects of bias due to confounding when assessing comparative effec-
tiveness using large observational datasets. Many fundamental questions of
comparative effectiveness relate to small but clinically important differences
in treatment effects that cannot be detected by current nonexperimental
methods (Tunis, 2009). Clinical trials will always be essential to CER, but
more efficient, larger, simpler, and pragmatic designs are needed.
The Potential of Existing Data
CER may also draw from analyses of existing data, such as that held
by payers, health care delivery systems, and electronic health records.
ARRA’s $40 billion support for advancing health information technology
and implementing an interoperable electronic health record system with
compatible data definitions and formats can help make these ambitious
aspirations a reality (Office of National Coordinator for Health Informa-
tion Technology, 2009).
Claims data from large national insurers, electronic health records
maintained by large integrated health systems, data collected through
practice-based research networks, and patient registry data hold tremen-
dous potential for CER. Harnessing these sources of existing data could
markedly enhance the timeliness and value of CER. Existing data sources
can be used for many research purposes: to study prognosis, risks and
harms, and etiology of disease (Cupples et al., 1988); to analyze trends
over time and capture long-term outcomes (Fung et al., 2004); to examine
the causes of geographic variation (Wennberg and Fisher, 2008); to analyze
racial and ethnic disparities in both access to and outcomes of health care
(Peterson and Yancy, 2009); to study low prevalence conditions (many
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
BOX 6-1
IOM Recommendations for Changes to the HIPAA Privacy
Rule and Associated Guidance Beyond the HIPAA Privacy
Rule: Enhancing Privacy, Improving Health through Research
A. HHS should reduce variability in interpretations of the HIPAA Privacy
Rule in health research by covered entities, Institutional Review Boards
(IRBs) and Privacy Boards through revised and expanded guidance and
harmonization.
1. HHS should develop a dynamic, ongoing process to increase empirical
knowledge about current “best practices” for privacy protection in respon-
sible research using protected health information (PHI), and promote the
use of those best practices.
2. HHS should encourage greater use of partially deidentified data called
“limited datasets” and develop clear guidance on how to set up and comply
with the associated data use agreements more efficiently and effectively,
in order to enhance privacy in research by expanding use and usability of
data with direct identifiers removed.
3. HHS should clarify the distinctions between “research” and “practice” to
ensure appropriate IRB and Privacy Board oversight of PHI disclosures for
these activities.
4. HHS guidance documents should simplify the HIPAA Privacy Rule’s provi-
sions regarding the use of PHI in activities preparatory to research and
harmonize those provisions with the Common Rule, in order to facilitate
appropriate IRB and Privacy Board oversight of identification and recruit-
ment of potential research participants.
B. HHS should develop guidance materials to facilitate more effective use
of existing data and materials for health research and public health
purposes.
methods, use and analysis of large datasets, cost-effectiveness analysis,
clinical prediction rules, measurement of patient-reported and clinical
outcomes, and communicating research findings to patients, providers,
and others. The CER Program will have to ensure the participation of
individuals with a sound foundation in these areas.
Current Workforce Capacity
The significant increase in CER activity will create a substantial need
for the types of expertise just described. Gauging the capacity of the current
CER workforce is difficult because so many disciplines are involved and so
OCR for page 139
ESSENTIAL PRIORITIES
1. HHS should develop guidance that clearly states that individuals can au-
thorize use of PHI stored in databases or associated with biospecimen
banks for specified future research under the HIPAA Privacy Rule with
IRB/Privacy Board oversight, as is allowed under the Common Rule, in
order to facilitate use of repositories for health research.
2. HHS should develop clear guidance for use of a single form that permits
individuals to authorize use and disclosure of health information in a clinical
trial and to authorize the storage of their biospecimens collected in conjunc-
tion with the clinical trial, in order to simplify authorization for interrelated
research activities.
3. HHS should clarify the circumstances under which DNA samples or se-
quences are considered PHI, in order to facilitate appropriate use of DNA
in health research.
4. HHS should develop a mechanism for linking data from multiple sources so
that more useful datasets can be made available for research in a manner
that protects privacy, confidentiality, and security.
C. HHS should revise provisions of the HIPAA Privacy Rule that entail heavy
burdens for covered entities and impede research without providing
substantive improvements in patient privacy.
1. HHS should reform the requirements for the accounting of disclosures of
PHI for research.
2. HHS should simplify the criteria that IRBs and Privacy Boards use in mak-
ing determinations for when they can waive the requirements to obtain au-
thorization from each patient whose PHI will be used for a research study,
in order to facilitate appropriate authorization requirements for responsible
research.
SOURCE: IOM (2009a).
many educational pathways to the field exist. For these reasons, no one has
yet analyzed the current workforce to see if it is sufficient to respond to the
ARRA mandate for expanding CER. Nonetheless, ARRA’s infusion of $1.1
billion into CER will clearly stress the limits of the current CER workforce.
ARRA appropriations increased AHRQ’s CER budget tenfold. Aggregate
current NIH spending on CER is not known, but the Institutes will receive
at least an additional $400 million to conduct CER.
Recommendation 9: The CER Program should develop and support
the workforce for CER to ensure the nation’s capacity to carry out the
CER mission. Important next steps include the following:
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
• Development of a strategic plan for research workforce development
• Long-term, sufficient funding for early career development including
expanding grants for graduate and postgraduate training opportuni-
ties in comparative effectiveness methods as well as career develop-
ment grants and mid-career merit awards
Ensuring a Highly Skilled CER Workforce
The committee agreed that, at the outset, the CER Program should de-
velop a strategic plan for research workforce development. The plan should
include assessments of both the capacity of the current workforce to carry
out the Program’s research agenda and the capacity and effectiveness of
current training programs for producing researchers with the relevant skills.
Developing an adequate CER workforce will involve the training, deploy-
ment, and collaboration of a significant number of professional disciplines.
Data on education paths and training programs for CER investigators are
scarce.
The NIH Roadmap for Medical Research, together with the Clini-
cal and Translational Science Consortium are two mechanisms by which
workforce development can be efficiently achieved (National Center for
Research Resources, 2009; NIH, 2009). Training grants, such as K12, K30,
and T32, should incorporate concepts of CER in their curricula exposing
young scientists to CER and expanding the opportunities for participation
in CER.
CER is a fast-growing field that has experienced changes over time.
At the present state of development of CER, it appears to be growing as a
cohesive discipline. However, the career path is ill defined, and other areas
of clinical research compete for the best and the brightest investigators.
To be attractive to them, the field needs sustainable research funding and
must adhere to high standards of research quality and scientific integrity, be
open to new ideas and people, and provide excitement about the potential
to contribute to health research and health care practice overall. The CER
Program should secure long-term, sufficient funding for career development
including expanding grants for graduate and postgraduate training oppor-
tunities in comparative effectiveness methods, as well as career development
grants and mid-career merit awards. Without adequate training and secure,
stable financial support, talented investigators are likely to pursue other
areas of research. Undoubtedly, a stable funding stream for CER will attract
investigators to CER, as will a sense that the nation places a high priority
on CER as a partial but important part of paying for health care reform
and improving the quality of care.
OCR for page 139
ESSENTIAL PRIORITIES
bRINGING KNOWLEDGE INTO PRACTICE
Many stakeholders and members of the public asked the committee to
prioritize CER topics related to the comparative effectiveness of methods
for bringing proven health care interventions into everyday clinical practice
(see Chapter 5). Dougherty and Conway have proposed that three steps in
knowledge translation must occur before research can improve health care
quality and value: (1) translation of basic biomedical science into clinical ef-
ficacy knowledge, (2) translation of clinical efficacy knowledge into clinical
effectiveness knowledge, and (3) translation of clinical effectiveness knowl-
edge into health system improvement (Dougherty and Conway, 2008).
Biomedical research has traditionally focused on steps one and two. The
Clinical and Translational Science Consortium is now beginning to expand
research networks and emphasize community engagement. But, the health
care system will not benefit from CER without the third translational step,
and more effort can be made by the Consortium to assess the integration of
new findings into practice and their impact on health outcomes.
The CER Program should require researchers to publish all federally
funded CER studies and make the research readily available to the pub-
lic. Health care professionals and patients must use CER results to make
informed decisions that integrate the best available evidence, the patients’
preferences, and specific characteristics of the patient (Mattews, 2009;
Weinstein et al., 2007).
Recommendation 10: The CER Program should promote rapid adop-
tion of CER findings and conduct research to identify the most effective
strategies for disseminating new and existing CER findings to health
care professionals, consumers, patients, and caregivers and for helping
them to implement these results in daily clinical practice.
The American health research infrastructure lacks a systematic way to
translate knowledge from research to practice. The translation of research
findings into practice is slow and incomplete. Many barriers exist: perverse
reimbursement incentives, physician perceptions about patients’ expecta-
tions, and patients’ concerns about denials of care or reluctance to question
clinicians (Shojania and Grimshaw, 2005). These barriers and others should
be addressed and, insofar as possible, overcome. Knowledge translation
research must be a high priority.
CONCLuSION
In summary, the HHS Secretary’s CER agenda will fall far short of its
potential without effective coordination and governance of the enterprise.
OCR for page 139
0 INITIAL NATIONAL PRIORITIES FOR CER
The research agenda will involve a broad array of study designs, the full
range of health care services, and an extensive corps of experts in diverse
professional disciplines. However, an ambitious research enterprise alone
will not improve health care in the United States without the Secretary’s
attention to high fidelity translation of knowledge into practice. Moreover,
consumers, patients, and caregivers as well as their health care providers
must be involved in all aspects of CER to ensure its relevance to everyday
health care delivery.
The $1.1 billion ARRA investment in CER is an unprecedented vote of
confidence in patient-centered research. The CER program should be held
accountable to its mission. Sustained program evaluation and continuous
quality improvement must be a bedrock feature of the enterprise.
REFERENCES
Academy of Medical Sciences. 2006. Personal data for public good: Using health information
in medical research. http://www.acmedsci.ac.uk/images/project/Personal.pdf (accessed
August 28, 2008).
Andejeski, Y., I. T. Bisceglio, K. Dickersin, J. E. Johnson, S. I. Robinson, H. S. Smith, F. M.
Visco, and I. M. Rich. 2002a. Quantitative impact of including consumers in the scientific
review of breast cancer research proposals. Journal of Women’s Health & Gender-Based
Medicine 11(4):379-388.
Andejeski, Y., E. S. Breslau, E. Hart, N. Lythcott, L. Alexander, I. Rich, I. Bisceglio, H. S.
Smith, and F. M. Visco. 2002b. Benefits and drawbacks of including consumer reviewers
in the scientific merit review of breast cancer research. Journal of Women’s Health &
Gender-Based Medicine 11(2):119-136.
Aryal, K. R., and H. Al-khaffaf. 2006. Venous thromboembolic complications following air
travel: What’s the quantitative risk? A literature review. European Journal of Vascular
and Endovascular Surgery 31(2):187-199.
Bell, C. L., J. Davis, R. C. Harrigan, E. Somogyi-Zalud, M. K. G. Tanabe, and K. H. Masaki.
2009. Factors associated with place of death for elderly Japanese American men: The
Honolulu heart program and Honolulu-Asia aging study. Journal of the American Geri-
atrics Society 57(4):714-718.
Berry, D. A. 2003. Statistical innovations in cancer research. In Cancer Medicine th Edition,
Edited by J. Holland, E. Frei, D. W. Kufe, R. E. Pollock, R. R. Weichselbaum, R. C. Bast
and T. S. Gansler. Hamilton: BC Decker. Pp. 465-478.
Braunstein, J. B., N. S. Sherber, S. P. Schulman, E. L. Ding, and N. R. Powe. 2008. Race, medi -
cal researcher distrust, perceived harm, and willingness to participate in cardiovascular
prevention trials. Medicine 87(1):1-9.
Brown, D. R., M. N. Fouad, K. Basen-Engquist, and G. Tortolero-LUna. 2000. Recruitment
and retention of minority women in cancer screening, prevention and treatment trials.
Annals of Epidemiology 10:S13-S21.
Brown, J., J. Holmes, J. Maro, B. Syat, K. Lane, R. Lazarus, and R. Platt. 2009. Developing
a distributed research network and cooperative to conduct population-based studies
and safety surveillance. In Report : Design Specifications for Network Prototype and
Research Cooperative.
OCR for page 139
ESSENTIAL PRIORITIES
Chan, A. W., A. Hrobjartsson, M. T. Haahr, P. C. Gøtzsche, and D. G. Altman. 2004. Em-
pirical evidence for selective reporting of outcomes in randomized trials: Comparison of
protocols to published articles. JAMA 291:2457-2465.
Chou, R., and M. Helfand. 2005. Challenges in systematic reviews that assess treatment
harms. Annals of Internal Medicine 142(12 [Part 2]):1090-1099.
CMS (Centers for Medicare & Medicaid). 2009. Prescription drug coverage general informa-
tion: Overview. http://www.cms.hhs.gov/PrescriptionDrugCovGenIn/ (accessed May 9,
2009).
Cupples, L., R. D’Agostino, and D. Kiely. 1988. The Framingham heart study. Section . An
Epidemiological Investigation of Cardiovascular Disease Survival Following Cardiovas-
cular Events: 0 Year Follow-up.
Cystic Fibrosis Foundation. 2009. Patient registry report. http://www.cff.org/treatments/Care-
CenterNetwork/PatientRegistryReport/ (accessed June 16, 2009).
de Simone, G., R. B. Devereux, M. Chinali, M. J. Roman, E. T. Lee, H. E. Resnick, and B. V.
Howard. 2009. Metabolic syndrome and left ventricular hypertrophy in the prediction of
cardiovascular events: The strong heart study. Nutrition, Metabolism and Cardiovascular
Diseases 19(2):98-104.
Deber, R. B., N. Kraetschmer, and J. Irvine. 1996. What role do patients wish to play in treat-
ment decision making. Archives of Internal Medicine 156:1414-1420.
Degner, L. F., and C. A. Russell. 1988. Preferences for treatment control among adults with
cancer. Research in Nursing & Health 11:367-374.
Dickersin, K. 2005. Publication bias: Recognizing the problem, understanding its origins and
scope, and preventing harm. In Publication Bias in Meta-analysis: Prevention, Assess-
ment, and Adjustments, Edited by H. Rothstein, A. Sutton and M. Borenstein. London,
UK: John Wiley and Sons, Ltd.
Dickersin, K., and Y.-I. Min. 1993. Publication bias: The problem that won’t go away. In
Doing More Good than Harm: The Evaluation of Health Care Interventions. Edited by
K.S. Warren and F. Mosteller. New York Academy of Sciences. 135-148.
Dougherty, D., and P. H. Conway. 2008. The “3T’s” road may transform US health care: The
“how” of high-quality care. JAMA 299(19):2319-2321.
Entwistle, V. A., M. J. Renfrew, S. Yearley, J. Forrester, and T. Lamont. 1998. Lay perspectives:
Advantages for health research. BMJ 316:463-366.
Faridi, Z., J. Grunbaum, B. S. Gray, A. Franks, and E. Simoes. 2007. Community-based
participatory research: Necessary next steps. Preventing Chronic Disease: Public Health
Research, Practice and Policy 4(3):1-5.
Farmer, D., S. A. Jackson, F. Camacho, and M. A. Hall. 2007. Attitudes of African American
and low socioeconomic status White women toward medical research. Journal of Health
Care for the Poor and Underserved 18:85-99.
FDA (Food and Drug Administration). 2008. FDA’s Sentinel Initiative. http://www.fda.gov/oc/
initiatives/advance/sentinel/ (accessed May 5, 2009).
———. 2009. Information for patient advocates http://www.fda.gov/oashi/home.html (ac-
cessed May 13, 2009).
Ford, E. W., N. Menachemi, L. T. Peterson, and T. R. Huerta. 2009. Resistance is futile: But
it is slowing the pace of EHR adoption nonetheless. Journal of the American Medical
Informatics Association 16(3):274-281.
Fretheim, A., H. J. Schanemann, and A. D. Oxman. 2006. Improving the use of research evi-
dence in guideline development: 3. Group composition and consultation process. Health
Research Policy and Systems 4.
Fung, M. M., R. Bettencourt, and E. Barrett-Connor. 2004. Heart disease risk factors predict
erectile dysfunction 25 years later: The rancho bernardo study. Journal of the American
College of Cardiology 43(8):1405-1411.
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
Furberg, C. D., J. T. Wright Jr, B. R. Davis, J. A. Cutler, M. Alderman, H. Black, W. Cushman,
R. Grimm, L. J. Haywood, F. Leenen, S. Oparil, J. Probstfield, P. Whelton, C. Nwachuku,
D. Gordon, M. Proschan, P. Einhom, C. E. Ford, L. B. Piller, I. K. Dunn, D. Goff, S.
Pressel, J. Bettencourt, B. DeLeon, L. M. Simpson, J. Blanton, T. Geraci, S. M. Walsh,
C. Nelson, M. Rahman, A. Juratovac, R. Pospisil, L. Carroll, S. Sullivan, J. Russo, G.
Barone, R. Christian, S. Feldman, T. Lucente, D. Calhoun, K. Jenkins, P. McDowell, J.
Johnson, C. Kingry, J. Alzate, K. L. Margolis, L. A. Holland-Klemme, B. Jaeger, J. Wil-
liamson, G. Louis, P. Ragusa, A. Williard, R. L. S. Ferguson, J. Tanner, J. Eckfeldt, R.
Crow, and J. Pelosi. 2002. Major outcomes in high-risk hypertensive patients randomized
to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic: The
antihypertensive and lipid-lowering treatment to prevent heart attack trial (ALLHAT).
JAMA 288(23):2981-2997.
Giuliano, A. R., N. Mokuau, C. Hughes, G. Tortelero-Luna, B. Risendal, R. C. S. Ho, T.
E. Prewitt, and W. J. McCaskill-Stevens. 2000. Participation of minorities in cancer
research: The influence of structural, cultural, and linguistic factors. Annals of Epide-
miology 10:S22-S34.
Gliklich, R. E., and N. A. Dreyer. 2007. Registries for evaluating patient outcomes: A user’s
guide. Rockville, MD: Agency for Healthcare Research and Quality.
Godwin, M., L. Ruhland, I. Casson, S. MacDonald, D. Delva, R. Birtwhistle, M. Lam, and R.
Seguin. 2003. Pragmatic controlled clinical trials in primary care: The struggle between
external and internal validity. BMC Medical Research Methodology 3:1-7.
Guadagnoli, E., and P. Ward. 1998. Patient participation in decision-making. Social Science
& Medicine 47(3):329-339.
Hanley, B., A. Truesdale, A. King, D. Elbourne, and I. Chalmers. 2001. Involving consumers
in designing, conducting and interpreting randomised controlled trials: Questionnaire
survey. BMJ 322:519-523.
The Henry J. Kaiser Foundation. 2009. Medicaid & CHIP. http://www.statehealthfacts.org/
comparecat.jsp?cat=4 (accessed May 9, 2009).
Hubbard, G., L. Kidd, and E. Donaghy. 2008. Involving people affected by cancer in research:
A review of literature. European Journal of Cancer Care 17(3):233-244.
INVOLVE. 2009. Promoting public involvement in NHS, public health and social care re-
search. http://www.invo.org.uk/ (accessed May 13, 2009).
IOM (Institute of Medicine). 1997. A review of the Department of Defense’s program for
breast cancer research. Washington, DC: National Academy Press.
———. 2001. Crossing the quality chasm: A new health system for the st century, National
Academy Press. http://www.nap.edu/catalog/10027.html (accessed April 20, 2009).
———. 2002. Responsible research: A systems approach to protecting research participants.
Washington, DC: The National Academies Press.
———. 2007. Learning what works best: The nation’s need for evidence on comparative
effectiveness in health care. http://www.iom.edu/ebm-effectiveness (accessed April 15,
2009).
———. 2008. Knowing what works in health care: A roadmap for the nation. Edited by J.
Eden, B. Wheatley, B. J. McNeil and H. Sox. Washington, DC: The National Academies
Press.
———. 2009a. Beyond the HIPAA privacy rule: Enhancing privacy, improving health through
research. Edited by Nass, S. J., L. A. Levit and L. O. Gostin. Washington, DC: The Na-
tional Academies Press.
———. 2009b. Conflict of interest in medical research, education, and practice. Edited by B.
Lo, and M. J. Field. Washington, DC: The National Academies Press.
OCR for page 139
ESSENTIAL PRIORITIES
———. 2009c. A framework for the workforce required for comparative effectiveness re-
search. In Learning What Works Infrastructure Required to Learn Which Care is Best
(in press). Washington, DC: The National Academies Press.
Johnson, B., M. Abraham, J. Conway, L. Simmons, S. Edgman-Levitan, P. Sodomka, J.
Schlucter, and D. Ford. 2008. Partnering with patients and families to design a patient-
and family-centered health care system: Recommendations and promising practices.
http://www.ihi.org/NR/rdonlyres/C810CCBB-2DEB-4678-994A-57D9B703F98D/0/
PartneringwithPatientsandFamiliesRecommendationsApr08.pdf (accessed June 12, 2009).
Jones, L., and K. Wells. 2007. Strategies for academic and clinician engagement in community-
participatory partnered research. JAMA 297(4):407-410.
Kent, D. M., and R. A. Hayward. 2007. Limitations of applying summary results of clinical
trials to individual patients. JAMA 298:1209-1212.
Lindenmeyer, A., H. Hearnshaw, J. Sturt, R. Ormerod, and G. Altchison. 2007. Assessment
of the benefits of user involvement in health research from the Warwick Diabetes Care
Research User Group: A qualitative case study. Health Expectations 10:268-277.
Mansell, D., R. M. Poses, L. Kazis, and C. A. Duefield. 2000. Clinical factors that influence
patients’ desire for participation in decisions about illness. Archives of Internal Medicine
160:2991-2996.
Mattews, S. C. 2009. Physician autonomy and informed decision making: Finding the balance
for patient safety and quality. JAMA 300(24):2913-2915.
Mazur, D. J., and D. H. Hickam. 1997. Patients’ preferences for risk disclosure and role in
decision making for invasive medical procedures. Journal of General Internal Medicine
12:114-117.
McClellan, M., and J. Benner. 2009. Comparative effectiveness research: Will it bend the
health care cost curve and improve quality? In Implementing Comparative Effectiveness
Research: Priorities, Methods, and Impact. Washington, DC: Brookings.
Meier, B. 2009. New effort reopens a medical minefield The New York Times, B1.
Merlini, L., P. Kishnani, B. Byrne, W. Maller-Felber, L. Case, and A. v. d. Ploeg. 2008. The
Pompe registry: Centralized data collection to track the natural course of Pompe disease.
Clinical Therapeutics 30(SUPPL. 1).
Minkler, M., A. G. Blackwell, M. Thompson, and H. Tamir. 2003. Community-based par-
ticipatory research: Implications for public health funding. American Journal of Public
Health 93(8):1210-1213.
National Center for Research Resources. 2009. Clinical and translational science awards.
http://www.ncrr.nih.gov/clinical_research_resources/clinical_and_translational_science_
awards/ (accessed June 16, 2009).
National Heart, Lung, and Blood Institute. 2009. Women’s health initiative: Observational
fact sheet. http://www.nhlbi.nih.gov/whi/os.htm (accessed May 23, 2009).
NIH (National Institutes of Health). 2009. NIH roadmap for medical research. http://nihroad-
map.nih.gov/ (accessed June 16, 2009).
Office of National Coordinator for Health Information Technology. 2009. Information re-
lated to the Economic Recovery Act of 2009. www.HealthIT.hhs.gov (accessed May 11,
2009).
Oliver, S. R., R. W. Rees, L. Clarke-Jones, R. Milne, A. R. Oakley, J. Gabbay, K. Stein, P.
Buchanan, and G. Gyte. 2008. A multidimensional conceptual framework for analysing
public involvement in health services research. Health Expectations 11:72-84.
Omenn, G. S. 1999. Caring for the community: The role of partnerships. Academic Medicine
74(7):782-789.
Peterson, E., and C. W. Yancy. 2009. Eliminating racial and ethnic disparities in cardiac care.
New England Journal of Medicine 360(12):1172-1174.
OCR for page 139
INITIAL NATIONAL PRIORITIES FOR CER
President’s Commission for the Study of Ethical Problems in Medicine and Biomedical and
Behavioral Research. 1982. Making health care decisions: The ethical and legal impli-
cations of informed consent in the patient-provider relationship. http://bioethics.gov/
reports/past_commissions/making_health_care_decisions.pdf (accessed June 25, 2009).
Rawlins, M. 2008. De testimonio: On the evidence for decisions about the use of therapeutic
interventions. The Lancet 372(9656):2152-2161.
Rising, K., P. Bacchetti, and L. Bero. 2008. Reporting bias in drug trials submitted to the
food and drug administration: Review of publication and presentation. Public Library
of Science Medicine 5(11):e217.
Sabin, J. E., K. Mazor, V. Meterko, S. L. Goff, and R. Platt. 2008. Comparing drug effective-
ness at health plans: The ethics of cluster randomized trials. Hastings Center Report
38(5):39-48.
Saunders, C., A. Girgis, P. Butow, S. Crossing, and A. Penman. 2008. From inclusion to
independence—Training consumers to review research. Health Research Policy and
Systems 6(3).
Schwartz, R. S., G. D. Curfman, S. Morrissey, and J. M. Drazen. 2008. Full disclosure and the
funding of biomedical research. New England Journal of Medicine 358(17):1850-1851.
Shojania, K. G., and J. M. Grimshaw. 2005. Evidence-based quality improvement: The state
of the science. Health Affairs 24(1):138-150.
Staniszewska, S., N. Jones, M. Newburn, and S. Marshall. 2007. User involvement in the
development of a research bid: Barriers, enablers and impacts. Health Expectations
10:173-183.
Tatsioni, A., D. A. Zarin, N. Aronson, D. J. Samson, C. R. Flamm, C. Schmid, and J. Lau.
2005. Challenges in systematic reviews of diagnostic technologies. Annals of Internal
Medicine 142(12 Part 2):1048-1055.
Teva Pharmaceuticals. 2008. Teva clozapine patient registry. https://clozapineregistry.com/
AboutRegistry/GeneralOverview.aspx (accessed June 16, 2009).
Tunis, S. 2009. Strategies to improve comparative effectiveness research methods and data
infrastructure. In Implementing Comparative Effectiveness Research: Priorities, Methods,
and Impact. Washington, DC: Brookings.
Turner, E. H., A. M. Matthews, E. Linardatos, R. A. Tell, and R. Rosenthal. 2008. Selective
publication of antidepressant trials and its influence on apparent efficacy. New England
Journal of Medicine 358(3):252-260.
United States Cochrane Center Consumers United for Evidence-based Healthcare. 2009.
Understanding evidence-based healthcare: A foundation for action. http://apps1.jhsph.
edu/cochrane/CUEwebcourse.htm (accessed May 23, 2009).
Van Spall, H. G. C., A. Toren, A. Kiss, and R. A. Fowler. 2007. Eligibility criteria of random-
ized controlled trials published in high-impact general medical journals: A systematic
sampling review. JAMA 297(11):1233-1240.
Weinfurt, K. P., D. M. Seils, J. P. Tzeng, L. Lin, K. A. Schulman, and R. M. Califf. 2008.
Consistency of financial interest disclosures in the biomedical literature: The case of
coronary stents. PLoS ONE 3(5).
Weinstein, J. N., K. Clay, and T. S. Morgan. 2007. Informed patient choice: Patient-centered
valuing of surgical risks and benefits. Health Affairs 26(3):726-730.
Wendler, D., R. Kington, J. Madans, G. Van Wye, H. Christ-Schmidt, L. A. Pratt, O. W.
Brawley, C. P. Gross, and E. Emanuel. 2006. Are racial and ethnic minorities less willing
to participate in health research? PLoS Medicine 3(2):201-210.
Wennberg, J. E., and E. S. Fisher. 2008. Tracking the care of patients with severe chronic ill-
ness: The Dartmouth Atlas of health care 2008. Dartmouth Institute for Health Policy
and Clinical Practice, Center for Health Policy Research. http://www.dartmouthatlas.
org/atlases/2008%5FChronic%5FCare%5FAtlas.pdf (accessed June 8, 2009).
OCR for page 139
ESSENTIAL PRIORITIES
Westfall, J. M., J. Mold, and L. P. Fagnan. 2007. Practice-based research—“Blue highways”
on the NIH roadmap. JAMA 297:403-406.
Williams, I. C., and G. Corbie-Smith. 2006. Investigator beliefs and reported success in recruit-
ing minority participants. Contemporary Clinical Trials 27:580-586.
Zerhouni, E. A. 2005. Translational and clinical science—Time for a new vision. New England
Journal of Medicine 353(15):1621-1623.
OCR for page 139