Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 57
1
The Need and Potential Returns for
Comparative Effectiveness Research
INTRODUCTION
Understanding the comparative effectiveness of candidate interven-
tions for similar conditions is essential for improving the development and
delivery of effective health care. Many publications have highlighted the
shortfalls of health care delivered in the United States, including the fre-
quency of medical errors (IOM, 2000); wide variation in practice patterns,
driven more by services available than medical needs (Fisher and Wennberg,
2003); the slow translation of research discoveries into medical practice
(Balas and Boren, 2000; Woolf, 2008); the limited quality of the evidence
developed to guide healthcare decision makers (Atkins et al., 2004; Califf,
2004; IOM, 2008a; Tunis and Pearson, 2006); and the adverse conse-
quences of care administered with adequate evidence (IOM, 2001). While
each highlights a different problem or concern with the current healthcare
system, collectively these findings reveal systemic inadequacies in current
approaches to developing evidence to help guide the health decisions of
policy makers, physicians, and patients.
Underscoring the pressing need for better insights into the relative effec-
tiveness of therapeutics and treatments are the rising and unsustainable costs
of health care and the relatively low returns for those high-cost investments.
In 2009, spending on health care totaled $2.5 trillion, or over 17 percent of
the nation’s gross domestic product. Healthcare costs are becoming increas-
ingly burdensome, with annual out-of-pocket costs to consumers steadily
increasing (KFF, 2009). Some experts suggest that medical costs due to illness
and injury contribute to a significant proportion, perhaps half, of bankrupt-
OCR for page 58
LEARNING WHAT WORKS
cies filed by American families (Himmelstein et al., 2005). The Congressional
Budget Office estimates that if left unchecked, health expenditures will rise to
25 percent of the gross national product by 2025 (CBO, 2007).
Developing and using information on which treatments work best for
whom is imperative to achieving better value from national healthcare
expenditures. Of the more than $2.5 trillion spent in 2009 on health in
the United States, available estimates indicate that less than one-tenth of 1
percent has been devoted to such research (AcademyHealth, 2005; Moses et
al., 2005). Recently, policy makers have demonstrated substantial interest in
comparative effectiveness research (CER) (Jacobson, 2007), with attention
and discussion focused on increased funding and on the structure, place-
ment, and governance of an entity or agency charged with developing CER
information (Kupersmith et al., 2005; Wilensky, 2006). With the passage
of the American Recovery and Reinvestment Act of 2009, $1.1 billion were
made available to the National Institutes of Health (NIH), the Agency for
Healthcare Research and Quality (AHRQ), and the Secretary of Health and
Human Services for the conduct of CER and to encourage data resource
development and use for such analyses.1 These funds provided an important
down payment on efforts to move to a system focused on delivering high-
value care and driven by the best evidence, and formal recommendations
have been made by the Institute of Medicine (IOM) (2009) and the Federal
Coordinating Council for CER (FCC, 2009). With the 2010 passage of
the ACA, and establishment of the Patient-Centered Outcomes Research
Institute (PCORI), the capacity for sustained investment has developed.
Appendices C, D, and E offer additional background.
The infrastructure needed to expand capacity for CER extends beyond
developing data resources (e.g., registries, databases, data networks). Inno-
vative research strategies are needed to improve the efficiency and relevance
of clinical research as well as to ensure the appropriate translation and use
of CER information by decision makers. Consideration is also needed of
how best to align the substantial promise offered by health information
technologies—to gather and disseminate needed data and information—with
the needs of CER. These technologies offer opportunities to reduce costs and
improve the quality of health care (e.g., e-prescribing, remote monitoring,
public health records, electronic health records [EHRs]) and will increase
access to new types of data and modes of communication (Litan, 2008).
Adopting such innovations requires infrastructure development. Careful
investments in the requisite workforce, systems, and technologies can also
enhance the nation’s capacity to learn from health care delivered.
Consideration of such long-term strategies as well as the identifica-
tion of areas where appropriate investment and coordination will enable
1 American Recovery and Reinvestment Act. 2009. HR1, 111th Cong, 1st Sess.
OCR for page 59
9
THE NEED AND POTENTIAL RETURNS FOR CER
immediate progress were the focus of the July 30–31, 2008, workshop,
Learning What Works: Infrastructure Required for Comparative Effective-
ness Research. The meeting’s discussions were motivated by many of the
issues discussed above and the resulting need to explore key elements and
opportunities for infrastructure development (see Box 1-1 on p. 64).
Expanded capacity for comparative effectiveness research can provide
information and insights helpful to important care decisions of patients,
providers, and policy makers, but progress will require informed and care-
ful investment in key infrastructure elements. Summarized below are pos-
sible implications of CER for healthcare stakeholders, an assessment of
activities under way, and options to enhance national CER currently under
consideration. The workshop’s two keynote presentations offered addi-
tional perspectives on infrastructure needs by describing a long-term vision
and potential returns for a healthcare system informed by CER.
Mark B. McClellan reflects on the core elements of a robust and
sustainable capacity for CER and how these immediate needs might also
fit into a long-term strategy to support the functions of a learning health
system. Noting that form should follow function, he outlines four key
evidence gaps that should inform infrastructure development: (1) baselines
for evaluations, such as disease models and natural histories; (2) safety; (3)
comparative effectiveness of interventions; and (4) comparative effective-
ness of treatment strategies and practice patterns. Efforts should focus on
all of these areas that fall short in order to develop a healthcare delivery
system that provides better outcomes for each kind of patient at much
lower cost. Gail R. Wilensky notes that the potential returns from increased
investment in CER are enormous, and she offers some suggestions on the
elements required for progress, including establishing a center charged with
creating better information. It will also be important to develop and use
the approaches, data resources, and analyses most useful to producing the
information needed and to recognize that all stakeholders need to be a part
of the decision-making process.
Background material for the workshop was assembled and prepared
by staff of the IOM’s Roundtable on Value & Science-Driven Health Care,
founded in 2006 to provide a trusted venue for major healthcare stakehold-
ers to consider and advance their mutual interests in the enhanced devel-
opment and use of evidence in health care. The Roundtable has defined
science-driven health care broadly to mean “to the greatest extent pos-
sible, the decisions that shape the health and health care of Americans—by
patients, providers, payers, and policy makers alike—will be grounded on
a reliable evidence base, will account appropriately for individual varia-
tion in patient needs, and will support the generation of new insights on
clinical effectiveness” (IOM Roundtable on Value & Science-Driven Health
Care, 2009). An expanded capacity to develop evidence on the compara-
OCR for page 60
0 LEARNING WHAT WORKS
tive benefits and risks of healthcare treatments and strategies is an essential
step toward a learning health system, and it has been the principal focus of
the Roundtable’s working group on sustainable capacity. At the working
group’s request, in 2007 IOM staff authored an issue overview white paper,
Learning What Works: The Nation’s Need for Evidence on Comparative
Effectiveness in Health Care (IOM, 2007). This background brief also
provided context for the July 30–31 workshop, Learning What Works:
Infrastructure Required for Comparative Effectiveness Research, and was
included in the meeting’s briefing materials. It is summarized below and
included in full in Appendix A.
THE NATION’S NEED FOR EVIDENCE ON
COMPARATIVE EFFECTIVENESS IN HEALTH CARE:
LEARNING WHAT WORKS BEST
J. Michael McGinnis, LeighAnne Olsen, Dara Aisner, Pamela Bradley,
Daniel O’Neill, and Katharine Bothner
(IOM Roundtable on Value & Science-Driven Health Care staff)
A core objective for the nation is achieving the best health outcome for
every patient. This objective cannot be accomplished until better evidence
is available upon which to base healthcare decisions and until existing
knowledge is applied more effectively. Each need is vitally important. It
is known, for example, that failure to deliver proven interventions is a
substantial challenge to the quality of health care for Americans, and it is
a key concern of the IOM Roundtable on Value & Science-Driven Health
Care (IOM, 2007). Yet, with the current pace of change, the most rapidly
growing problem is the healthcare system’s inability to produce the needed
evidence in a timely fashion. Medical-care decision making is now strained,
at both the level of the individual patient and the level of the population
as a whole, by the growing number of diagnostic and therapeutic options
for which evidence is insufficient to make a clear choice. The consequences
can be seen in the broad geographic variation in the intensity of services
delivered for the same outcome, in the occurrence of medical errors, in
patient and provider confusion about which interventions deliver the most
value, and in the costs of care.
A testament to innovation is the fact that new pharmaceuticals, medical
devices, biologics, and procedures are introduced constantly, and the pace is
quickening. From 1991 to 2003 the number of medical device patents per
year doubled, and biotechnology patents tripled. Between 1993 and 2004
there was an 80 percent increase in the number of prescriptions received
by Americans. A recent review suggests that half or more of the growth in
medical spending in recent years is attributable to changes in technology.
OCR for page 61
THE NEED AND POTENTIAL RETURNS FOR CER
In addition to the growth in application of drugs, devices, biologics,
and procedures, the world of health care is about to experience dramatic
new insights into the genetic variation in individual responses to different
diagnostic and treatment interventions (AdvaMed, 2004; Biotechnology
Industry Organization, 2006; Foster et al., 2002; Gelijins and Rosenberg,
1994). The age of personalized medicine will soon be a reality, if the capac-
ity can be developed to contend with these insights. Today the average
clinical encounter already requires a health provider to manage more vari-
ables than would be considered reasonable given what is known about the
capabilities of the human mind. Over the next decade, that same encounter
will require contending with perhaps an order of magnitude more complex
(IOM, 2007).
These developments hold fundamental implications for health pros-
pects, and, to capture and use them effectively and efficiently, a proportion-
ate commitment is required to understand their advantages and appropriate
applications. It is both a capacity investment and a resource allocation
problem. Of the nation’s more than $2.5 trillion in 2009 health expen-
ditures, only a tiny fraction was devoted to CER. If only 1 percent of the
nation’s healthcare spending were devoted to understanding the effective-
ness of the care purchased, the total for effectiveness research would come
to approximately $20 billion annually—about 10 times the amount in
2009. In contrast, even accounting for the support from all private and pub-
lic sources, the aggregate national commitment to assessing the effectiveness
of clinical interventions is far below the standard that any company would
expect to invest in work to evaluate and improve its products.
Regardless of individual perspectives on reform of the many challenging
issues in health policy today, there is little question about the critical need
for patients and providers to have better information with which to make
their decisions about the comparative advantages of healthcare options.
What follows is a summary of the issues and options and is intended to
inform discussions of how to proceed on this matter of central importance
to health and health care. It does not provide recommendations.
Implications for Stakeholders
For patients, the stakes are very clear. Every patient should be able to
feel confident that there is solid evidence that the care received is the care
most appropriate to the circumstances. Yet, increasingly this is not the case.
In a 2005 survey, 60 percent of Americans said they didn’t believe that the
United States had the best healthcare system in the world, 41 percent said
they knew of a time when they or a family member had received the wrong
care, and 56 percent said there should be more investment in clinical and
health services research. Health providers feel similar tensions. No health
OCR for page 62
2 LEARNING WHAT WORKS
professional should be put in the position of uncertainty about the evidence
in support of the care provided at his/her behest. Yet, with the pace of
advances in medical procedures, pharmaceuticals, devices, and biotech-
nology, a sometimes confusing array of choices is presented for patients,
their healthcare providers, and the healthcare organizations in which care
is delivered. The integrity and reputation of healthcare delivery organiza-
tions is dependent on their ability to ensure the quality and appropriate-
ness of the care delivered within their walls. Any decision support system
is only as good as the information built into the model and should include
the comparative advantages or disadvantages of different diagnostic and
therapeutic options.
Healthcare manufacturers, focused as they are on returns on invest-
ment, inherently understand the importance of improving the value propo-
sition in patient care. But their stakes go deeper. Manufacturers directly
bear the economic burden of delays and inefficiencies when information
is not available about the advantage of their products, not to mention the
challenges of public and shareholder backlash when problems are identi-
fied too late. Without a sizable improvement in our evaluation capacity, the
slower pace of understanding how and when interventions work best will
retard the application of innovative treatments.
From a purchasing perspective, the need for better information is of
central importance to those who pay for health care: patients, employers,
insurers, and the government. Over half of the nation’s health expenditures
are borne by the private sector, including a sizable share by employers. For
the fourth consecutive year, chief executive officers of U.S. companies have
cited healthcare costs as their number one economic concern. Employers
now pay 78 percent more for health care than 5 years ago, and it has been
suggested by some that this increased financial burden makes it more diffi-
cult for American companies and workers to compete in the global market-
place. Often acting on behalf of employers, insurers represent the front line
of the economic choices that have to be made about payment for healthcare
services. This means drawing conclusions about the comparative advan-
tages or disadvantages of proposed diagnostic or treatment interventions in
the face of a paucity of such information, especially information applicable
to real-world circumstances. As a payer, the government accounts for about
45 percent of health expenditures in the United States, including care that
it delivers directly in its own facilities. Whether as a payer or a provider,
the government has a central interest in ensuring that its clients receive the
care that is most appropriate and of the greatest value.
OCR for page 63
THE NEED AND POTENTIAL RETURNS FOR CER
Current Activities in Clinical Effectiveness Research
Currently, activities to assess the effectiveness of healthcare interven-
tions are broad but underresourced and fall far short of the need (IOM,
2007). CER can be described as either primary or secondary. Primary refers
to the direct generation of evidence through the use of a specific experimen-
tal methodology. Secondary refers to the synthesis of evidence from multiple
primary studies in order to draw conclusions for practice. Within the over-
all umbrella of CER, the most practical need is for studies of comparative
effectiveness, the comparison of one diagnostic or treatment option to one
or more others (Wilensky, 2006).
The largest investment in CER has been made by industry, with industry-
sponsored clinical trials representing a significant portion of health manu-
facturer investments in research and development (R&D). For example,
about 40 percent of pharmaceutical R&D investments goes to the phase
3 and phase 4 trials, which have particular relevance to clinical effective-
ness (PhRMA, 2006). Many of these studies are conducted with academic
investigators, and others are managed by contract research organizations.
Relatively few of the studies are comparative, or head-to-head, studies.
Outside of industry, several government agencies support CER, includ-
ing AHRQ, which has a specific mandate and a small appropriation for
CER. In 2005, the total appropriations to all federal agencies—the NIH, the
Veterans Health Administration, the Department of Defense, the Centers for
Medicare & Medicaid Services (CMS), the Food and Drug Administration
(FDA), AHRQ, and the Centers for Disease Control and Prevention—for all
health services research amounted to about $1.5 billion, and only a mod-
est portion of this was devoted to clinical effectiveness research, far below
the industry level. Additional work, also modest, is undertaken by certain
of the larger healthcare delivery organizations. Evidence synthesis activity
is supported by the insurance industry, professional societies, healthcare
organizations, and government. AHRQ has established a network of 13
AHRQ-sponsored evidence-based practice centers that review literature
and produce evidence reports, including comparative effectiveness reviews.
Organizations interested in evidence reviews will often draw upon syntheses
performed by several well-established technology assessment entities (IOM,
2008).
Activities and Needs Related to Comparative Effectiveness Research
Although there is a great deal of interest and activity surrounding
the topic of clinical effectiveness, the aggregate research capacity is very
thin, and the products fall substantially short of the need. Because of the
scant resources available for the support of primary CER—head-to-head
OCR for page 64
LEARNING WHAT WORKS
BOX 1-1
Issues Motivating the Discussion
1. ubstantial demand for greater insights into the comparative clinical
S
effectiveness of clinical interventions and care processes to improve
the effectiveness and value of health care.
2. xpanded interest and activity in the work needed—e.g., compara-
E
tive effectiveness research, systematic reviews, innovative research
strategies, clinical registries, coverage with evidence development.
3. urrently fragmented and largely uncoordinated selection of studies,
C
study design and conduct, evidence synthesis; methods validation and
improvement, and development and dissemination of guidelines.
4. xpanding gap in workforce with skills to develop data sources and
E
systems, design and conduct innovative studies, translate results, and
guide application.
5. pportunities presented by the attention of recent initiatives and the in-
O
creasing possibility of developing an entity and resources for expanded
work on the comparative effectiveness of clinical interventions.
6. rowing appreciation of the importance of assessing the infrastructure
G
needed for this work—e.g., workforce needs, data linkage and improve-
ment, new methodologies, research networks, technical assistance.
7. esirability of a trusted, common venue to identify and character-
D
ize the need categories, begin to estimate the shortfalls, consider
approaches to addressing the shortfalls, and identify priority next
steps.
studies—much of the work is, of necessity, secondary evidence synthesis.
Yet the most pressing needs that clinicians and their patients have are for
reliable studies upon which to base their decisions. The elements of the
needs have been characterized in various ways, and can be grouped into
the key areas indicated in Box 1-1. The key challenges that must be faced
in each of these areas are summarized in Table 1-1 (Buto and Juhn, 2006;
Clancy, 2006; Health Industry Forum, 2006; Hopayian, 2001; Kupersmith
et al., 2005; Rowe et al., 2006).
Models for a Stronger Approach to Comparative Effectiveness Research
To narrow the rapidly growing gap between the available evidence on
clinical effectiveness and the evidence necessary for sound clinical decision
OCR for page 65
THE NEED AND POTENTIAL RETURNS FOR CER
TABLE 1-1 Prominent Comparative Effectiveness Research Activities and
Needs—Key Challenges
Issue Key Challenges
Head-to-head studies Scant resources; rapidly increasing need;
comparison choice
Systematic reviews Few primary studies; inconsistent methods;
uncoordinated
Comparative value insights Little agreement on metrics or role of costs; cost
fluctuation
Priority setting Fragmentation; inefficiency; no mechanism for
coordination
Study designs and tools Clinical trial time/cost/limits; large dataset
mining methods
Research life-cycle links Efficacy–effectiveness disjuncture; postapproval
surveillance
Evidence standards Standards not adapted to needs; inconsistency in
application
Practice guidance Disparate approaches; conflicting
recommendations
Coverage guidance Narrow evidence base; limited means for
provisional coverage
Application tools Public misperceptions; incentive structures;
decision support
SOURCE: IOM, 2007.
making, various organizations and recent public articles have called for
the creation of a new entity and a quantum increase in spending—several
billion dollars—on CER. The various approaches to building the required
capacity can be grouped into four categories according to the funding pat-
terns for their support (Box 1-2). Each of the approaches is based on an
existing or recent model. Although presented as discrete models for discus-
sion purposes, they are not mutually exclusive.
The most straightforward public-funded approach is an expanded and
appropriated mandate to an existing or newly created federal agency, and
the agency whose mandate most closely parallels these priorities is AHRQ.
Through its Effective Health Care program, AHRQ has an existing frame-
work into which many elements of the identified needs can easily fit. Other
executive branch models include locating the primary capacity in the NIH,
putting it elsewhere in the Department of Health and Human Services, or
creating it as a free-standing operational federal agency.
Other possibilities include approaches that are privately funded,
although this raises issues of independence and objectivity, as well as
approaches with a blend of public and private funding, which could have
various governing and execution structures. In the latter category are those
OCR for page 66
LEARNING WHAT WORKS
BOX 1-2
Models for Enhancing Capacity
Incremental funding augmentations
• Incremental model
Publicly funded entity
• Executive branch agency model
• Independent government commission model
• Legislative branch office model
Privately funded entity
• Operating foundation model
• Investment tax credit cooperative model
Public–private funded entity
• user fee public model
• Federally funded research and development center public model
• Independent cooperative model
• Independent quasi-governmental authority model
SOuRCE: IOM, 2007.
approaches based on the quasi-governmental federally funded research
and development centers (FFRDCs), which are funded primarily by the
federal government but which are allowed to have up to 30 percent of
their funding from private sources. The FFRDCs are private entities man-
aged by nongovernmental organizations and are based on the examples of
free-standing independent quasi-governmental entities such as the Federal
Reserve Board, which serves as the nation’s central banking system, and
the IOM and the Transportation Research Board (TRB) at the National
Academies. TRB, from its National Academies locus, houses publicly and
privately funded work in transportation that is conceptually similar in
structure to what is envisioned for CER (IOM, 2008a; Kupersmith et al.,
2005; Wilensky, 2006).
Decision and Implementation Considerations
Weighing the relative strengths and weaknesses of the various models
can begin with certain touchstone principles that have been suggested to
OCR for page 67
THE NEED AND POTENTIAL RETURNS FOR CER
help guide their consideration. These include the characteristics of the
approaches with respect to the following:
Scientific credibility: ability to gain the trust and confidence of
•
the public, the scientific community, and the other stakeholders
involved.
Political independence: well-insulated from the political processes
•
that interests from all perspectives will seek to leverage.
Stakeholder neutrality: ability to engage with all stakeholders—
•
patients, providers, employers, manufacturers, and insurers—in an
independent, even-handed fashion.
Participatory governance: affording the opportunity for relevant
•
stakeholders to engage as appropriate in setting priorities and
agendas, while safeguarding the scientific integrity.
Investigator integrity: management and conduct of the research
•
processes, and the determination and validation of research results
completely insulated from outside influence.
Agenda flexibility: organizational decision making, resource alloca-
•
tion, and program conduct with the flexibility to respond quickly
to emerging issues and changing circumstances.
Infrastructure efficiency: use where possible existing capacity for
•
the establishment of scientific standards and for the management
and conduct of studies.
Transparency of processes and results: specification and availability
•
of the data on which determinations are based, and clarity as to the
processes and tools used in their evaluation.
Other implementation considerations include those related to funding
and program management. As noted earlier, funding estimates are in the
range of several billion dollars. This is a sizable amount, although it is
not particularly large in the context of the total U.S. health expenditures
or in the context of the efficiencies that could be gained. Suggestions for
funding mechanisms range from direct annual federal appropriation or a
small set-aside from the Medicare Trust Fund to the structuring of propor-
tionately matching contributions, including set-asides from Medicare fund
expenditures, from private health insurance premiums, or from manufac-
turers’ R&D expenditures (Health Industry Forum, 2006; Hopayian, 2001;
Kupersmith et al., 2005; Wilensky, 2006). There can be many variations
on these themes, but, ultimately, the source of the funds invested is not so
important as the value of the return for the outcomes and efficiency of the
nation’s health care.
OCR for page 76
LEARNING WHAT WORKS
The geographic variations in the kinds of care delivered to similar
Medicare patient populations are at least partly the result of the lack of
evidence or consensus on treatment strategies. Experts have suggested that
it should be possible to reduce costs in Medicare by 20 percent or more
without consequences for patient outcomes—if these variations could be
addressed. These variations in costs from area to area are the result of the set
of sometimes subtle differences in practice patterns, especially for chronic-
disease management. Among a population of patients, for example, the
rate at which they are seen for follow-up varies significantly. Thus, relevant
effectiveness questions include: What proportion of patients make it to more
frequent follow-up? How often are they referred to specialists, and to which
type of specialists? Which diagnostic tests are done and when? What minor
procedures are performed on these patients and when? In this context, the
evidence generated from head-to-head comparisons of treatments in experi-
mental settings is unlikely to address the root causes of this variation.
Resolving questions related to differences in practice will likely require
other methods besides randomized clinical trials (RCTs). To compare
such practices and determine which relevant policies factor into variation,
research needs to account for changes in delivery systems, changes in benefit
designs, and changes in payments to providers that could influence how
certain practices might lead to better outcomes at a lower cost. Such assess-
ments should be part of the science of healthcare delivery, and knowledge
gained through such studies could influence practice.
These questions can be studied in real-world medical practice, where
similar patient populations are exposed to different health policies and
therefore may face different treatment options and strategies. These kinds
of studies could help close the gap between what is known to work and
what is actually delivered in medical practice. Such studies, for example,
would help answer questions regarding the lack of long-term adherence to
certain medications among the chronically ill. More broadly, comparisons
of treatment strategies could enhance our understanding of the underlying
issues related to the coordination and integration of care—the lack of which
constitutes a major problem in our healthcare system today.
The infrastructure needed to address these challenges should involve
broad collaboration among many stakeholders, including AHRQ, other
researchers, health plans, employers, consumers and patients, and provider
groups. Consensus is needed to identify the best methodological approaches
for developing the kind of evidence that can show which reforms in pay-
ments, benefits, and support systems for healthcare professionals and for
consumers can lead to the best results. In this case, methods development
should focus on improving observational methods since strategies and poli-
cies can only be studied in real-world practice and are not amenable to the
idealized academic clinical trial setting. Such studies, however, could be
OCR for page 77
THE NEED AND POTENTIAL RETURNS FOR CER
very useful in uncovering key opportunities for improving outcomes while
lowering costs.
Closing Observations
This paper provided examples of what an infrastructure can do to
advance knowledge about which care is best as well as some insights about
the different elements of infrastructure that can support a learning health
system. Unless new infrastructure is informed by current gaps in evidence—
and the ultimate goal of delivering care that produces better outcomes for
different patient types at much lower costs—it will not be possible to close
all the gaps that exist today. Fortunately, there is a tremendous opportu-
nity to meaningfully expand the evidence base, as a result of the advances
and collaborations discussed in the chapters that follow. The efforts of all
stakeholders are necessary to transform health care into a system that learns
much more effectively from actual practice.
THE POTENTIAL RETURNS FROM
EVIDENCE-DRIVEN HEALTH CARE
Gail R. Wilensky, Ph.D., Senior Fellow, Project HOPE
Interest in the potential of comparative clinical effectiveness informa-
tion as a strategy to help Americans learn to “spend smarter” has been
growing among those at both ends of the political spectrum, and it can
best be understood as part of the concern about healthcare quality and
value, and the drive toward the increased use of evidenced-based medi-
cine. Other countries have focused on the use of comparative effectiveness
information primarily as a strategy for new drug approval in their national
health systems. The potential economic gains are even greater for medi-
cal procedures where even less comparative effectiveness information has
traditionally been available, since procedures account for much more of
the healthcare dollar. Substantial attention has been given to the important
decisions that need to be made about the appropriate structure, placement,
financing, and function of an agency devoted to comparative effectiveness.
It will be equally important to focus on how best to align financial incen-
tives to encourage the use of better information in clinical decision making.
The potential for better information to improve health outcomes and help
moderate spending increases is enormous, and an understanding of this
information is dependent upon how to capture some of the potential sav-
ings that CER could bring.
As a prelude to this discussion, I want to note that statistically significant
information will not necessarily have clinical or policy importance, in terms
OCR for page 78
LEARNING WHAT WORKS
of guiding clinical decision making. Early in my career as a health econo-
mist, I codirected the National Medical Care Expenditure Survey (NMCES)
at what was then called the National Center for Health Services Research.
One of the lessons that I learned is that when very large samples are being
assessed—such as 40,000 individuals, the sample size for the NMCES—
almost any difference is statistically significant, but many of those differences
were not relevant in terms of driving any conceivable policy decision. In the
case of very small samples, on the other hand, what appear to be large dif-
ferences may, in fact, not be statistically significant, which means that they
should be used only with great caution in making policy decisions.
The problems that can arise from information bias are a major reason it
may be important to consider new data collection, including the possibility
of new prospective trials and other costly data-collection strategies, even
when it looks like observed differences are substantial. This need stems
from the possibility of self-selection or biased selection being introduced
in analyses of observed data. An obvious example concerns the presumed
advantages of hormone-replacement therapy (HRT). Prior to the Women’s
Health Initiative studies, relationships had been observed between the use of
HRT and a variety of positive outcomes, such as improved cardiovascular
health or lower rates of dementia. Unfortunately, data from the Women’s
Health Initiative showed these supposed advantages to be a function of
selection bias related to the characteristics of the women who were using
HRT. Therefore, the sizeable differences that had been observed were, in
fact, not meaningful in terms of causal interpretation.
Such cases serve as reminders that it is important in designing a study
not only to lay out the hypotheses and the data that will serve to support
or not support the hypotheses but also to take great care in searching for
correlations among the independent variables and other potential drivers
of statistical bias in the data to be sampled. In addition, the differences that
are likely to exist between various groups will help to determine the neces-
sary sample size needed for the study. Finally, some determination will be
needed as to whether the likely differences are ones that would be relevant
at a clinical or policy level.
With that introduction, let me turn now to the kinds of data that will
be relevant for comparative effectiveness analyses, remembering that the
focus for these analyses is generally a medical condition and the various
alternative strategies that can be used for treating that medical condition.
It will be important to ensure that data are collected for various subgroups
in the population that may be differentially affected by a particular medical
condition. At the moment, these distinctions may be defined in terms of age
and sex or other demographic characteristics, but ultimately it may be pos-
sible to differentiate probable outcomes based on an individual’s genotype,
phenotype, or metabolic type.
OCR for page 79
9
THE NEED AND POTENTIAL RETURNS FOR CER
While there has been debate about the data that are most appropriate
to use for comparative effectiveness, it can be argued that data need to be
collected from as many sources as is possible. This includes not only the so-
called gold standard of double-blinded RCTs, but the use of real-world pro-
spective trials that Sean R. Tunis and others have been developing allow for
inclusion of individuals with comorbidities, epidemiological studies, medical
record analyses, registry data, administrative data, and so forth. There have
been occasions where researchers have spoken as though only data reflecting
the results of double-blinded RCTs should be regarded as appropriate for
decision making, but comparative effectiveness analyses need to include data
from many sources, although it will be important to make clear the robust-
ness of the data collection strategies and methodologies used in the analyses.
Presumably the conclusion made from the data will reflect the robustness of
the data and the statistical analyses used in the assessment.
All data have limitations and are subject to error, including the results
from RCTs. Specifying these limitations and biases and correcting for them
wherever possible is appropriate and should be made available as part of
the data release. It will also be important to find ways to reduce the costs
and time required for the collection of new prospective data, given the
amount of new data collection that is likely to be needed. Efforts by Bryan
Luce in developing his Pragmatic Approaches to Comparative Effectiveness
initiative, along with the work of Don Berry that makes use of Bayesian
statistical approaches to establish shorter end points in certain types of
clinical trials, represent other important efforts in this vein.
Even with these strategies to reduce the costs of new prospective trials,
it is the anticipated need for a substantial amount of new data that makes
the cost involved with comparative effectiveness significant. My guess is
that, when fully operational, such efforts could cost several billions of
dollars a year—perhaps in the neighborhood of $4 billion to $6 billion a
year—although an investment of several hundred million dollars would
probably be enough to make a serious start.
The first step in considering an analysis of the comparative effective-
ness of various treatments for a particular medical condition is to assess
the data that already exist and the analyses that have already been done.
It now appears that this step may also require significant investments in
time and effort. The IOM report Knowing What Works in Health Care: A
Roadmap for the Nation served as a wake-up call that making better and
more effective use of the information that exists is harder and more chal-
lenging than many of us had previously thought (IOM, 2008). Obtaining
systematic reviews of existing data will also be more controversial than had
previously been recognized.
Setting priorities for comparative effectiveness analyses should be
informed by a two-step process: first, focus on those medical conditions—
OCR for page 80
0 LEARNING WHAT WORKS
Medicare diagnosis-related groups might be a useful proxy—that are high-
cost/high-volume areas in health care; and second, focus on those medical
conditions that are subject to substantial variation in terms of how they
are treated. As an economist who is looking at comparative effectiveness
as a way to learn how to “spend smarter,” I suggest that the best place to
focus early efforts would be those conditions on which a lot of money is
spent and for which there is a great deal of geographic variation, since that
suggests that differences in opinion exist about how best to treat the condi-
tion or, in any case, that differences exist in how the condition is actually
treated. It would also be appropriate to look at issues of clinical relevance,
disease burden, and the various subgroups that are particularly affected.
Such considerations would certainly help determine, at a policy level, the
relative importance of given interventions.
An important early step for more effective CER will be the creation of
either a new center or a series of centers—my preference would be for a
single center—that is part of the government or is a public–private enter-
prise and that is responsible for funding comparative clinical effectiveness
studies. Unlike some of my colleagues, I believe it would be unwise—both
at a technical level and, even more importantly, at a political level—to
include cost-effectiveness analysis as part of the activities of a center for
comparative clinical effectiveness. Cost-effectiveness information should be
a component in reimbursement decisions made by payers and even in clini-
cal decision making by clinicians and individuals, but these analyses should
be kept separate and carried out in separate places.
One reason is that the amount of effort required to increase knowledge
about comparative clinical effectiveness is of a much different magnitude
in terms of the kinds of studies, the cost of the studies, and the length of
time these studies will require. Also, at a technical level, some of the issues
involved in cost effectiveness, particularly when it comes to such issues as
discount rates, get into areas for which there are no definitive answers.
Moreover, the number that one chooses to use has a significant effect on
the outcome calculated. Other kinds of technical challenges that arise in
cost-effectiveness analysis involve which cost to use and whose perspec-
tive to use: Should it be society’s perspective? Medicare’s? The employer’s?
Other issues are when in the lifetime of a technology the cost is measured
and whose costs are being considered.
As important as the technical reasons are for keeping comparative clini-
cal effectiveness analyses separate from comparative cost-effectiveness analy-
ses, the political reasons are even more important. Cost-effectiveness analyses
have long been held in suspicion by industry and many patient advocacy
groups as a strategy to prevent them from providing or receiving the latest
innovations and technologies in medical care. While this is an issue that ulti-
mately will have to be dealt with, without better information about the likely
OCR for page 81
THE NEED AND POTENTIAL RETURNS FOR CER
effects of different medical interventions in treating various medical condi-
tions, particularly high-cost conditions, it will not be possible to effectively
make use of the information on cost effectiveness. It is therefore urgent that
the nation make the investment in comparative clinical effectiveness informa-
tion and keep it as protected as possible, for all users—clinicians, patients,
and payers, both public and private—to have available. It will be crucial to
have a common data source that we can turn to that captures what is known
about the likely clinical outcomes of various kinds of treatments for various
subgroups of the population before we get into the next round of much more
difficult decision making about how we make use of that information. Hav-
ing said that, we need to have better information about cost effectiveness as
well. Fortunately, that can be funded more quickly and at substantially less
cost. A portion of the funding stream that is used to fund studies in compara-
tive clinical effectiveness can be provided to CMS to fund studies in the cost
effectiveness of alternative medical treatments.
In determining the best way to fund CER, it is important to take into
account the issue of the preferred versus the practical. The preferred strat-
egy would be to use a direct appropriation, as is the case with the NIH,
since the information generated is clearly a public good as the economist
uses the term. Unfortunately, the practical reality is that relying on a direct
appropriation is likely to produce an unreliable funding stream. An alter-
native would be to use a combination of direct appropriations with fees,
which would resemble an all-payer system, for people who are covered
by private plans as well as a contribution from the public players such
as Medicare. If it appeared that a direct appropriation to CMS for cost-
effectiveness analysis was unreliable, a small portion of the funding stream
could be diverted to CMS in order to fund the cost-effectiveness studies
that are important for Medicare. It will also be important to ensure that
the information on cost effectiveness that is generated is valid in terms of
objectivity and credibility—just as information needs to be for compara-
tive clinical effectiveness—or it will not be trusted. However, unlike the
comparative clinical effectiveness information, payers could generate better
information about cost effectiveness as long as they do so in a way that
keeps the generation of the information transparent. Alternatively, the
cost-effectiveness analysis could done by AHRQ or other entities in the
federal government, and it could continue to be done in the private sector
by private-sector payers as well.
The question then becomes how do you begin to use this information?
Several important principles apply. First, the concept of expecting and
allowing for different players to use this information differently is very
important. If there is a sense that a single entity can and is making deci-
sions about how information on comparative clinical effectiveness and cost
effectiveness will be used, there will be a great deal of resistance by patient
OCR for page 82
2 LEARNING WHAT WORKS
advocates as well as by industry. That does not mean that such information
should not be part of a realignment of financial incentives to reward clini-
cians and institutions in terms of how they practice and to encourage posi-
tive patient behaviors, but rather that it ought not be relegated to a single
unitary decision-making entity. Such a monopoly would be inappropriate
in a country as large and diverse as ours, and it would also be a political
nightmare for politicians.
As part of the need to realign financial incentives so that physicians and
other clinicians, as well as institutions, are rewarded for producing good
clinical outcomes, a first step could be to have information available from
comparative clinical effectiveness as part of a change in reimbursement
policies. This would be consistent with the development of discussions on
value-based insurance, where the amount of the copayment varies with the
likelihood of a good clinical outcome for a particular intervention. The
objective would be that procedures likely to produce good clinical out-
comes for patients in particular categories or subcategories of the popula-
tion would have low copayments or no copayments, while those medical
procedures unlikely to have good clinical outcome would be made more
expensive, although not disallowed. In this sense, comparative effective-
ness information would be considered not so much as a coverage issue as
a reimbursement issue.
Changes in the statutes governing Medicare would be required before
we could begin to think about copayments on a variable basis, but that
approach is better than some alternatives that have been proposed. Cur-
rently there is no statutory authority, when it comes to either coverage or
reimbursement, to allow the agency to introduce concepts of cost or cost
effectiveness. The ideas outlined here provide a way to introduce these
concepts into the reimbursement process and to do so in a way that allows
different private payers to use the information differently. It is one of the
many changes that will need to occur.
In addition to major investments in comparative clinical effectiveness, it
will be important to use a variety of strategies to improve evidence develop-
ment. One of the ways this could occur would be by tying the local coverage
decision making that now goes on under Medicare with evidence develop-
ment. There has been a great deal of discussion over the last couple of
decades about whether it is appropriate for carriers, the local payers, to have
their own coverage authority, at least on an interim basis, before a national
coverage decision is made. Before I had been sworn in as administrator of
the Health Care Financing Administration, now CMS, I had thought that
one of my goals would be to remove local coverage decision making on the
grounds that this is a national program and that the benefits ought to be
the same everywhere. In ensuing discussions, however, it became apparent
that to do so would introduce a very conservative bias to the coverage of
OCR for page 83
THE NEED AND POTENTIAL RETURNS FOR CER
new innovations and technologies in Medicare. If local carriers are going
to continue making interim coverage decisions, these decisions should only
occur with evidence development. Such a change would need to be statuto-
rily driven rather than have the agency attempt to do this administratively,
but it seems to be the kind of change that could be introduced to advance
evidence development in as many ways as is possible. This would make it
possible to harness the diversity that exists in the U.S. healthcare delivery
system in order to improve the knowledge base. The concept proposed here
is just one example of how to continue having the local coverage variation
that already exists, but to do so in ways that still contribute to knowledge
and, therefore, to improved decision making in the future.
In conclusion, the following steps need to occur. First, a center or entity
should be established that is charged with creating better information on
comparative clinical effectiveness. Initially, such a center could be started
with an investment of perhaps a few hundred million dollars. In the long
term, such a center would require funding on the order of several billion
dollars to sustain its effectiveness. Second, priority setting ought to be based
on both cost and geographic variation, at least as general guidance, but
with allowances made to include the economic and clinical burdens of dis-
ease in making decisions. Third, it should be recognized that all stakehold-
ers need to be a part of the decision-making process. Better to have them
on the inside participating in the decision making about what is analyzed
and how to treat various types of information than to have them attacking
the process from the outside.
As noted earlier, it is important to generate information on cost, but
the estimates should be done separately from the comparative clinical effec-
tiveness analysis, both in the public and the private sector. It is important
also to have credibility, objectivity, and transparency associated as much
with the cost analysis as with the generation of comparative clinical effec-
tiveness information. In addition, we need to recognize that as important
as it is to have information on clinical or cost effectiveness out there, the
necessary gains are not likely to be made unless the reimbursement system
is changed to make use of the information through value-based insur-
ance, through changing how we reimburse clinicians and institutions, and
through rewarding the kind of behavior that needs to be encouraged rather
than just paying more for doing more.
It will be particularly important to begin to give CMS the legislative
authority to introduce what is known about clinical and cost effectiveness
into its reimbursement decisions. As indicated, this would be preferable to
having comparative clinical effectiveness become part of the coverage deci-
sion, which is too heavy a burden to use going forward.
And finally, it is important to understand that if the growth in medical
spending is to be slowed from its current rate of 2.5 percent faster than
OCR for page 84
LEARNING WHAT WORKS
the economy to something more tolerable, it must be recognized that such
a slowing will mean less increased cashflow over time than industry, clini-
cians, institutions, and patients have been used to seeing come into the
system. None of them are likely to appreciate the consequences, and there
will likely be charges that clinicians are being prevented from providing the
best care possible to their patients. Not only may patient advocates and
clinicians feel they are being denied “the best care that is out there,” but
industry may also feel that it is being prevented from having the opportu-
nity to sell what could be the lifesaving or quality-improving strategy that
people want. Having credible information to indicate the contrary will be
a critical first step—although only a first step—if the United States is ever
to learn how to treat patients better and spend smarter.
REFERENCES
ACA (Patient Protection and Affordable Care Act). 2010. Public Law 111-148.
AcademyHealth. 2005. Placement, coordination, and funding of health services research within
the federal government. In AcademyHealth Report, September 2005. http://www.acad-
emyhealth.org/files/publications/placementreport.pdf (accessed September 3, 2010).
AdvaMed (Advanced Medical Technology Association). 2004. The medical technology indus-
try at a glance. http://www.omnex.com/training/iso13485/articles/ADVAMED-Medical-
Device-Industry-Facts.pdf (accessed September 3, 2010). P. 16, Chart 3.6.
Atkins, D., D. Best, P. A. Briss, M. Eccles, Y. Falck-Ytter, S. Flottorp, G. H. Guyatt, R. T.
Harbour, M. C. Haugh, D. Henry, S. Hill, R. Jaeschke, G. Leng, A. Liberati, N. Magrini,
J. Mason, P. Middleton, J. Mrukowicz, D. O’Connell, A. D. Oxman, B. Phillips, H. J.
Schunemann, T. T. Edejer, H. Varonen, G. E. Vist, J. W. Williams, Jr., and S. Zaza. 2004.
Grading quality of evidence and strength of recommendations. British Medical Journal
328(7454):1490.
Balas, E. A., and S. A. Boren. 2000. Managing clinical knowledge for healthcare improve-
improve-
ments. In Yearbook of medical informatics, edited by V. Schattauer. Stuttgart, Germany:
Schattauer Publishing. Pp. 65-70.
Biotechnology Industry Organization. 2006. Chart: Total biotechnology patents granted per
year. http://www.bio.org/speeches/pubs/er/statistics.asp (accessed July 15, 2010).
Buto, K., and P. Juhn. 2006. Can a center for comparative effectiveness information succeed?
Perspectives from a health care company. Health Affairs 25:w586-w588.
Califf, R. M. 2004. Defining the balance of risk and benefit in the era of genomics and pro-
teomics. Health Affairs 23(1):77-87.
CBO (Congressional Budget Office). 2007 (December). Research on the comparative ef-
fectiveness of medical treatments. http://www.cbo.gov/ftpdocs/88xx/doc8891/12-18-
ComparativeEffectiveness.pdf (accessed July 15, 2010).
Clancy, C. M. 2006. Getting to “smart” health care. Health Affairs 25(6):w589-w592.
FCC (Federal Coordinating Council for Comparative Effectiveness Research). 2009. Federal
Coordinating Council Report to the President and to Congress on comparative effec-
tiveness research. http://www.hhs.gov/recovery/programs/cer/cerannualrpt.pdf (accessed
September 3, 2010).
Fisher, E. S., and J. E. Wennberg. 2003. Health care quality, geographic variations, and the
challenge of supply-sensitive care. Perspectives in Biology and Medicine 46(1):69-79.
OCR for page 85
THE NEED AND POTENTIAL RETURNS FOR CER
Foster, L., D. Nanu, and S. Alcock. 2002. Patenting in the biopharmaceutical industry—
comparing the U.S. with Europe. Drug Plus International 1(1).
Gelijins, A. C., and N. Rosenberg. 1994. The dynamics of technological change in medicine.
Health Affairs 13(3):28-46.
Health Industry Forum. 2006. Comparative Effectiveness Forum: Executive summary. http://
healthforum.brandeis.edu/meetings/materials/2006-30-Nov./ExecBrief.pdf (accessed July
20, 2010).
Himmelstein, D., E. Warren, D. Thorne, and S. Woolhander. 2005. Illness and injury as
contributors to bankruptcy. Health Affairs (Web Exclusive):W5-W63. http://content.
healthaffairs.org/cgi/content/full/hlthaff.w5.63/DC1.
Hopayian, K. 2001. The need for caution in interpreting high quality systematic reviews. Brit-
ish Medical Journal 323(7314):681-684.
IOM (Institute of Medicine). 2000. To err is human: Building a safer health system. Washing-
ton, DC: National Academy Press.
———. 2001. Crossing the quality chasm. Washington, DC: National Academy Press.
———. 2007. Learning what works best: The nations need for evidence on comparative
effectiveness in health care. http://www.iom.edu/ebm-effectiveness (accessed August 5,
2010).
———. 2008. Knowing what works in health care: A roadmap for the nation. Washington,
DC: The National Academies Press.
———. 2009. Initial national priorities for comparative effectiveness research. Washington,
DC: The National Academies Press.
IOM Roundtable on Value & Science-Driven Health Care. 2009. Charter and vision state-
ment: Roundtable on value & science-driven health care. Washington, DC.
Jacobson, G. 2007. Comparative clinical effectiveness and cost effectiveness research: Back-
ground, history and overview. United States Congressional Research Service. http://
www.cbo.gov/ftpdocs/88xx/doc8891/12-18-ComparativeEffectiveness.pdf (accesed July
15, 2010).
KFF (Kaiser Family Foundation). 2009. Trends in health care costs and spending. http://www.
kff.org/insurance/upload/7692_02.pdf (accessed January 19, 2011).
Kupersmith, J., N. Sung, M. Genel, H. Slavkin, R. Califf, R. Bonow, L. Sherwood, N. Reame,
V. Catanese, C. Baase, J. Feussner, A. Dobs, H. Tilson, and E. A. Reece. 2005. Creating a
new structure for research on health care effectiveness. Journal of Investigative Medicine
53(2):67-72.
Litan, R. E. 2008. Vital signs via broadband: Remote health monitoring transmits savings,
enhances lives. http://www.betterhealthcaretogether.org/www/docs/broadband.pdf (ac-
cessed August 5, 2010).
Moses, H., 3rd, E. R. Dorsey, D. H. Matheson, and S. O. Thier. 2005. Financial anatomy of
biomedical research. Journal of the American Medical Association 294(11):1333-1342.
PhRMA (Pharmaceutical Research and Manufacturers of America). 2006. Pharma-
ceutical industry profile. P. 49, Table 4. http://www.phrma.org/files/attachments/
2006%20Industry%20Profile.pdf (accessed August 5, 2010).
Rowe, J. W., D. A. Cortese, and J. M. McGinnis. 2006. The emerging context for advances in
comparative effectiveness assessment. Health Affairs 25(6):w593-w595.
Tricoci, P., J. M. Allen, R. M. Califf, and S. C. Smith, Jr. 2009. Scientific evidence underlying
the ACC/AHA clinical practice guidelines. Journal of the American Medical Association
301(8):831-841.
Tunis, S. R., and S. D. Pearson. 2006. Coverage options for promising technologies: Medicare’s
“coverage with evidence development.” Health Affairs 25(5):1218-1230.
OCR for page 86
LEARNING WHAT WORKS
Wilensky, G. R. 2006. Developing a center for comparative effectiveness information. Health
Affairs 25(6):w572-w585.
Woolf, S. H. 2008. The power of prevention and what it requires. Journal of the American
Medical Association 299(20):2437-2439.
