Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Summary
In November 1989, Congress amended the Public Health Service Act to
create the Agency for Health Care Polipy and Research (AHCPR). Under
the terms of Public Law 101-239, this agency has broad responsibilities
for supporting research, data development, and other activities that will
"enhance the quality, appropriateness, and effectiveness of health care
seIvlces....
" The needs and priorities of the Medicare program are an
important but not exclusive focus of the agency.
Many of AHCPR's responsibilities formerly belonged to the National
Center for Health Services Research, which the agency replaced, but
the emphasis on outcomes and effectiveness research is now consider-
ably stronger. Other functions of the agency are new, in particular those
involving a joint public-pnvate enterprise to develop, disseminate, and eval-
uate practice guidelines under the sponsorship of the agency Forum for
Quality and Effectiveness in Health Care.
Shortly after its creation, AHCPR requested advice from the Institute
of Medicine (IOM) on how it might approach these new responsibilities
for practice guidelines. The IOM agreed to appoint a study committee that
would work quickly to provide technical assistance and advice on definition
of terms, specification of key attributes of good guidelines, and certain
aspects of planning for implementation and evaluation. This report largely
confines itself to these fairly specific and limited tasks. It is not a how-to-
do-it manual, a methodology text, a priority-setting exercise, or a primer
on guidelines for those seeking an introduction to the subject. The report
does, however, aim to encourage more standardization and consistent in
1
OCR for page 2
2
Cr [NICAL PRACTICE GUIDEr rNES
guidelines development, whether such development is supported directly by
the Forum or is undertaken independently by medical societies and other
organizations.
The committee believes that the AHCPR initiative, taken as a whole,
has real potential to advance the state of the art for practice guidelines,
strengthen the knowledge base for health care practice, and, hence, improve
the appropriateness and effectiveness of health care. One objective of this
report is to encourage realistic expectations about this potential by building
a broader understanding of the difficult but important steps needed to
move toward the goals for practice guidelines stated in P.L 101-239, or, as
it is also called, the Omnibus Budget Reconciliation Act of 1989 (OBRA
89~.
CONTEXT
The committee began its work with an understanding that the legis-
lation establishing AHCPR is one consequence of accumulated public and
private frustrations about the perceived health and economic consequences
of inappropriate medical care. These frustrations and perceptions stem
from many sources including ceaselessly escalating health care costs, wide
variations in medical practice patterns, evidence that some health services
are of little or no value, and claims that various kinds of financial, educa-
tional, and organizational incentives can reduce inappropriate utilization.
The combination of high expenditures and doubts about the value
of that spending explains polic~makers' interest in improving the scope
and application of knowledge about what works and what does not work
in medical car~and at what price. AHCPR is supporting an extensive
agenda of outcomes and effectiveness research. In fact, the major part
of the agen~y's work involves expanding the scope of knowledge rather
than applying it Of AHCPR's appropriation of nearly $100 million for
fiscal year 1990, it planned to obligate around $2 million for the Forum's
work on practice guidelines, compared with more than $30 million for
outcomes research. Still, the agency's responsibilities for practice guidelines
reflect congressional recognition of the practical need for ways to translate
knowledge into patient and practitioner decisions that improve the value
the nation receives for its health care spending.
More generally, the creation of a practice guidelines function within
AHCPR can be seen as part of a significant cultural shift, a move away
from unexamined reliance on professional judgment toward more structured
support and accountability for such judgment. Reflecting the first element
of this shift, guidelines are intended to assist practitioners and patients in
making health care decisions; reflecting the second aspect, they are to serve
OCR for page 3
SUMMARY
3
as a foundation for instruments to evaluate practitioner and health system
performance.
OVERVIEW OF PRACTICE GUIDELINES INITL\TIVES
Public and private activities related to practice guidelines can be con-
ceptualized, ideally, as having three basic stages: development, intervention,
and evaluation. The second and third stages should again ideally involve
feedback loops to the first stage to prompt the revision of guidelines when
omissions, technical obsolescence, or other problems with a set of guidelines
are identified. Guidelines are thus dynamic, not static. They reflect the
interplay of scientific and technological progress, real-world organizational
pressures, and changes in social values. ~ date, most government and
other initiatives emphasize the first of the three stages, the development of
guidelines.
PUBLIC INITIATIVES
Under OBRA 89, Congress created within AHCPR the Office of the
Forum for Quality and Effectiveness in Health Care. The Forum must
"arrange for" the development and periodic review and updating of
(1) clinically relevant guidelines that may be used by physicians, educatom, and
health care practitioners to assist in determining how diseases, disorders, and
other health conditions can most effectively and appropriately be prevented,
diagnosed, treated, and managed clinically; and
(V standards of quality, performance measures, and medical review cntena
through which health care providers and other appropriate entities may assess
or review the provision of health care and assure the quality of such care.
The phrase "arrange for" is one key indicator of the extent to which
the drafters of OBRA 89 sought to create a public-private enterprise with
respect to guidelines development. Their vision was that the Forum would
itself develop no guidelines; guidelines were not to be federal creations.
By January 1, 1991, the Forum must arrange for the development of
an initial set of guidelines, standards, performance measures, and review
criteria for at least three clinical treatments or conditions. The agency is
also responsible for updating the guidelines developed under its auspices.
Other key provisions of the legislation include the following.
.
The Forum can contract with public and nonprofit private organi-
zations to develop and update guidelines, or it can convene expert panels
to either develop guidelines or review guidelines developed by contractors.
· The director of the Forum must establish the standards for methods
and procedures to be followed by the contractors and expert panels. The
guidelines can be pilot-tested.
OCR for page 4
4
CLINICAL PRACTICE GUIDELINES
· Guidelines developed by private organizations independently of the
agency program may be adopted by contractors or expert panels if they
meet the requirements established by the legislation.
The director of the Forum is to promote dissemination of guide-
lines through organizations representing health care providers, health care
consumers, peer review organizations, accrediting bodies, and other appro-
priate entities. In addition, the guidelines must be presented In formats
appropriate for use by practitioners, medical educators, and medical care
reviewers.
· The Secretary of the Department of Health and Human Services
(DHHS) "shall provide for the use of the [initial sets of] guidelines . . . to
improve the quality, effectiveness, and appropriateness of care" provided
under the Medicare program. Presumably, providing for the use of the
guidelines will require that the Health Care Financing Administration
(HCFA) and its contracting fiscal intermediaries, carriers, and peer review
organizations take steps to incorporate review criteria and other evaluation
instn~ments into their programs for reviewing care provided to Medicare
beneficiaries.
.
· The director of the Forum is to conduct and support evaluations
of the impact of guidelines on clinical practice. More specifically, for the
guidelines developed by January 1, 1991, the Secretary of DHHS must
determine the impact of the initial sets of guidelines on the cost, quality,
appropriateness, and effectiveness of medical care provided under the
Medicare program and report the results to Congress by January 1, 1993.
With respect to the research agenda of AHCPR, the director of the
Forum is to recommend research projects related to the outcomes of health
care, the processes for developing guidelines, and the use of guidelines.
In general, when it examines clinical conditions as potential subjects
for guidelines, the agency is instructed to consider how guidelines, stan-
dards, performance measures, and review criteria can be expected to (1)
improve health for a significant number of individuals, (2) reduce clinically
significant variations in services and procedures provided by physicians, and
(3) reduce clinically significant variations in the outcomes of health care.
Moreover, the administrator of the agency is to consult with the Health
Care Financing Administration and ensure that the needs and priorities of
the Medicare program are appropriately reflected in the development of
guidelines. For the initial three sets of guidelines, the legislation provides
more specific selection priorities related to expenditures and services for
Medicare beneficiaries.
In addition to the new agency, other agencies of the federal govern-
ment have related responsibilities. These agencies include the National
Institutes of Health (NIH), the U.S. Preventive Services Ask Force, and
OCR for page 5
SUMMARY
the Health Care Financing Administration and its contracting carriers, fiscal
intermediaries, and peer review organizations.
PRIVATE INITIATIVES
Guidelines for clinical practice, broadly defined, are not new. The
processes of organized clinical education require various sorts of guide-
lines, as do the processes of professional licensure, board certification,
quality assurance, utilization review, and other aspects of health services
administration. However, the interest of the medical community and others
in practice guidelines has grown exponentially in recent years. Moreover,
there is greater emphasis today on formal procedures and methods for
arriving at a widely scrutinized and endorsed consensus.
Among the medical groups involved for some years with the develop-
ment of guidelines are the American Academy of Family Physicians, the
American College of Cardiology, the American College of Physicians, and
the American Society of Anesthesiologists. In the research community, the
RAND Corporation has pioneered the development of important tools and
strategies. Newer initiatives are being undertaken or planned by the Amer-
ican Board of Medical Specialties, the American Medical Association, the
Council of Medical Specialty Societies, many individual specialty societies,
and the academic medical and health services research community.
Insurers, health maintenance organizations (HMOs), utilization man-
agement firms, and similar organizations have not ignored the potential
of practice guidelines to serge as ~ basis for refusing payment for inap-
propriate care. For example, several years ago, the Blue Cross and Blue
Shield Association began its Medical Necessity Project, which worked with
researchers and some medical organizations to identity obsolete procedures
and set guidelines for the appropriate use of many diagnostic and treatment
services. The Health Insurance Association of America recently established
a similar function, and the Group Health Association of America has been
sponsoring programs on guidelines development. Individual members of
each of these associations are involved in additional efforts to develop or
adapt guidelines to meet the needs of their different health plans. The
activities of dozens of firms supplying utilization management services to
health plan sponsors have drawn attention to the quality, scope, and acces-
sib~lity of the criteria they use to review care on a prospective or concurrent
basis.
lithe guidelines development efforts of private organizations are thus
proceeding on many fronts. Some coordinating strategies are emerging,
but important problems remain unexplained conflicts among guidelines,
neglected topics, lack of follow-up, and incomplete public disclosure of the
evidence, participants, and methods used to develop sets of guidelines. No
OCR for page 6
6
CLINICAL PRACTICE GUIDELINES
independent entity exists to certify that guidelines are sound ~ method and
content, and no "national bureau of standards" is available to set standards
for methods of guidelines development or their content. The legislation
creating AHCPR is one response to some of these problems.
INSTITUTE OF MEDICINE COMMITTEE AND PROJECI
The major immediate goal of this project was to help the Forum
prepare to work with contractors or expert panels to meet its January 1991
deadline for developing at least three sets of guidelines. This assistance is
also meant to help the Forum assess the soundness of the products (the
guidelines) emerging from these contractors or panels. A second IOM
project, described at the end of this summary, will develop a practical
evaluation instrument for this purpose.
1b conduct the study requested by AHCPR, the IOM appointed a com-
mittee of experts in January 1990 that met in February and again in April.
The committee included practicing physicians, individuals experienced in
the development of guidelines, current and potential users of guidelines,
and representatives of relevant other disciplines such as nursing, law, and
economics. Staff from the Forum attended both committee meetings and
received copies of all draft and background materials prepared for the
committee.
FINDINGS AND CONCLUSIONS
STATE OF THE ART
The committee arrived at several general observations about the state
of the art of practice guidelines development. Most generally, the prm
cess of systematic development, implementation, and evaluation of practice
guidelines based on rigorous clinical research and soundly generated profes-
sional consensus, although progressing, has deficiencies in method, scope,
and substance. Conflicts in terminology and technique characterize the
field; they are notable for the confusion they create and for what they
reflect about differences in values, experiences, and interests among differ-
ent parties. Public and private development activities are multiplying, but
the means for coordinating these efforts to resolve inconsistencies, fill in
gaps, track applications and results, and assess the soundness of particular
guidelines are limited. Disproportionately more attention is paid to de-
veloping guidelines than to implementing or evaluating them. Moreover,
efforts to develop guidelines are necessarily constrained by inadequacies in
the quality and quantity of scientific evidence on the effectiveness of many
services.
OCR for page 7
SUMMARY
AHCPR AND THE FORUM
7
As a consequence of the above factors, AHCPR and the Forum have,
at present, a somewhat restricted foundation for their work. In addition,
at least three other variables must be taken into account in estimating
what the agency is likely to be able to accomplish early in its guide-
lines effort. First, although OBRA 89 addresses some concerns about
guidelines development, implementation, and evaluation, it appropriately
does not describe a precise course of action. The agency will need time to
devise and revise practical and defensible approaches to guidelines develop-
ment.
Second, given that both the function and the organizational units (par-
ticularly the Forum) are new to the Department of Health and Human
Services, the legislative timetables for guidelines development and, partic-
ularly, evaluation are unrealistically short. Moreover, the Forum has few
staff to support the new functions, a deficit that is not likely to change in the
near term. In the immediate future, these constraints and complications
are facts of life; the agency is acting to meet its deadlines in as timely and
meaningful a way as possible. Over the longer run, however, the committee
hopes that experience with the practicalities of guidelines development will
lead Congress and the agency to moderate the development and evaluation
timetables or to expand the resources devoted to the tasks, or both.
Third, within the government, meeting the challenge of developing
good practice guidelines cannot be solely the responsibility of the Fo-
rum. For instance, AHCPR's Medical Treatment Effectiveness Program
(MEDIEP) will generate information of immediate importance for prac-
tice guidelines. Lacunae in data identified during the guidelines develop-
ment process should highlight areas that AHCPR can target for research
funding. Outside AHCPR, the work of other agencies in the Public Health
Service (PHS), most notably NIH's randomized controlled trials, will be
essential to the long-term utility of guidelines, especially insofar as those
teals include broad measures of outcomes important to patients. Outside
the PHS, the agency needs to maintain close links with the Health Care
Financing Administration, in part because of provisions of OBRA 89 but
more importantly because HCFAs data on the Medicare population (and,
to a lesser extent, on the Medicaid population) should be valuable for
developing, implementing, and evaluating guidelines.
ROLES OF PUBLIC AND PRIVATE SECTORS
Despite the focus of this study on advice to a federal agency, the com-
mittee believes that the government's role in arranging for the development
OCR for page 8
8
CLINICAL PRACTICE GUIDELINES
of practice guidelines may in the end be fairly modest. Indeed, the contem-
poraneous efforts of many different organizations in the private sector may
significantly outpace what this one agency can do. The predominance of the
private sector should be even greater for guidelines implementation, where
most initiative must rest with private organizations and individuals. Even
when the government plays the principal role in funding and disseminating
guidelines on certain topics or clinical conditions, these guidelines will be
tailored or adjusted by providers, health plans, and others to reflect differ-
ent patient populations, delivery settings, practitioner sells and attitudes,
levels of resources, perceptions of risk, and other factors. The committee
expects that the processes of guidelines development, Implementation, and
evaluation will always need to be pursued by both the public and private
sectors.
RECOMMENDATIONS: DEFINITIONS
If the Forum is to proceed confidently with its mission, it needs clear
and broadly acceptable definitions of four key terms used in OBRA 89:
(1) practice guidelines, (2) medical review criteria, (3) standards of quality,
and (4) performance measures. Neither the final legislation nor preceding
House or Senate bills offered definitions of these particular terms, and
the literature on practice guidelines and related topics is characterized by
significant diversity in common and professional usage.
This report aims to provide definitions that are insofar as possible-
parsimonious, clear, not tautological, consistent with customary professional
and legislative usage, and socially and practically acceptable to important
interests. The committee recommends that the agency work with the
following definitions.
PRACTICE GUIDELINES are systematically developed statements to assist
practitioner and patient decisions about appropriate health care for specific
clinical circumstances.
MEDICAL REVIEW CRITERIA are systematically developed statements
that can be used to assess the appropriateness of specific health care
decisions, services, and outcomes.
STANDARDS OF QUALITY are authoritative statements of (1) minimum
levels of acceptable performance or results, (2) excellent levels of perfor-
mance or results, or (3) the range of acceptable performance or results
PERFORMANCE MEASURES (Provisional) are methods or instruments
to estimate or monitor the extent to which the actions of a health care
practitioner or provider conform to practice guidelines, medical review
criteria, or standards of quality.
OCR for page 9
SUMMARY
The committee recognizes that the definitions will not resolve all
arguments over what these and related terms mean, but it does believe
that these four statements will bring a degree of badly needed clarity and
uniformity to the field. Moreover, these definitions can be used by the
Forum and, indeed, have already been incorporated into its work
One underlying premise highlighted by these definitions is that these
four terms are not synonymous. Assisting physicians, nurses, other prac-
titioners, and patients in making decisions (through practice guidelines) is
not the same as evaluating practice (using medical review criteria, standards
of quality, and performance measures). Therefore, although the definitions
may evolve, it is important to underscore that the phrases and concepts
are not equivalent and should not be used interchangeably. For various
practical or technical reasons, some elements of a set of guidelines may
have no corresponding review criteria or other evaluation tools.
Not part of the committee's definition of practice guidelines, but central
to its view of the field, is the precept that every set of guidelines should be
accompanied by a statement of the strength of the scientific evidence and
the expert judgment behind them and by projections of the relevant health
and COSt outcomes. The committee has not tried to distinguish types or
levels of practice guidelines (for example, Levels 1 or 2), although this type
of discrimination may be useful.
RECOMMENDATIONS: ATTRIBUTES OF GOOD GUIDELINES
Developing practice guidelines is a challenging task that requires di-
verse skills ranging from analysis of scientific evidence tO management of
group decisionmaking to presentation of complex information in under-
standable forms. Jo arrange for the development of guidelines by expert
panels and contractors, the Forum must be able tO state its expectations
for the process and then assess the soundness of the resulting products.
(OBRA 89 calls this establishing "standards and criteria" for the process. 1b
avoid confusion, this report substitutes the term attributes for the statutory
language. As described at the end of this summary, the IOM is preparing
in provisional form a practical assessment instrument for the agency.)
This report distinguishes between the priorities for selecting particular
targets for guidelines and the desirable attributes of guidelines. Priority
setting is a crucial but separate task for which OBRA 89 provides guidance
Drawing on its members' experience and expertise and the work of
past IOM committees and other relevant organizations, the committee
recommends that the agency use the following eight attributes, properties,
or characteristics to instruct expert panels or contractors and to assess their
products. The attributes focus on practice guidelines and not on medical
review criteria and other tools for evaluating practice. The committee
OCR for page 10
10
CONICAL PRACTICE GUIDELINES
expects this set of attributes to be tested, reassessed, and, if necessary,
revised.
VALIDITY: Practice guidelines are valid if, when followed, they lead to the
health and cost outcomes projected for them, other things being equal A
prospective assessment of validity will consider the projected health out-
comes and costs of alternative courses of action, the relationship between
the evidence and recommendations, the substance and quality of the sci-
entific and clinical evidence cited, and the means used to evaluate the
evidence.
RELIABILIlY/REPRODUCIBILIlY: Practice guidelines are reliable and
reproducible (1) if-given the same evidence and methods for guidelines
development" another set of experts would produce essentially the same
statements and (2) if given the same circumstances the guidelines are
interpreted and applied consistently by practitioners or other appropriate
parties. A prospective assessment of reliability may consider the results of
independent external reviews and pretests of the guidelines.
CLINICAL APPLICABILITY: Practice guidelines should be as inclusive of
appropriately defined patient populations as scientific and clinical evidence
and expert judgment permit, and they should e~nlicitiv.stnte the nnn~,l~tion~
to which statements apply.
--a ~a---- ~~~~~ ~~ r-r~~~~~~
CLINICAL FLEXIBILITY: Practice guidelines should identify the specifi-
cally known or generally expected exceptions to their recommendations.
C=RI1Y: Practice guidelines should use unambiguous language, define
terms precisely, and use logical, easy-to-follow modes of presentation.
MULTIDISCIPLINARY PROCESS: Practice guidelines should be devel-
oped by a process that includes participation by representatives of key
affected groups. Participation may include seating on panels that develop
guidelines, providing evidence and viewpoints to the panels, and reviewing
draft guidelines.
SCHEDULED REVIEW: Practice guidelines should include statements
about when they should be reviewed to determine whether revisions are
warranted, given new clinical evidence or changing professional consensus.
DOCUMENTATION: The procedures followed in developing guidelines,
the participants involved, the evidence used, the assumptions and rado-
nales accepted, and the analytic methods employed should be meticulously
documented and described.
OCR for page 11
SUMMARY
11
The themes that under these attributes are credibility and account-
abili~. The link between a set of guidelines and the scientific evidence must
be explicit, and scientific and clinical evidence should take precedence over
expert judgment. Every set of guidelines should be accompanied by a clear
statement of the strength of the relevant scientific evidence and expert
judgment. Linen the empirical evidence has important limitations and ex-
perts reach conclusions that are not consistent with the evidence, then the
rationale for departing from the evidence, such as it is, should be carefully
explained. When expert judgment proceeds ~ the absence of direct em-
pirical evidence about a particular clinical practice, as is often necessary,
the general scientific reasoning and normative principles supporting the
judgments should be described.
This report does not take a position on whether cost considerations
should be explicitly factored into practice recommendations, although some
committee members had strong views that they should be. However, doc-
umentation of projected health outcomes and costs to the degree possible
is important to help developers and users of guidelines better understand
the implications of following or not following the guidelines.
One of the committee's strongest recommendations is that the process
of developing guidelines include participation by representatives of key
affected groups and disciplines. Such participation increases the likelihood
(1) that all relevant scientific evidence will be located and critically evalu-
ated; (2) that practical problems with using the guidelines will be identified
and addressed; and (3) that affected groups will see the guidelines as
credible and will cooperate in implementing them. Participation by physi-
aans, nurses, patients, and others can be achieved in several ways including
membership on the development panel, testimony at public hearings, par-
ticipation in focus groups, consultation during site visits, and provision of
comments on draft guidelines.
The stringency of the attributes, especially taken together, is well
recognized. Realistically, neither existing guidelines nor those likely to
be developed by the agency in the foreseeable future will "score well"
on all eight properties simultaneously. Indeed, near-perfect scores may
always lie in the realm of aspiration rather than attainment. Moreover, the
process of developing, assessing, using, evaluating, and revising guidelines
will be evolutionary. There is today no proven "right way" to conduct
this endeavor, even if there clearly are some "better ways." In addition, a
balance needs to be maintained between an ideal process and one that is
feasible. Guidelines that satisfactorily reflect the eight attributes proposed
here may not be products of an ideal process? but in the committee's view
they will be defensible.
OCR for page 12
~2
Cr JNICAL PRACTICE GUIDEr rNES
RECOMMENDATIONS: IMPLEMENTATION AND EVALUATION
Although AHCPR and the Forum are responsible for implementing the
government program for guidelines established in OBRA 89, the main work
of implementing the guidelines themselves will be in the hands of physicians,
nurses, health care administrators, and others. The agency, however, has
important responsibilities for evaluating the impact of guidelines. The
committee's discussions were limited by its charge and centered primarily
on how the processes of implementation and evaluation can reinforce and
extend the eight attributes of guidelines defined earlier.
One committee recommendation is that the agency instruct its expert
panels and contractors to keep implementation and evaluation in mind
as they develop guidelines. The tension between extraordinarily detailed
or sophisticated guidelines and those that can be translated into usable
medical review criteria or patient education materials must be recognized
and dealt with during the development process, not after the fact.
Keeping implementation and evaluation in mind during guidelines
development means, among other things, understanding the following.
· The credibility of the development process, the participants, and
the scientific grounding of guidelines must be clear to intended users.
· A truly multidisciplinary approach to guidelines development will
facilitate acceptance and use of guidelines by members of the groups
represented and by other, secondary target groups.
· Guidelines should be specific, comprehensive, and flexible enough
to be useful in the varied settings and circumstances of everyday medical
practice and in the evolving programs to assess the appropriateness of care
provided in these settings.
· Guidelines language, logic, and symbols should be easy to fol-
low and unambiguous, so that movement from guidelines statements to
educational tools, review criteria, or other instruments is unimpeded.
· The guidelines should specie what information about the clinical
problem, the patient's circumstances and preferences, and the delivery
setting should be recorded to permit later evaluation of the appropriateness
of care Judged against criteria generated from the guidelines).
The implementation of guidelines is a diffuse, difficult-to-track process
that will depend on many factors besides the quality and credibility of
the guidelines. Among those factors are (1) the funding for dissemination
and other implementation activities; (2) the supports and incentives for
the guidelines to be used by physicians, nurses, health plans, and others;
(3) the accessibility, scope, accuracy, and timeliness of a variety of intra-
and interorganizational information systems; and (4) the ability of mul-
tiple parties to plan and execute the various steps needed to implement
guidelines.
OCR for page 13
SUMMARY
13
Users of guidelines will vary in their objectives and circumstances, and
strategies for meeting particular user objectives will differ in their cost-
effectiveness and manageability. Thus, different objectives and resources
may call for different choices among the formats for guidelines (that is, their
physical layout and logic), different roles for the available dissemination
media, and different kinds of administrative supports for users of guidelines.
Organizations with more resources (for example, libraries, video centers,
telephone hotlines, personal computers, and network information systems)
will be able to assist the use of guidelines in ways that are out of reach for
less resource-rich organizations.
Medical review criteria and other evaluation instruments, if properly
developed and sensitively applied, can create incentives for adherence
to practice guidelines. If improperly developed and applied, they can
undermine support for practice guidelines. Building on earlier IOM reports
on utilization management and quality assurance, the committee discussed
a few broad principles for the constructive use of medical review criteria
and other evaluation tools derived from practice guidelines.
First, review criteria should be public with respect to their content and
their development process. Second, when criteria are used to assess quality
of care, deny payment for specific services, or take similar steps, an appeals
process must be provided that is free from unreasonable complexity, delay,
or other barriers. Third, review organizations should provide constructive
information and feedback to physicians and other clinicians with the aim
of improving practice rather than punishing missteps.
Fourth, review organizations should make their renew activities as
manageable and nonintrusive as possible. Such organizations contribute to
the perceived and real "hassle factor'' in medical care, which grows out
of burgeoning demands by payers and others for more information on,
and justification for, health services delivered or proposed. The Forum
needs to be sensitive to this issue. It should also work with HCFA and
other organizations to minimize negative effects from poor translation of
otherwise good guidelines into review criteria, unduly stringent application
of these criteria, or both.
The committee recognizes the critical importance of strong systems
support for implementation. The complexities of creating such support
within and across organizations are beyond the SGOpe of this report but will
be an important issue in further IOM work on practice guidelines.
With respect to the evaluation responsibilities of AHCPR and the
Forum, the committee believes that the OBRA 89 provisions for evaluation
are laudable but that the timetable is unrealistic. The Secretary of Health
and Human Services is due to report on the impact of the first three
guidelines by January 1, 1993. It is unlikely that measurable effects of
these guidelines on the costs or quality of care will appear that quickly, and
OCR for page 14
~4
CLINICAL PRACTICE GUIDE~N:ES
even if they do, it is unlikely that appropriate data on patient outcomes
and program costs will be available and analyzed. Instead of a full-fledged
evaluation, the agency can more reasonably be expected to provide a report
on its evaluation plan, the steps being taken to implement the plan, and
any preliminary evidence of impact.
DIVERSITY IN CLINICAL PRACTICES AND GUII)ELINES
In its discussions, the committee repeatedly returned to questions of
diversity in clinical practice and inconsistency among guidelines. Diversity
in clinical practice can be acceptable or unacceptable. It may be reasonable
when the scientific evidence to support different courses of care is uncertain.
In addition, some degree of diversity may be warranted by differences in
individual patient characteristics and preferences and variations in delivery
system capacities related to locale, resources, and patient populations.
However, even though practice variation based on scientific uncertainty
or differences in values may be acceptable, both science and values are
open to change. Thus, what is perceived as acceptable diversity in clinical
practice may change over time.
Diversity in practice is unacceptable when it stems from poor practi-
tioner skills, poor management of delivery systems, ignorance, or deliberate
disregard of well-documented preferable practices. It should not be toler-
ated when it is a self-serving disguise for bad practices that harm people or
waste scarce resources.
Guidelines can clarify what is acceptable and unacceptable variation in
clinical practice, but that clarification itself has limits that may lead different
groups to different and even inconsistent guidelines. Weak evidence is still
weak evidence, although the processes described in Chapter 3 should
allow the best use of whatever evidence is available. Nonetheless, these
processes still leave room for differences of expert option about such issues
as whether a flaw in research design "matters" or whether differences in
results between two treatment alternatives are "clinically important" or only
"statistically significant."
Inconsistency among guidelines can also arise from variations in values
and tolerance for rise People may simply differ in how they perceive
different health outcomes and how they judge when benefits outweigh harms
enough to make a seIvice worth providing. One way to approach this kind
of variation is to try to establish practitioner and patient attitudes toward
different benefits and harms and then identify what is known about the
probabilities of those different outcomes. In some cases, the developers of
guidelines may take the further step of applying their own values, but others
considering the guidelines later could look at the same information and
perhaps come to different conclusions. Also, for some services and clinical
OCR for page 15
SUMMARY
15
conditions, the developers of guidelines may choose not to recommend a
single course of action but to lay out alternative courses of treatment that
may be appropriate depending on, for example, the preferences of a patient
or the characteristics of a delivery setting or community.
In sum, merely identifying inconsistencies in guidelines says nothing
about the legitimacy of those inconsistencies. Some inconsistencies may
arise from biased or inept development processes. Some may result from
reasonable differences in the interpretation of scientific evidence or in the
application of patient, practitioner, or social values. Other inconsistencies
may essentially disappear when the rationales for specific recommendations
are closely examined. The challenge is to determine which explanation
applies. Meticulous documentation of the evidence and rationales for
guidelines will make this dete~ination easier.
EXPECTATIONS FOR PRACTICE GUIDELINES
Today the field of guidelines development is a confusing mix of high
expectations, competing organizations, conflicting philosophies, and ill-
defined or incompatible objectives. It suffers from imperfect and incom-
plete scientific knowledge as well as imperfect and uneven means of applying
that knowledge. Despite the good intentions of many involved parties, the
enterprise lacks clearly articulated goals, coherent structures, and credible
mechanisms for evaluating, improving, and coordinating guidelines devel-
opment to meet social needs for good-quality, affordable health care.
This situation will not change overnight, even though many promising
activities, including those sponsored by AHCPR, are under way. Thus,
expectations of quick results should be restrained. Otherwise, dashed
hopes may lead to calls for premature abandonment of a useful strategy
for improving the appropriate use of health services and to the adoption
of more draconian measures to control costs.)
The committee is also concerned about other expectations or assump-
tions that may be unrealistic. One such assumption is that guidelines
development is a relatively simple or straightforward undertaldng. It is
not. For many clinical conditions and services, the science base is limited.
Methods for analyzing evidence and developing expert opinions vary, but
none of the rigorous methods can be properly applied by novices. Even
in cases where considerable research has been done and sound methods
1 In this vein, see the 1973 book by political scientists Jeered L Pressman and Aaron B. W~l-
davsly, Implementation: How Great ~ectanons ~ Washington Are Dashed in Oakland,. Or, Why
It's A~na~ng that Federal Programs Work at All, Thus Being a Saga of Die Economic Development
A~nmistranon as Told by Two Syrnpathenc Observers Who Seek to Build Morals on a Faundanon
of Rained Hopes (Berkeley, Calif.: University of California Press).
OCR for page 16
16
CLINICAL PRACTICE GUIDELINES
applied tO analyze it, honest clinicians and analysts may Still come to dif-
ferent conclusions from the same evidence. Agreement on facts may not
be matched by agreement on what health benefits are desirable at which
economic cost with what tolerable accompanying health risks.
Such conflicts about the interpretation of evidence and application of
value judgments cannot be ignored. Indeed, the whole process of guidelines
development has to be undertaken with great care at every stage: selecting
participants, clarifying biases, adopting procedures and methods, identifying
and analyzing evidence, considering alternatives, providing for independent
reviews, preparing clear recommendations, and disclosing all important
information about the process. Most of this report reinforces these points.
A second assumption of concern to the committee is that there is
one right way to develop guidelines. There is not. Vanations in the
topics, the clinical disciplines involved, the purposes, and the audiences
for guidelines will jUSti~ some differences ~ the specific methods for
developing guidelines. However, tO grant some methodological diversity is
not tO accept all approaches as equally good. Much remains to be tried
and learned about the strengths and weaknesses of different methods.
A further questionable expectation, which is sometimes explicit but
often unstated, is that practice guidelines will help control health care
costs. They may not. The reasons for caution on this point are several. For
instance, variation in practice does not, by itself, demonstrate that high-use
patterns are necessarily the inappropriate ones. Moreover, even if high use
can be identified as inappropnate, such identification does not automatically
change behavior. An array of incentives for behavior change may be tried,
but not all will succeed. Even if behavior changes, expenditures may not.
Some guidelines undoubtedly will save money by reducing the use of
inappropriate services; some will increase costs by encouraging more use
of underutilized services; and some will shift COStS from one service or
place or payer tO another. The net impact on costs cannot be predicted
with confidence, even if the priorities for guidelines development focus
on clinical conditions for which overuse of expensive services is suspected.
Nevertheless, if guidelines do succeed in improving the appropriateness
and hence the vale of this country's expenditures for medical care, then
the endeavor will be a success.
This committee believes that AHCPR's practice guidelines effort has
real potential to advance the state of the art in this field. The conditions
for such success are demanding but not out of reach. In particular, expecta-
tions for the agency and for practice guidelines per se- must be realistic
regarding timetables and results. All parties concerned must act in good
faith and keep the credibility and accountability of their actions in mind.
Strict regard for the scientific rigor of the process is critical as is avoid-
ance of premature closure on a single method of guidelines development.
OCR for page 17
SUM~4RY
17
Attention tO implementation and evaluation needs tO be factored into the
development process at an early stage.
The Forum can underscore itS intent tO examine critically and improve
its program and products in at least three ways. First, it should ask its expert
panels for feedback on the strengths and weaknesses of the procedures
followed. Second, it should pretest (or arrange for the pretesting of) all
guidelines developed under its aegis. This can be done on a pilot basis in a
real delivery setting, on a set of prototypical cases, or through both methods.
Third, it should try to evaluate the effectiveness of intermediate actions
(for example, formatting, dissemination, incentives) that are necessary if
guidelines are to have their intended effects on health practices, outcomes,
and costs. Each of these steps can be part of a learning process for the
Forum and others.
NEXT STEPS FOR TlIE INSTITUTE OF MEDICINE
In May 1990, a new IOM committee began an 18-month study of the
development, implementation, evaluation, and revision of clinical practice
guidelines. Many of the issues raised in this report will be examined
in depth during this second project, which is supported by the John A.
Hartford Foundation, Inc., and the Public Health Service. In preparing its
report and recommendations, the new committee will
.
describe existing initiatives to develop, implement, and evaluate
practice guidelines;
· identify the strengths and limitations of these efforts in light of the
objectives and concerns of specific interest groups and society in general;
describe different models of public and private action that might
serve as prototypes for better structuring activities related to guidelines;
· analyze and assess the strengths, weaknesses, uncertainties, and
trade-offs of different models in responding to identified problems and
objectives; arid
propose a framework for better structuring the development, im-
plementation, evaluation, and revision of practice guidelines.
.
.
In addition, the new committee will propose a practical methodology
for AH:CPR and others tO employ in assessing guidelines before recom-
mending or using them. It will focus on how the guidelines were developed,
their scientific basis, their relevance to clinical practice, their Claris, and
other characteristics. Such initial assessments will not substitute for later
evaluations by government and others of the impact of a set of guidelines.
The committee's recommendations will cover both government and
private activities, and its report will identify legislative, management, and
OCR for page 18
18
CQ1VIG4L PRACTICE GUIDELINES
other steps necessary lo implement the recommendations. An active pro-
gram of disseminating the committee's findings and recommendation
planned. The committee report should be released in the fall of 1991.
FINAL COMMENT
Fulfilling the mission and potential of the Forum will require heroic
effort from a small staff, serious commitment from participants in the
expert panels, and honest and practical support from the many involved
and interested parties. Serious conceptual and practical issues remain to
be confronted and resolved on a tight timetable with a modest budget.
The undertaking will be strengthened if expectations are neither naively
optimistic nor cynically pessimistic. This report suggests some of the ways in
which the agency with the help of many others and a stance of constructive
realism-can move to meet its mandate.
Representative terms from entire chapter:
guidelines development
