Summary

Clinicians can no longer stay abreast of the rapidly expanding knowledge bases related to health. The number of randomized controlled trials published in MEDLINE (a medical literature database) grew from 5,000 per year in 1978–1985 to 25,000 per year in 1994–2001. Furthermore, contentions that much of the literature may be biased and not applicable to important subsets of target populations have caused its quality to be suspect. Overall, clinicians increasingly are barraged with a vast volume of evidence of uncertain value. Hence, critically appraised and synthesized scientific evidence has become fundamental to clinical practice. At the same time, and particularly under conditions of uncertainty regarding optimal decisions, clinician experiential knowledge and skill (the “art of medicine”) and patient values and preferences remain essential contributors to quality healthcare practice, in a complex interplay with science.

Clinical practice guidelines (CPGs) embody and support the interrelationships among these critical contributors to clinical decision making. Rather than dictating a one-size-fits-all approach to patient care, CPGs are able to enhance clinician and patient decision making by clearly describing and appraising the scientific evidence and reasoning (the likely benefits and harms) behind clinical recommendations, making them relevant to the individual patient encounter.

Although it remains important for CPGs to be evaluated fully for their effectiveness in improving health, when rigorously devel-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary Clinicians can no longer stay abreast of the rapidly expand- ing knowledge bases related to health. The number of random- ized controlled trials published in MEDLINE (a medical literature database) grew from 5,000 per year in 1978–1985 to 25,000 per year in 1994–2001. Furthermore, contentions that much of the literature may be biased and not applicable to important subsets of target populations have caused its quality to be suspect. Overall, clini- cians increasingly are barraged with a vast volume of evidence of uncertain value. Hence, critically appraised and synthesized scien - tific evidence has become fundamental to clinical practice. At the same time, and particularly under conditions of uncertainty regard- ing optimal decisions, clinician experiential knowledge and skill (the “art of medicine”) and patient values and preferences remain essential contributors to quality healthcare practice, in a complex interplay with science. Clinical practice guidelines (CPGs) embody and support the interrelationships among these critical contributors to clinical deci - sion making. Rather than dictating a one-size-fits-all approach to patient care, CPGs are able to enhance clinician and patient deci- sion making by clearly describing and appraising the scientific evi- dence and reasoning (the likely benefits and harms) behind clinical recommendations, making them relevant to the individual patient encounter. Although it remains important for CPGs to be evaluated fully for their effectiveness in improving health, when rigorously devel- 1

OCR for page 1
2 CLINICAL PRACTICE GUIDELINES WE CAN TRUST oped, they have the power to translate the complexity of scientific research findings into recommendations for clinical practice and potentially enhance healthcare quality and outcomes. However, the current state of CPG development has yet to meet this potential. CPG DEVELOPMENT CHALLENGES Clinical practice guidelines are ubiquitous in our healthcare sys- tem. The Guidelines International Network database currently lists more than 3,700 guidelines from 39 countries. Its U.S. counterpart, the National Guideline Clearinghouse (NGC), accepted 722 guide - lines to its database in 2008 alone, so that its total collection is nearly 2,700. CPG developers and users are characterized by varied organi- zations such as clinical specialty societies, disease advocacy groups, federal and local agencies, health plans, and commercial companies. However, CPGs suffer from shortcomings in the guideline develop- ment process, often compounding limitations inherent in their sci- entific evidentiary bases. Certain factors commonly undermine the quality and trustworthiness of CPGs. These include variable quality of individual scientific studies; limitations in systematic reviews (SRs) upon which CPGs are based; lack of transparency of develop - ment groups’ methodologies (particularly with respect to evidence quality and strength of recommendation appraisals); failure to con- vene multi-stakeholder, multi-disciplinary guideline development groups, and corresponding non-reconciliation of conflicting guide - lines; unmanaged conflicts of interest (COI); and overall failure to use rigorous methodologies in CPG development. Furthermore, evi- dence supporting clinical decision making and CPG development relevant to subpopulations, such as patients with comorbidities, the socially and economically disadvantaged, and those with rare condi- tions, is usually absent. More generally, the quality of CPG development processes and guideline developer adherence to quality standards have remained unsatisfactory and unreliable for decades. Non-standardized devel- opment results in substantial variation in clinical recommendations. At the same time, CPGs produced within a structured environment, in which a systematic procedure or “Guidelines for Guidelines” are available to direct production are more likely to be of higher quality. Furthermore, documentation of guideline development is enhanced by developer use of appraisal instruments or tools for systemati - cally assessing and reporting the quality of guideline development processes. While uniformly endorsed standards for clinical prac- tice guidelines development do not yet exist, there appears to be

OCR for page 1
3 SUMMARY widespread agreement regarding elements basic to quality CPG development. The concept that quality standards should inform CPG devel- opment is a pervasive concern globally, underscored by increasing calls for international standards to hasten rigorous CPG develop - ment and appraisal. Although a number and variety of guideline development appraisal tools (e.g., The Appraisal of Guidelines for Research and Evaluation [AGREE] Tool), which point to standards, are available, they inadequately reflect the full range of quality CPG development. They commonly focus on development process and form, with only a small number attending to the quality of evidence and the strength of recommendations. Furthermore, COI, the role of judgment in the derivation of recommendations, prioritization of the recommendations, development group composition, and how to assure patient-centeredness all lack sufficient attention in cur- rent standards for CPG development. These appraisal tools also are not designed for prospective application to guideline development. There are no agreed-on standards for prospective enhancement of high-quality, trustworthy clinical practice guidelines. COMMITTEE CHARGE In 2008, the Institute of Medicine (IOM) report Knowing What Works in Health Care recommended that the U.S. Secretary of Health and Human Services create a public–private program to develop (or endorse) and promote a common set of standards addressing the structure, process, reporting, and final products of systematic reviews of comparative effectiveness research and evidence-based clinical practice guidelines. Congress, through the Medicare Improve- ments for Patients and Providers Act of 2008, subsequently called on the Secretary to contract with the IOM, through the Agency for Health- care Research and Quality (AHRQ), to undertake two studies: (1) to “identify the methodological standard for conducting systematic reviews of clinical effectiveness research on health and health care in order to ensure that organizations conducting such reviews have information on methods that are objective, scientifically valid, and consistent,” and (2) to focus on “the best methods used in develop- ing clinical practice guidelines in order to ensure that organizations developing such guidelines have information on approaches that are objective, scientifically valid, and consistent.” The IOM formed two committees, the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research and the Committee on Standards for Developing Trustworthy Clinical

OCR for page 1
4 CLINICAL PRACTICE GUIDELINES WE CAN TRUST Practice Guidelines, to meet the above requests. The two commit- tees worked independently, but in coordination with each other, because the topics were related. While the SR committee attended exclusively to methods for SR development, from formulation of the research question to derivation of the final report, the CPG commit- tee worked from the premise that SRs reflecting the methodological standard, as defined by the SR committee, are instrumental to a trustworthy guideline development process. The CPG committee defined “standard” as a process, action, or procedure for developing CPGs that is deemed essential to pro- ducing scientifically valid, transparent, and reproducible results. The committee examined existing standards for guidelines develop- ment, assessing whether any would ensure development of trust - worthy clinical practice guidelines. Special attention was given to standards incorporating systems for appraising quality of evidence and strength of recommendations. The committee also considered methods for modifying CPGs for patients with multiple conditions; ways to reduce the number of overlapping guidelines and harmo- nize CPGs on the same topic; strategies to promote and evaluate adoption of development standards and trustworthy CPGs; means to distinguish trustworthy CPGs; and procedures for identifying guideline recommendations potentially appropriate for measuring the quality of healthcare systems or clinicians. CLINICAL PRACTICE GUIDELINES: A NEW DEFINITION The literature assessing the best methods for guideline devel - opment has evolved dramatically in the 20 years since the IOM’s first report on the subject, Clinical Practice Guidelines: Directions for a New Program, which defined CPGs as “systematically developed statements to assist practitioner and patient decisions about appro - priate health care for specific clinical circumstances.” The commit- tee saw the need to update this definition, in accordance with the AHRQ contract, and to better reflect current consensus on what constitutes a CPG, including aspects of guideline development that the committee believes are defining characteristics. The new defini- tion is as follows: Clinical practice guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assess- ment of the benefits and harms of alternative care options. To be trustworthy, guidelines should

OCR for page 1
5 SUMMARY • be based on a systematic review of the existing evidence; • be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups; • consider important patient subgroups and patient prefer- ences, as appropriate; • be based on an explicit and transparent process that mini- mizes distortions, biases, and conflicts of interest; • provide a clear explanation of the logical relationships be- tween alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of the recommendations; and • be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations. The new definition provides a clear distinction between the term “CPG” and other forms of clinical guidance derived from widely disparate development processes (e.g., consensus statements, expert advice, and appropriate use criteria). Furthermore, it underscores systematic review and both benefits and harms assessment as essen- tial characteristics of CPGs. Although the committee recognizes that other forms of clinical guidance may have value, addressing those other forms was beyond the scope of this report. Furthermore, the committee is aware that, for many clinical domains, high-quality evidence is lacking or even nonexistent. However, even given such constraints, guideline developers may still produce trustwor- thy CPGs if their development reflects those committee standards detailed below. STANDARDS FOR TRUSTWORTHY CLINICAL PRACTICE GUIDELINES As enumerated below, the committee’s proposed standards reflect the latest literature, expert consensus, and public comment, and the committee hopes they represent an important advance in the newest and best practice standards for CPG development. The committee expects its standards to be assessed for reliability and validity (including applicability), and to evolve as the science and experience demand. The committee has given increased attention to aspects of COI, such as details of guideline development group exclusions; aspects of guideline group composition, including train- ing of patient and consumer representatives in evidence appraisal; the specific nature of working relationships between systematic review teams and CPG developers; critical steps in establishing evi - dence foundations for clinical recommendations and rating recom-

OCR for page 1
6 CLINICAL PRACTICE GUIDELINES WE CAN TRUST mendations’ strength; external review of the CPG, including speci- fied mechanisms for ensuring public stakeholder comment; and elements essential to CPG updating, including ongoing monitoring and review of the CPG-relevant scientific literature and factors indi- cating the need for updates. The eight standards extend across the development process from conception to completion to revision. The standards should provide sufficient flexibility to be applicable to all guideline development groups (whether evidence in a particu- lar clinical area is lacking or abundant), unlike many development methodologies, which are specific to a particular guideline develop- ment entity and clinical problem. The committee’s eight proposed standards follow. STANDARDS FOR DEVELOPING TRUSTWORTHY CLINICAL PRACTICE GUIDELINES (CPGS) 1. Establishing Transparency 1.1 The processes by which a CPG is developed and funded should be detailed explicitly and publicly accessible. 2. Management of Conflict of Interest (COI) 2.1 Prior to selection of the guideline development group (GDG), individuals being considered for membership should declare all interests and activities potentially re- sulting in COI with development group activity, by writ- ten disclosure to those convening the GDG: • Disclosure should reflect all current and planned commercial (including services from which a clini- cian derives a substantial proportion of income), non- commercial, intellectual, institutional, and patient– public activities pertinent to the potential scope of the CPG. 2.2 Disclosure of COIs within GDG: • All COI of each GDG member should be reported and discussed by the prospective development group prior to the onset of his or her work. • Each panel member should explain how his or her COI could influence the CPG development process or specific recommendations. 2.3 Divestment • Members of the GDG should divest themselves of financial investments they or their family members have in, and not participate in marketing activities or advisory boards of, entities whose interests could be affected by CPG recommendations.

OCR for page 1
7 SUMMARY 2.4 Exclusions • Whenever possible GDG members should not have COI. • In some circumstances, a GDG may not be able to per- form its work without members who have COIs, such as relevant clinical specialists who receive a substan- tial portion of their incomes from services pertinent to the CPG. • Members with COIs should represent not more than a minority of the GDG. • The chair or cochairs should not be a person(s) with COI. • Funders should have no role in CPG development. 3. Guideline Development Group Composition 3.1 The GDG should be multidisciplinary and balanced, com- prising a variety of methodological experts and clinicians, and populations expected to be affected by the CPG. 3.2 Patient and public involvement should be facilitated by including (at least at the time of clinical question formu- lation and draft CPG review) a current or former patient, and a patient advocate or patient/consumer organization representative in the GDG. 3.3 Strategies to increase effective participation of patient and consumer representatives, including training in appraisal of evidence, should be adopted by GDGs. 4. Clinical Practice Guideline–Systematic Review Intersection 4.1 Clinical practice guideline developers should use sys - tematic reviews that meet standards set by the Institute of Medicine’s Committee on Standards for Systematic Reviews of Comparative Effectiveness Research. 4.2 When systematic reviews are conducted specifically to in- form particular guidelines, the GDG and systematic re- view team should interact regarding the scope, approach, and output of both processes. 5. Establishing Evidence Foundations for and Rating Strength of Recommendations 5.1 For each recommendation, the following should be provided: • An explanation of the reasoning underlying the rec- ommendation, including o a clear description of potential benefits and harms; o a summary of relevant available evidence (and evidentiary gaps), description of the quality (in- cluding applicability), quantity (including com-

OCR for page 1
8 CLINICAL PRACTICE GUIDELINES WE CAN TRUST pleteness), and consistency of the aggregate avail- able evidence; o an explanation of the part played by values, opin- ion, theory, and clinical experience in deriving the recommendation. • A rating of the level of confidence in (certainty regard- ing) the evidence underpinning the recommendation • A rating of the strength of the recommendation in light of the preceding bullets • A description and explanation of any differences of opinion regarding the recommendation 6. Articulation of Recommendations 6.1 Recommendations should be articulated in a standardized form detailing precisely what the recommended action is, and under what circumstances it should be performed. 6.2 Strong recommendations should be worded so that com- pliance with the recommendation(s) can be evaluated. 7. External Review 7.1 External reviewers should comprise a full spectrum of relevant stakeholders, including scientific and clinical ex- perts, organizations (e.g., health care, specialty societies), agencies (e.g., federal government), patients, and repre- sentatives of the public. 7.2 The authorship of external reviews submitted by individ- uals and/or organizations should be kept confidential un- less that protection has been waived by the reviewer(s). 7.3 The GDG should consider all external reviewer com- ments and keep a written record of the rationale for mod- ifying or not modifying a CPG in response to reviewers’ comments. 7.4 A draft of the CPG at the external review stage or imme- diately following it (i.e., prior to the final draft) should be made available to the general public for comment. Reasonable notice of impending publication should be provided to interested public stakeholders. 8. Updating 8.1 The CPG publication date, date of pertinent systematic evidence review, and proposed date for future CPG review should be documented in the CPG. 8.2 Literature should be monitored regularly following CPG publication to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.

OCR for page 1
9 SUMMARY 8.3 CPGs should be updated when new evidence suggests the need for modification of clinically important recom- mendations. For example, a CPG should be updated if new evidence shows that a recommended intervention causes previously unknown substantial harm; that a new intervention is significantly superior to a previously rec- ommended intervention from an efficacy or harms per- spective; or that a recommendation can be applied to new populations. The committee derived several recommendations directly rele- vant to the ultimate effectiveness of the eight standards in increasing the quality and trustworthiness of CPGs and enhancing healthcare quality and patient outcomes. RECOMMENDATIONS FOR IDENTIFYING AND EVALUATING TRUSTWORTHY CLINICAL PRACTICE GUIDELINES The committee views all eight proposed standards as essential elements in the development of trustworthy guidelines. Thus, the committee recommends the following: To be trustworthy, a clinical practice guideline should comply with proposed standards 1–8. Optimally, CPG developers should adhere to these proposed standards and CPG users should adopt CPGs compliant with these proposed standards. Some guideline developers will readily adapt their development process to embrace these eight standards; however, not all develop- ers will be able to do so, and a process of evolutionary adoption over time may be more practical. Although certain standards, such as those directed to patient and public involvement in the CPG development process and external review, may appear particularly resource intensive, strategies to increase effective public participa - tion can minimize this burden. The committee understands that the uniqueness of guideline development contexts may seemingly preclude certain developers from fully adhering to the standards the committee has proposed. For example, certain clinical areas (e.g., rare malignant tumors) are characterized by an exceptional dearth of scientific literature and an

OCR for page 1
10 CLINICAL PRACTICE GUIDELINES WE CAN TRUST urgent need to deliver patient care. The committee recognizes that developers in this instance may conclude they are unable to comply with Standard 4: “Clinical practice guideline developers should use systematic reviews that meet standards set by the IOM’s Committee on Standards for Systematic Reviews of Comparative Effectiveness Research.” However, SRs that conclude there are no high-quality randomized controlled trials or observational studies on a particular clinical question would still fulfill Standard 4. In all cases, whether evidence is limited or abundant, guideline development groups should comply with the complementary Standard 5: “Establishing evidence foundations for and rating strength of recommendations” by providing a summary of relevant available evidence (and evi- dentiary gaps), descriptions of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence; an explanation of the part played by values, opinion, theory, or clinical experience in deriving recommendations; a judgment regarding the level of confidence in (certainty regarding) the evidence underpinning the recommendations; and a rating of the strength of recommendations. For certain clinical areas, such as rare diseases, there may be no disease group or clinical specialty society with resources to develop trustworthy CPGs. In these cases, outside funding assistance could spur the development of needed guidelines. The committee urges organizations desiring to produce such guidelines to coordinate their efforts and pool resources with related organizations. This could also strengthen their efforts to seek support from founda- tions, government agencies, and other sources without conflict. The Department of Health and Human Services (HHS) should promote the identification of best practices in CPG development, guided by the committee’s proposed standards, and should assist in training individuals in specific technical skills needed in the CPG process, particularly patient and consumer representatives. Furthermore, to encourage the promulgation and adoption of standards, the committee recommends HHS create a mechanism to identify trustworthy guidelines. Such identification will serve three purposes, as follows: • Promote wider adoption of the IOM standards by develop- ers because there will be an advantage attached to CPGs publicly identified as trustworthy. • Provide CPG users with an easy guide to identify trustwor- thy ones. • Promote adoption of trustworthy CPGs.

OCR for page 1
11 SUMMARY The committee recommends The Secretary of HHS should establish a public–private mechanism to examine, at the request of developer organiza- tions, the procedures they use to produce their clinical practice guidelines and to certify whether these organizations’ CPG development procedures comply with standards for trustwor- thy CPGs. Although AHRQ is not directly involved in CPG development, it does play a vital role in the dissemination and evaluation of guidelines and creation of guideline development methodologies. The NGC is a highly useful guideline dissemination tool. AHRQ should continue to operate this service, and expand its capacities to provide syntheses of recommendations by clinical topic and con - duct research on best guideline development practices. As a central repository for all CPGs, the committee does not believe the NGC should be restricted to listing only those CPGs identified as trust- worthy. However, the NGC’s contribution may be of questionable value when listing guidelines providing too little information for an informed reader to judge quality and trustworthiness. To be a constructive resource, the NGC should eliminate CPGs for which trustworthiness cannot be determined, and identify the trustwor- thiness of those retained. Further, the committee recommends that AHRQ pilot-test and assess the reliability and validity of the IOM’s proposed standards, and evaluate their effects on healthcare quality and patient outcomes. The committee expects its standards to evolve as science and experience regarding CPG development demand. The committee recommends The Agency for Healthcare Research and Quality (AHRQ) should do the following: • Require the National Guideline Clearinghouse (NGC) to provide a clear indication of the extent to which clinical practice guidelines (CPGs) submitted to it adhere to stan- dards for trustworthiness. • Conduct research on the causes of inconsistent CPGs, and strategies to encourage their harmonization. • Assess the strengths and weaknesses of proposed IOM standards by pilot-test; estimate the validity and reliabil- ity of proposed standards; evaluate the effectiveness of interventions to encourage standards’ implementation; and

OCR for page 1
12 CLINICAL PRACTICE GUIDELINES WE CAN TRUST evaluate the effects of standards on CPG development, healthcare quality, and patient outcomes. RECOMMENDATIONS FOR INDIVIDUAL AND ORGANIZATIONAL INTERVENTIONS FOR CPG IMPLEMENTATION Promoting uptake and use of CPGs at the point of care rep- resents a final translation hurdle to move scientific findings into practice. An important guiding principle for promoting adoption of CPGs is that attributes of the CPG (e.g., ease of use, strength of the evidence) as perceived by users and stakeholders are neither stable features nor sure determinants of adoption. Rather it is the interac- tion among characteristics of the CPG (e.g., specificity, clarity), the intended users (physicians, nurses, pharmacists), and a particular context of practice (e.g., inpatient, ambulatory, long-term care) that determines rate and extent of adoption. Active dissemination and adoption strategies used by implementers to promote use of trust- worthy CPGs include academic detailing; audit and feedback and public reporting of performance; opinion leaders; clinical remind- ers and quick reference guides; payment mechanisms; and shared decision-making aides. Organizations and health systems can also provide necessary resources, workflow modifications, and infrastructures for CPG implementation by all relevant users, and engage clinician stake- holders in the implementation process. Fundamentally, for trustwor- thy guidelines to affect quality of care and patient outcomes they must be implemented; hence, the committee offers the following recommendation: Effective multifaceted implementation strategies targeting all relevant populations affected by CPGs, should be employed by implementers to promote adherence to trustworthy CPGs. Increased adoption of electronic health records and computer- aided clinical decision support (CDS) will offer unique opportuni- ties to rapidly move clinical knowledge from the scientific literature to the patient encounter. To achieve this goal, guideline develop- ers should structure CPGs to facilitate ready implementation of electronic clinical decision support by health systems (e.g., clinical practices, payers, delivery systems, hospitals). Furthermore, CPG developers should specify definitive and important gaps in scientific evidence for practice recommendations, including those relevant to

OCR for page 1
13 SUMMARY the target population, to facilitate understanding of potential limi - tations of clinical decision support. Formal organizational relation- ships among CPG developers, implementers, and CDS designers are encouraged to align requirements for CDS with the needs and stan - dards of CPG developers. The committee recommends guideline developers and implementers take the following actions to advance this aim: Guideline developers should structure the format, vocabulary, and content of CPGs (e.g., specific statements of evidence, the target population) to facilitate ready implementation of computer-aided CDS by end-users. CPG developers, CPG implementers, and CDS designers should collaborate in an effort to align their needs with one another. CONCLUSION Clinical decisions are made under uncertainty. Yet, as medi- cal, biomedical, and health services research advance, translation of scientific evidence increasingly reduces uncertainty in clinical practice. However, requisite to this promise are clinician and patient access to trustworthy clinical practice guidelines informed by high- quality evidence and a guideline development process reflective of best practices. The committee believes the eight standards proposed herein, when embraced by guideline developers, have the capac - ity to increase quality and trustworthiness of CPGs and ultimately enhance healthcare quality and patient outcomes.

OCR for page 1