Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Clinical Practice Guidelines We Can Trust CLINICAL PRACTICE GUIDELINES WE CAN TRUST Committee on Standards for Developing Trustworthy Clinical Practice Guidelines Board on Health Care Services Robin Graham, Michelle Mancher, Dianne Miller Wolman, Sheldon Greenfield, and Earl Steinberg, Editors INSTITUTE OF MEDICINE OF THE NATIONAL ACADEMIES THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu
OCR for page R2
Clinical Practice Guidelines We Can Trust THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract No. HHSP23320042509XI between the National Academy of Sciences and the Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project. Library of Congress Cataloging-in-Publication Data Institute of Medicine (U.S.). Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical practice guidelines we can trust / Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, Institute of Medicine of the National Academies ; Robin Graham ... [et al.], editors. p. ; cm. Includes bibliographical references. ISBN 978-0-309-16422-1 (pbk.) — ISBN 978-0-309-16423-8 (pdf) 1. Medicine—Practice—Standards—United States. 2. Total quality management—United States. I. Graham, Robin, 1952- II. Title. [DNLM: 1. Practice Guidelines as Topic—standards—United States. 2. Total Quality Management—standards—United States. W 84.4 AA1] R728.I47 2011 610.68—dc23 2011017454 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. For more information about the Institute of Medicine, visit the IOM home page at: www.iom.edu. Copyright 2011 by the National Academy of Sciences. All rights reserved. Printed in the United States of America The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin. Cover credit: Photograph by Mai Le. Suggested citation: IOM (Institute of Medicine). 2011. Clinical Practice Guidelines We Can Trust. Washington, DC: The National Academies Press.
OCR for page R3
Clinical Practice Guidelines We Can Trust “Knowing is not enough; we must apply. Willing is not enough; we must do.” —Goethe INSTITUTE OF MEDICINE OF THE NATIONAL ACADEMIES Advising the Nation. Improving Health.
OCR for page R4
Clinical Practice Guidelines We Can Trust THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org
OCR for page R5
Clinical Practice Guidelines We Can Trust COMMITTEE ON STANDARDS FOR DEVELOPING TRUSTWORTHY CLINICAL PRACTICE GUIDELINES SHELDON GREENFIELD (Chair), Donald Bren Professor of Medicine and Executive Director, Health Policy Research Institute, University of California–Irvine EARL PHILLIP STEINBERG (Vice Chair), Senior VP, Clinical Strategy, Quality and Outcomes, WellPoint, Inc., and President and CEO, Resolution Health, Inc., Columbia, MD ANDREW AUERBACH, Associate Professor of Medicine in Residence, School of Medicine, University of California–San Francisco JERRY AVORN, Professor of Medicine, Harvard Medical School and Chief, Division of Pharmacoepidemiology and Pharmaeconomics, Brigham and Women’s Hospital, Boston, MA ROBERT S. GALVIN, Chief Executive Officer, Equity Healthcare, and Executive Director, Corporate Private Equity, The Blackstone Group RAYMOND GIBBONS, Consultant in CV Disease and Internal Medicine, Mayo Clinic, Rochester, MN JOSEPH LAU, Director, Tufts EPC, Tufts Medical Center, Boston, MA MONICA MORROW, Chief, Breast Service and Anne Burnett Windfohr Chair of Clinical Oncology, Memorial Sloan-Kettering Cancer Center, and Professor of Surgery, Weill Medical College of Cornell University, New York, NY CYNTHIA D. MULROW, Senior Deputy Editor, Annals of Internal Medicine, American College of Physicians, San Antonio, TX ARNOLD J. ROSOFF, Professor of Legal Studies & Health Care Management, The Wharton School; Professor of Family Medicine and Community Health, School of Medicine; and Senior Fellow, The Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA JOHN SANTA, Director, Consumer Reports Health Ratings Center, Yonkers, NY RICHARD N. SHIFFMAN, Professor of Pediatrics, Yale School of Medicine, New Haven, CT WALLY SMITH, Professor and Chair, Division of Quality Health Care, and Scientific Director, Center on Health Disparities, Virginia Commonwealth University, Richmond, VA WALTER F. STEWART, Associate Chief Research Officer, Center for Health Research, Geisinger Health System, Danville, PA ELLEN STOVALL, Senior Health Policy Advisor, National Coalition for Cancer Survivorship, Silver Spring, MD BRIAN L. STROM, Chair and Professor, Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA
OCR for page R6
Clinical Practice Guidelines We Can Trust MARITA G. TITLER, Professor and Associate Dean, Office of Practice and Clinical Scholarship, University of Michigan School of Nursing, Ann Arbor, MI Study Staff ROBIN GRAHAM, Senior Program Officer, Study Director DIANNE MILLER WOLMAN, Senior Program Officer (through December 2010) MICHELLE MANCHER, Research Associate ALLISON McFALL, Senior Program Assistant (through August 2010) JILLIAN LAFFREY, Senior Program Assistant (starting July 2010) ROGER HERDMAN, Director, Board on Health Care Services Consultants DAVID ATKINS, Director, Quality Enhancement Research Initiative, Veterans Health Administration RONEN AVRAHAM, Thomas Shelton Maxey Professor of Law, University of Texas School of Law JONATHAN D. DARER, Medical Director of Clinical Transformation, Innovation, Geisinger Health System MARTIN ECCLES, Professor of Clinical Effectiveness and The William Leech Professor of Primary Care Research, Institute of Health and Society, Newcastle University JEREMY GRIMSHAW, Director, Clinical Epidemiology Program, Ottawa Hospital Research Institute J.B. JONES, Research Investigator I, Geisinger Center for Health MORI KRANTZ, Associate Professor, Cardiology, University of Colorado, Denver Health Medical Center WILLIAM SAGE, Vice Provost for Health Affairs and James R. Dougherty Chair for Faculty Excellence in Law, University of Texas–Austin ANNE SALES, Adjunct Professor, Department of Family Medicine, Faculty of Medicine and Dentistry, University of Alberta HOLGER SCHÜNEMANN, Chair, Department of Clinical Epidemiology & Biostatistics, McMaster University PAUL SHEKELLE, Medical Center Director, Quality Assessment and Improvement Program, RAND Health DEAN SITTIG, Professor, School of Biomedical Informatics, UT–Memorial Hermann Center for Healthcare Quality & Safety, University of Texas Health Science Center at Houston LEIF SOLBERG, Director for Care Improvement Research, HealthPartners Research Foundation STEVE WOOLF, Director, VCU Center on Human Needs, Professor of Family Medicine, Virginia Commonwealth University
OCR for page R7
Clinical Practice Guidelines We Can Trust Reviewers This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: JESSE A. BERLIN, Vice President, Epidemiology, Johnson & Johnson Pharmaceutical Research and Development RICHARD BERMAN, President and CEO, LICAS JAKO BURGERS, Senior Researcher, IQ healthcare, Radboud University, Nijmegen Medical Centre MARJORIE FUNK, Professor, Yale School of Nursing ALAN M. GARBER, Henry J. Kaiser, Jr. Professor and Professor of Medicine, Director, Center for Health Policy, Director, Center for Primary Care and Outcomes Research, Stanford University MARGUERITE A. KOSTER, Practice Leader, Technology Assessment and Guidelines Unit, Kaiser Permanente Southern California
OCR for page R8
Clinical Practice Guidelines We Can Trust GREG PAWLSON, Executive Vice President, National Committee for Quality Assurance DAVID RANSOHOFF, Professor of Medicine, Clinical Professor of Epidemiology, Director, Clinical Research Curriculum (K30), University of North Carolina School of Medicine CAROL SAKALA, Director of Programs, Childbirth Connection VINCENZA SNOW, Medical Director, Vaccines, Specialty Care Medical Affairs, Pfizer, Inc. SHOSHANNA SOFAER, Professor, and Robert P. Luciano Chair of Health Care Policy, Baruch College, The City University of New York MARY E. TINETTI, Department of Internal Medicine/Geriatrics, Yale University School of Medicine KATRIN UHLIG, Division of Nephrology, Tufts Medical Center JAMES WEINSTEIN, Director, The Dartmouth Institute for Health Policy and Clinical Practice; President, The Dartmouth-Hitchcock Clinic; Peggy Y. Thomson Professor in the Evaluative Clinical Sciences Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by ENRIQUETA C. BOND of the Burroughs Wellcome Fund, and MARK R. CULLEN of Stanford University. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.
OCR for page R9
Clinical Practice Guidelines We Can Trust Foreword Many factors enter into healthcare decisions. What alternatives are available? What does the evidence suggest about their potential benefits and harms? How firm is the evidence? Is there reason to adjust expectations based on a particular patient’s age, gender, race, comorbidities, or other attributes? How might different patient preferences affect the best choice for a particular patient? Are there any social, economic, or other practical considerations that could affect the results of a particular care option? Clinical Practice Guidelines (CPGs) are intended to provide a systematic aid to making such complex medical decisions. When rigorously developed using a transparent process that combines scientific evidence, clinician experiential knowledge, and patient values, CPGs have the potential to improve many clinician and patient healthcare decisions, and enhance healthcare quality and outcomes. The present state of CPG development has yet to fully meet this promise. At the request of the U.S. Congress, the Institute of Medicine (IOM) undertook this study to develop a set of standards for developing rigorous, trustworthy clinical practice guidelines. The proposed standards cover a number of elements essential to developing sound practice guidelines, including transparency; conflict of interest; guideline development group composition; CPG–SR intersection; establishing evidence foundations for and strength of recommendations; articulation of recommendations; external review; and updating. This report and the eight proposed stan-
OCR for page R10
Clinical Practice Guidelines We Can Trust dards it contains are intended to reinforce the work of numerous researchers, developers and users of guidelines. This report clarifies where evidence and expert consensus buttress best CPG development practices, and where there is still much to learn. We hope and expect these standards to be pilot-tested, assessed for reliability and validity, evaluated for effectiveness, and to evolve as science and experience dictate. I want to thank the excellent committee who conducted this work, ably led by Sheldon Greenfield, chair, and Earl Steinberg, vice chair. The committee was assisted by dedicated IOM staff led by Robin Graham. A companion report will set out standards for conducting systematic reviews of comparative effectiveness research. I hope that these reports together will advance the state of the art of systematic review and clinical practice guideline development, and contribute to a more transparent, scientifically rigorous, and patient-centered healthcare system in the United States. Harvey V. Fineberg, M.D., Ph.D. President, Institute of Medicine February 2011
OCR for page R11
Clinical Practice Guidelines We Can Trust Preface In the early 1990s, the Institute of Medicine (IOM) issued several reports on clinical practice guidelines (CPGs). In the ensuing years, CPGs and guideline development groups have proliferated enormously to the point that the Agency for Healthcare Research and Quality’s National Guideline Clearinghouse contains nearly 2,700 CPGs. Parallel growth in CPGs has occurred in other countries; the Guidelines International Network’s database currently lists more than 3,700 CPGs. Although the numbers of CPGs and CPG developers have increased substantially, our understanding of the impact of CPGs on clinical practice and patient outcomes is limited. However, research has shown that CPGs have the potential to reduce inappropriate practice variation, enhance translation of research into practice, and improve healthcare quality and safety. CPGs also have had an important influence on development of physician and hospital performance measures. The data gathered from use of such measures have provided consumers with information on the quality of different healthcare providers and, in some instances, provided physicians and hospitals with an economic incentive to improve quality of care. At the same time, there has been considerable concern expressed by physicians, consumer groups, and other stakeholders about the quality of the processes supporting development of CPGs, and the resulting questionable validity of many CPGs and CPG-based clini-
OCR for page R14
Clinical Practice Guidelines We Can Trust This page intentionally left blank.
OCR for page R15
Clinical Practice Guidelines We Can Trust Acknowledgments The committee and staff are indebted to a number of individuals and organizations for their contributions to this report. The following individuals testified before the committee during public meetings or workshops: Workshop Participants William G. Adams, American Academy of Pediatrics David Atkins, Veterans Administration Michael Bettmann, American College of Radiology Zobeida Bonilla, Our Bodies Ourselves Kent Bottles, Institute for Clinical Systems Improvement Cynthia Boyd, Physician Expert in Multimorbidity, Johns Hopkins Department of Medicine Arleen Brown, Physician Expert in Health Disparities, UCLA Internal Medicine Vivian Coates, National Guideline Clearinghouse/ECRI Institute Joyce Dubow, AARP Laura Fochtmann, American Psychiatric Association Ted Ganiats, American Academy of Family Physicians Alice Jacobs, American College of Cardiology and American Heart Association Louis B. Jacques, Centers for Medicare & Medicaid Services Richard Kahn, Former Chief Scientific and Medical Officer of the American Diabetes Association
OCR for page R16
Clinical Practice Guidelines We Can Trust Karen Kelly-Thomas, National Association of Pediatric Nurse Practitioners Marguerite Koster, Kaiser Permanente Southern California Sandra Zelman Lewis, American College of Chest Physicians Joan McClure, National Comprehensive Cancer Network Elizabeth Mort, Quality and Safety, Massachusetts General Hospital, Massachusetts General Physicians Organization Jim Schibanoff, Milliman Care Guidelines Nita L. Siebel, National Cancer Institute Denise Simons-Morton, National Heart, Lung, and Blood Institute Vincenza Snow, American College of Physicians Katrin Uhlig, Tufts Medical Center, Boston Public Forum Testifiers Ethan Basch, American Society of Clinical Oncology Christopher Bever, American Academy of Neurology Terrie Cowley, The TMJ Association Steven Findlay, Consumers Union Merrill Goozner, Health Tech Review/GoozNews.com David Paul Harries, International Spine Intervention Society Belinda Ireland, BJC HealthCare Norman Kahn, Council of Medical Specialty Societies Lisa Mojer-Torres, Chair, Citizens’ Advisory Council for the Division of Addiction Services, State of New Jersey Katherine Nordal, American Psychological Association Jennifer Padberg, Infectious Diseases Society of America (IDSA) William Rich, American Academy of Ophthalmology Richard Rosenfeld, American Academy of Otolaryngology–Head and Neck Surgery Kathleen Sazama, Society for the Advancement of Blood Management (SABM) Aryeh Shander, Society for the Advancement of Blood Management (SABM) Christopher Wolfkiel, American College of Occupational and Environmental Medicine (ACOEM) Diana Zuckerman, National Research Center for Women & Families We also extend special thanks to the following individuals who were essential sources of information, generously giving their time and knowledge to further the committee’s efforts:
OCR for page R17
Clinical Practice Guidelines We Can Trust Josh Adler, Vice President, Business Development, Archimedes, Inc. Ned Calogne, Chief Medical Officer, Colorado Department of Public Health and Environment Paul Chrisp, Associate Director, Accreditation, NHS Evidence, NICE David Eddy, Founder, Archimedes, Inc. Jeremy Grimshaw, Director, Clinical Epidemiology Program, Ottawa Hospital Research Institute Richard Grol, Chair, Department of Quality of Care, Radboud University Nijmegen; Director, Centre for Quality of Care Research (WOK) Russell Harris, Professor of Medicine, Adjunct Professor of Epidemiology, Division of General Medicine, University of North Carolina at Chapel Hill Greg Pawlson, Executive Vice President, NCQA David Ransohoff, Professor of Medicine, Clinical Professor of Epidemiology, Director, Clinical Research Curriculum, University of North Carolina School of Medicine Holger Schünemann, Chair, Department of Clinical Epidemiology & Biostatistics, McMaster University Funding for this study was provided by the Agency for Healthcare Research and Quality (AHRQ). The committee appreciates the opportunity and support extended by AHRQ for the development of this report. Finally, many within the IOM were helpful to the study staff. We especially would like to thank Jillian Laffrey, a new member of our staff, for her great ingenuity and dedication, and the staff of the Systematic Reviews report for their insight and cooperation: Lea Binder, Jill Eden, Mai Le, and Laura Levit. Additionally, the staff would like to thank Clyde Behney, Greta Gorman, Cheryl Levey, William McLeod, Abbey Meltzer, Vilija Teel, and Lauren Tobias.
OCR for page R18
Clinical Practice Guidelines We Can Trust This page intentionally left blank.
OCR for page R19
Clinical Practice Guidelines We Can Trust Contents SUMMARY 1 1 INTRODUCTION 15 Building on Previous IOM Studies, 16 Current State of Clinical Practice Guidelines, 20 Study Scope, 21 Purpose and Updated Definition of Clinical Practice Guidelines, 25 Organization of Report, 27 2 BACKGROUND AND KEY STAKEHOLDERS IN GUIDELINES DEVELOPMENT AND USE 29 Background, 29 Current Major Stakeholders in Guidelines Development and Use, 36 3 TRUSTWORTHY CLINICAL PRACTICE GUIDELINES: CHALLENGES AND POTENTIAL 53 Introduction, 53 Development of Evidence-Based CPGs, 54 Empirical Assessment of Guideline Development Methodology, 63 Standardizing Guideline Development Quality Appraisal, 65
OCR for page R20
Clinical Practice Guidelines We Can Trust 4 CURRENT BEST PRACTICES AND PROPOSED STANDARDS FOR DEVELOPMENT OF TRUSTWORTHY CPGs: PART I, GETTING STARTED 75 Introduction, 75 Establishing Transparency, 76 Management of Conflict of Interest, 78 Guideline Development Group Composition and Group Process, 83 Clinical Practice Guideline—Systematic Review Intersection, 93 Determining Guideline Scope and Requisite Chain of Logic, 98 5 CURRENT BEST PRACTICES AND STANDARDS FOR DEVELOPMENT OF TRUSTWORTHY CPGs: PART II, TRAVERSING THE PROCESS 109 Introduction, 109 Establishing Evidence Foundations for and Rating Strength of Recommendations, 110 Articulation of Recommendations, 128 External Review, 131 Updating, 134 Conclusion, 137 6 PROMOTING ADOPTION OF CLINICAL PRACTICE GUIDELINES 145 Introduction, 146 Strategies for Implementation of CPG Recommendations, 147 Electronic Interventions for CPG Implementation, 161 Decision Analytic Modeling and CPG Implementation, 171 Legal Issues Affecting CPG Implementation, 172 7 DEVELOPMENT, IDENTIFICATION, AND EVALUATION OF TRUSTWORTHY CLINICAL PRACTICE GUIDELINES 187 Introduction, 188 Who Should Develop Guidelines?, 188 Should There Be a Process to Identify CPGs That Meet the Proposed Trustworthy Standards?, 190 Is There a Continuing Need for a Guideline Clearinghouse?, 196 Should There Be a Process to Harmonize Related CPGs?, 198
OCR for page R21
Clinical Practice Guidelines We Can Trust Should There Be a Process to Identify Which Recommendations Should Be Considered for Quality Measures?, 199 How Should CPG Development and Implementation Processes and Impact Be Evaluated?, 201 APPENDIXES A Abbreviations and Acronyms 205 B Workshop Agenda and Questions to Panelists 209 C Clinical Practice Guideline Appraisal Tools 213 D Systems for Rating the Strength of Evidence and Clinical Recommendations 231 E Literature Search Strategy: Clinical Practice Guidelines 255 F Committee Biographies 257
OCR for page R22
Clinical Practice Guidelines We Can Trust This page intentionally left blank.
OCR for page R23
Clinical Practice Guidelines We Can Trust Boxes, Figures, and Tables Chapter 1 Box 1-1 Recommended Attributes of CPGs, 18 Chapter 2 Box 2-1 Criteria for Inclusion in The National Guideline Clearinghouse (NGC), 44 Chapter 3 Boxes 3-1 Infectious Diseases Society of America Lyme Disease Guidelines (2006), 57 3-2 Colorectal Cancer Screening Guidelines (2008), 58 3-3 National Kidney Foundation’s Kidney Disease and Outcomes Quality Initiative Anemia Management Guidelines (2006), 60 Chapter 4 Table 4-1 Models of Interaction Between Clinical Practice Guideline (CPG) Groups and Systematic Review (SR) Teams, 94
OCR for page R24
Clinical Practice Guidelines We Can Trust Figure 4-1 Analytic framework and KQs, 100 Chapter 5 Table 5-1 Interpretation of the American College of Physicians’ Guideline Grading System, 117 Box 5-1 Guidelines with Conflicting Recommendations, 120 Chapter 6 Figure 6-1 Implementation model, 148 Tables 6-1 CDS Types and Features, 164 6-2 CDSC Guidance for CPG Development Activities, 169 Appendix C Table Examples of Tools for Measuring the Quality of Clinical Practice Guidelines Development, 214 Appendix D Table Selected Approaches to Rating Strength of Evidence and Clinical Recommendations, 232