FINDING WHAT WORKS IN HEALTH CARE

STANDARDS FOR SYSTEMATIC REVIEWS

Committee on Standards for Systematic Reviews of Comparative Effectiveness Research

Board on Health Care Services

Jill Eden, Laura Levit, Alfred Berg, and Sally Morton, Editors

INSTITUTE OF MEDICINE
OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Committee on Standards for Systematic Reviews of Comparative Effectiveness Research Board on Health Care Services Jill Eden, Laura Levit, Alfred Berg, and Sally Morton, Editors THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract No. HHSP23320042509X1 between the National Academy of Sciences and the Department of Health and Human Services. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that pro - vided support for this project. Library of Congress Cataloging-in-Publication Data Institute of Medicine (U.S.). Committee on Standards for Systematic Reviews of Comparative Effectiveness Research. Finding what works in health care : standards for systematic reviews / Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Board on Health Care Services, Institute of Medicine of the National Academies ; Jill Eden ... [et al.], editors. p. ; cm. Includes bibliographical references. ISBN 978-0-309-16425-2 (pbk.) — ISBN 978-0-309-16426-9 (pdf) 1. Medical care— Standards—United States. 2. Medical care—United States—Quality control. I. Eden, Jill. II. Title. [DNLM: 1. Comparative Effectiveness Research—standards—United States. 2. Outcome and Process Assessment (Health Care)—United States. W 84.3] RA399.A3I565 2011 610.28’9—dc23 2011017455 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. For more information about the Institute of Medicine, visit the IOM home page at: www. iom.edu. Copyright 2011 by the National Academy of Sciences. All rights reserved. Printed in the United States of America The serpent has been a symbol of long life, healing, and knowledge among almost all cultures and religions since the beginning of recorded history. The serpent adopted as a logotype by the Institute of Medicine is a relief carving from ancient Greece, now held by the Staatliche Museen in Berlin. Suggested citation: IOM (Institute of Medicine). 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press.

OCR for page R1
“Knowing is not enough; we must apply. Willing is not enough; we must do.” — Goethe Advising the Nation. Improving Health.

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal govern - ment on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its mem - bers, sharing with the National Academy of Sciences the responsibility for advis - ing the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in pro - viding services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1
COMMITTEE ON STANDARDS FOR SYSTEMATIC REVIEWS OF COMPARATIVE EFFECTIVENESS RESEARCH ALFRED O. BERG (Chair), Professor, Department of Family Medicine, University of Washington School of Medicine, Seattle, WA SALLY C. MORTON (Vice Chair), Professor and Chair, Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh JESSE A. BERLIN, Vice President, Epidemiology, Johnson & Johnson Pharmaceutical Research and Development, Titusville, NJ MOHIT BHANDARI, Canada Research Chair in Musculoskeletal Trauma, McMaster University, Orthopaedic Research Unity, Clarity Research Group, at the Hamilton Health Sciences–General Site (joined October 2009; resigned July 2010) GISELLE CORBIE-SMITH, Professor, Social Medicine, Medicine and Epidemiology, Departments of Social Medicine, Medicine and Epidemiology, University of North Carolina–Chapel Hill, Chapel Hill, NC KAY DICKERSIN, Professor of Epidemiology, Director, Center for Clinical Trials and U.S. Cochrane Center, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD JEREMY M. GRIMSHAW, Director, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada MARK HELFAND, Director, Oregon Evidence-Based Practice Center, Professor of Medicine and Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, OR VINCENT E. KERR, President, Care Solutions, UnitedHealthcare, Fairfield, CT MARGUERITE A. KOSTER, Practice Leader, Technology Assessment and Guidelines Unit, Kaiser Permanente Southern California, Pasadena, CA KATIE MASLOW, Scholar-in-Residence, Institute of Medicine, Washington, DC DAVID A. MRAZEK, Chair, Department of Psychiatry and Psychology, Professor of Pediatrics and Psychiatry, College of Medicine, Mayo Clinic, Rochester, MN CHRISTOPHER H. SCHMID, Director, Biostatistics Research Center; Professor of Medicine, Tufts Medical Center and Tufts University, Boston, MA ANNA MARIA SIEGA-RIZ, Professor of Epidemiology and Nutrition, Associate Chair of Epidemiology, University of North Carolina Gillings School of Global Public Health, Chapel Hill, NC v

OCR for page R1
HAROLD C. SOX, Editor Emeritus, Annals of Internal Medicine, American College of Physicians of Internal Medicine, Hanover, NH PAUL WALLACE, Medical Director, The Permanente Federation, Kaiser Permanente, Oakland, CA Study Staff JILL EDEN, Study Director LAURA LEVIT, Program Officer LEA BINDER, Research Associate (through May 2010) MAI LE, Research Assistant (starting September 2010) ALLISON McFALL, Senior Program Assistant (through August 2010) JILLIAN LAFFREY, Senior Program Assistant (starting July 2010) ROGER HERDMAN, Director, Board on Health Care Services Consultants JULIA KREIS, Harkness/Bosch Fellow in Health Care Policy and Practice, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD DAVID MOHER, Senior Scientist, Ottawa Hospital Research Institute (OHRI), Ottawa, Ontario, Canada ALEXANDER TSERTSVADZE, Ottawa Hospital Research Institute (OHRI), Ottawa, Ontario, Canada SALLY HOPEWELL, National Institute for Health Research, U.K. Cochrane Centre, Oxford, United Kingdom BRYAN LUCE, Senior Vice President, Science Policy, United BioSource Corporation, Chevy Chase, MD RACHAEL FLEURENCE, Senior Research Scientist, United BioSource Corporation, Chevy Chase, MD vi

OCR for page R1
Reviewers This report has been reviewed in draft form by individuals cho- sen for their diverse perspectives and technical expertise, in accor- dance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the insti- tution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectiv- ity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: NAOMI ARONSON, Executive Director, Blue Cross and Blue Shield Association, Technology Evaluation Center VIVIAN COATES, Vice President, Information Services and Technology Assessment, ECRI Institute JOHN P. A. IOANNIDIS, C.F. Rehnborg Professor in Disease Prevention, Professor of Medicine, and Director, Stanford Prevention Research Center, Stanford University ERIC B. LARSON, Executive Director, Group Health Research Institute JOSEPH LAU, Director, Tufts Evidence-based Practice Center, Tufts Medical Center vii

OCR for page R1
viii REVIEWERS ART LEVIN, Director, Center for Medical Consumers ALVIN I. MUSHLIN, Professor and Chairman, Department of Public Health, Weill Medical College, Cornell University PAUL SHEKELLE, Director, Southern California Evidence- Based Practice Center, The RAND Corporation RICHARD N. SHIFFMAN, Associate Director, Center for Medical Informatics, Yale School of Medicine LESLEY STEWART, Director, Centre for Reviews and Dissemination, University of York Although the reviewers listed above have provided many con- structive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was over- seen by ENRIQUETA C. BOND of the Burroughs Wellcome Fund, and MARK R. CULLEN of Stanford University. Appointed by the National Research Council and the Institute of Medicine, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional pro - cedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

OCR for page R1
Foreword Knowing what works in health care is of highest importance for patients, healthcare providers, and other decision makers. The most reliable way to identify benefits and harms associated with various treatment options is a systematic review of comparative effectiveness research. Increasingly recognized for their importance, systematic reviews are now being sponsored and conducted by a number of organizations across the United States. When conducted well, a systematic review identifies, appraises, and synthesizes the available body of evidence for a specific clinical question. However, not all of these reviews meet the appropriate standards of quality and methodology. At the request of the U.S. Congress, the Institute of Medicine (IOM) undertook this study to develop a set of stan - dards for conducting systematic reviews of comparative effective- ness research. The report will have direct implications for implementation of the Patient Protection and Affordable Care Act of 2010. This law estab- lished the first nonprofit, public–private Patient-Centered Outcomes Research Institute (PCORI). PCORI will be responsible for setting methodological standards for clinical effectiveness research, includ - ing systematic reviews of research findings. I hope this study will support PCORI’s development of standards to ensure that system- atic reviews meet a minimum level of objectivity, transparency, and scientific rigor. The IOM study should also help to inform other ix

OCR for page R1
x FOREWORD public sponsors of systematic reviews of comparative effectiveness research. To conduct this study, the Institute of Medicine convened a highly qualified committee with diverse backgrounds, ably led by Alfred Berg, chair, and Sally Morton, vice chair. The committee was assisted by dedicated IOM staff led by Jill Eden. This report draws on available evidence, review of expert guidance, and careful con- sideration of alternative standards according to specified criteria. While this report presents an initial list of standards for improving the quality of publicly funded systematic reviews, it also calls for continued investment in methodological research to identify bet- ter practices for future reviews. A companion report establishes standards for developing clinical practice guidelines. I hope these documents will help guide a robust systematic review enterprise for health in the United States. Harvey V. Fineberg, M.D., Ph.D. President, Institute of Medicine February 2011

OCR for page R1
Preface Page through any volume of a medical journal from the 1970s and read a clinical review. The authors are likely to be recognized as experts in the field, and the introduction will often open with “we reviewed the world’s medical literature,” moving on to reach clinical conclusions based as much on the experience and opinions of the authors as on the published evidence. Systematic literature searches, critical appraisal, quantitative meta-analysis, and documented path- ways linking the evidence to reaching clinical conclusions were virtually unknown. Today’s explicit, scientifically rigorous, transpar- ent, and publicly accountable systematic reviews (SRs) and clinical practice guidelines (CPGs) are the barely recognizable heirs to that earlier convention for giving clinical advice. Enormous progress has been made by a large and growing international community of clinicians, methodologists, statisticians, and other stakeholders in developing SRs and CPGs, yet problems remain. There are many competing systems for evaluating and syn- thesizing evidence, and there are no internationally agreed-upon standards for how to conduct an SR or create a CPG. In the United States, the decades-old interest in SRs and CPGs among public and private agencies is receiving a boost from the highlighting of the importance of both in debates about healthcare reform; a specific provision in the Medicare Improvements for Patients and Providers Act of 2008 brought two Institute of Medicine (IOM) committees into being, aimed at setting standards for SRs and CPGs. Furthermore, in the United States there is enormous interest in and high expecta- xi

OCR for page R1
xii PREFACE tions for the newly created Patient-Centered Outcomes Research Institute, whose authorizing legislation specifically names SRs and CPGs as important components in developing a national program of comparative effectiveness research. As both SR and CPG reports indicate, the term “standard” is problematic. Our two committees found a sparse evidence base that directly evaluates alternative approaches to SRs and CPGs. The SR committee thus relied on available literature, expert guidance from organizations engaged in SRs, and its own criteria and internal discussions to propose a set of standards, recognizing that any such recommendations must be considered provisional pending further development of the evidence. Collectively the standards set a high bar that will be difficult to achieve for many SRs, yet the evidence and experience are not reassuring that it is safe to cut corners if resources are limited. The standards will be especially valuable for SRs of high-stakes clinical questions with broad population impact, where the use of public funds to get the right answer justifies care- ful attention to the rigor with which the SR is conducted. The best practices collected in this report should be thoughtfully considered by anyone conducting an SR. In the end the most important stan- dard is to be transparent in reporting what was done and why. Importantly, the committee concludes with recommendations that the United States invest in a program to improve both the science of SRs (with attention to both scientific rigor and feasibility/cost) and the environment that supports them, including a process to update standards as the evidence improves. Finally, one of the most professionally satisfying benefits of lead- ing an IOM committee is the opportunity to work with commit - tee members with an amazing breadth and depth of experience, and IOM staff whose anticipation and completion of the next steps always appears effortless. We are deeply grateful that this committee and staff have again demonstrated the process at its best. In conclusion, the committee believes we are at an important juncture in the development of SRs and CPGs, and that timely invest- ment in both will produce an excellent return in improving health care and patient outcomes. We hope our recommended standards will serve as a useful milestone as the United States joins interna - tional partners to advance the science and improve the environment for SRs and CPGs. Alfred O. Berg, Chair Sally C. Morton, Vice Chair Committee on Standards for Systematic Reviews of Comparative Effectiveness Research

OCR for page R1
Acknowledgments The committee and staff are indebted to a number of individuals and organizations for their contributions to this report. The follow - ing individuals provided testimony to the committee: Naomi Aronson, Executive Director, Technology Evaluation Center, Blue Cross and Blue Shield Association Kalipso Chalkidou, Director, NICE International, National Institute for Health and Clinical Excellence Yngve Falck-Ytter, Assistant Professor of Medicine, Case Western Reserve University; Director of Hepatology, V Medical Center, A Cleveland Rebekah Gee, Assistant Professor of Clinical Medicine, Department of Obstetrics and Gynecology, Tulane University, American College of Obstetricians and Gynecologists Louis B. Jacques, Director, Coverage & Analysis Group, Office of Clinical Standards & Quality, Centers for Medicare & Medicaid Services Sandra Zelman Lewis, Assistant Vice President, Health & Science Policy, American College of Chest Physicians Virginia Moyer, Section Head, Academic General Pediatrics, Baylor College of Medicine, American Academy of Pediatrics Edmund Pezalla, National Medical Director and Chief Clinical Officer, Aetna Pharmacy Management xiii

OCR for page R1
xiv ACKNOWLEDGMENTS Alan Rosenberg, Vice President of Technology Assessment, Wellpoint Health Networks Carol Sakala, Director of Programs, Childbirth Connection Gail Shearer, Former Director, Consumer Reports Best Buy Drugs, and Former Director, Health Policy Analysis, Consumers Union David Shern, President and CEO, Mental Health America David B. Wilson, Crime and Justice Group Cochair, Steering Committee, The Campbell Collaboration We also extend special thanks to the following individuals who were essential sources of information, generously giving their time and knowledge to further the committee’s efforts: Stephanie Chang, EPC Program Director, AHRQ Julian Higgins, Senior Statistician, MRC Biostatistics Unit, Institute of Public Health, University of Cambridge Alison Little, Medical Director, Drug Effectiveness Review Project Rose Relevo, Research Librarian, AHRQ Scientific Resource Center Karen M. Schoelles, Director, ECRI Institute Evidence-based Practice Center Lesley Stewart, Director, Centre for Reviews and Dissemination Funding for this study was provided by the Agency for Health- care Research and Quality (AHRQ). The committee appreciates the opportunity and support extended by AHRQ for the development of this report. Finally, many within the Institute of Medicine were helpful to the study staff. We especially would like to thank the staff of the Clinical Practice Guidelines We Can Trust report for their collabora- tive spirit: Robin Graham, Michelle Mancher, and Dianne Wolman. Additionally, the staff would like to thank Clyde Behney, Michelle O. Crosby-Nagy, Greta Gorman, Cheryl Levey, William McLeod, Abbey Meltzer, Vilija Teel, and Lauren Tobias

OCR for page R1
Contents SUMMARY 1 1 INTRODUCTION 17 Committee Charge, 19 What Is Comparative Effectiveness Research?, 19 Study Scope, 23 Conceptual Framework, 26 Methods of the Study, 28 Current Landscape, 34 Organization of the Report, 35 2 STANDARDS FOR INITIATING A SYSTEMATIC 45 REVIEW Establishing the Review Team, 47 Recommended Standards for Organizing the Review Team, 53 Ensuring User and Stakeholder Input, 55 Recommended Standards for Ensuring User and Stakeholder Input, 65 Formulating the Topic, 67 Recommended Standards for Formulating the Topic, 71 Developing the Systematic Review Protocol, 72 Recommended Standards for Developing the Systematic Review Protocol, 74 xv

OCR for page R1
xvi CONTENTS 3 STANDARDS FOR FINDING AND ASSESSING 81 INDIVIDUAL STUDIES The Search Process, 83 Recommended Standards for the Search Process, 106 Screening and Selecting Studies, 108 Recommended Standards for Screening and Selecting Studies, 114 Managing Data Collection, 117 Recommended Standard for Extracting Data, 122 Critical Appraisal of Individual Studies, 123 Recommended Standards for Assessing the Quality and Relevance of Individual Studies, 137 4 STANDARDS FOR SYNTHESIZING THE BODY 155 OF EVIDENCE A Note on Terminology, 157 Recommended Standard for Assessing and Describing the Quality of a Body of Evidence, 172 Qualitative Synthesis of the Body of Evidence, 173 Recommended Standards for Qualitative Synthesis, 176 Meta-Analysis, 177 Recommended Standards for Meta-Analysis, 187 5 STANDARDS FOR REPORTING SYSTEMATIC 195 REVIEWS Reporting Guidelines, 198 Systematic Reviews Published in Journals, 202 Recommended Standard for Preparing the Final Report, 203 Recommended Standard for Report Review, 214 Recommended Standard for Publishing the Final Report, 217 6 IMPROVING THE QUALITY OF SYSTEMATIC 223 REVIEWS: DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS Improving the Science of Systematic Reviews, 225 Improving the Environment for Supporting Systematic Reviews, 229 Conclusions and Recommendations, 232

OCR for page R1
xvii CONTENTS APPENDIXES A Abbreviations and Acronyms 235 B Glossary 239 C Workshop Agenda and Questions to Panelists 243 D Expert Guidance for Chapter 2: Standards for Initiating a Systematic Review 249 E Expert Guidance for Chapter 3: Standards for Finding and Assessing Individual Studies 265 F Expert Guidance for Chapter 4: Standards for Synthesizing the Body of Evidence 281 G Expert Guidance for Chapter 5: Standards for Reporting Systematic Reviews 291 H Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Checklist 303 I Committee Biographies 307

OCR for page R1

OCR for page R1
Boxes, Figures, and Tables Summary Boxes S-1 Charge to the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, 2 S-2 Recommended Standards for Initiating a Systematic Review, 6 S-3 Recommended Standards for Finding and Assessing Individual Studies, 8 S-4 Recommended Standards for Synthesizing the Body of Evidence, 10 S-5 Recommended Standards for Reporting Systematic Reviews, 14 Chapter 1 Boxes 1-1 Charge to the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, 20 1-2 Types of Comparative Effectiveness Research Studies, 21 1-3 Four Examples of the Use of Observational Studies in Systematic Reviews of Comparative Effectiveness Research, 22 1-4 Charge to the Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, 27 1-5 Numbering System for the Committee’s Recommended Systematic Review Standards, 33 xix

OCR for page R1
xx BOXES, FIGURES, AND TABLES Tables 1-1 Statutory Requirements for the Patient-Centered Outcomes Research Institute, 25 1-2 Committee Criteria for Assessing Potential Standards and Elements for Systematic Reviews, 30 1-3 Examples of U.S. Governmental Organizations That Produce Systematic Reviews, 35 1-4 Examples of Private U.S. Organizations That Produce Systematic Reviews, 36 1-5 Examples of International Organizations That Produce Systematic Reviews, 37 1-6 Examples of Organizations That Use Systematic Reviews, 38 Chapter 2 Boxes 2-1 Recommended Standards for Initiating a Systematic Review, 48 2-2 International Committee of Medical Journal Editors Types of Conflict-of-Interest Disclosures, 52 2-3 Organizations and Groups Included in the Commissioned Paper on Consumer Involvement in Systematic Reviews, 57 2-4 Examples of Interaction Between Systematic Review (SR) and Clinical Practice Guideline (CPG) Teams, 62 Figure 2-1 Analytic framework for a new enteral supplement to heal bedsores, 70 Tables 2-1 Models of Interaction Between Systematic Review (SR) and Clinical Practice Guideline (CPG) Teams, 63 2-2 PICO Format for Formulating an Evidence Question, 68 Chapter 3 Boxes 3-1 Recommended Standards for Finding and Assessing Individual Studies, 84 3-2 Bibliographic Databases, 96 3-3 Subject-Specific Databases, 98

OCR for page R1
xxi BOXES, FIGURES, AND TABLES 3-4 Grey-Literature Databases, 99 3-5 Study Selection Criteria for a Systematic Review of Therapies for Children with Autism Spectrum Disorders (ASD), 111 3-6 Types of Data Extracted from Individual Studies, 118 3-7 Cochrane Back Pain Group Criteria for Internal Validity of Randomized Trials of Back Pain, 134 3-8 Cochrane Risk of Bias Tool Domains, 135 3-9 USPSTF Criteria for Grading the Internal Validity of Individual Studies (RCTs and Cohort Studies), 137 Figure 3-1 Example of a flow chart, 115 Tables 3-1 Types of Reporting Biases, 88 3-2 Expert Suggestions for Conducting the Search Process and Addressing Reporting Bias, 95 3-3 Expert Suggestions for Screening Publications and Extracting Data from Eligible Studies, 109 3-4 Types of Bias in Individual Studies, 124 3-5 Characteristics of Individual Studies That May Affect Applicability, 127 Chapter 4 Boxes 4-1 Recommended Standards for Synthesizing the Body of Evidence, 158 4-2 Key Concepts Used in the GRADE Approach to Assessing the Quality of a Body of Evidence, 165 4-3 Key Concepts Used in the AHRQ Approach to Assessing the Quality of a Body of Evidence, 166 4-4 Research Trends in Meta-Analysis, 186 Figure 4-1 Forest plot, 183 Tables 4-1 Examples of Approaches to Assessing the Body of Evidence for Therapeutic Interventions, 161 4-2 Key Purposes of the Qualitative Synthesis, 174

OCR for page R1
xxii BOXES, FIGURES, AND TABLES Chapter 5 Boxes 5-1 Recommended Standards for Reporting Systematic Reviews, 197 5-2 Requirements for Research Funded by the Patient-Centered Outcomes Research Institute, 199 5-3 A History of Reporting Guidelines for Comparative Effectiveness Research, 201 5-4 Example of a Structured Abstract: Clinical Utility of Cancer Family History Collection in Primary Care, 206 5-5 Example of a Plain-Language Summary: Antenatal Corticosteroids for Accelerating Fetal Lung Maturation for Women at Risk of Preterm Birth, 208 5-6 Reporting Funding and Conflict of Interest: Selected Examples, 215 Tables 5-1 Topics to Include in the Methods Section, 210 5-2 Topics to Include in the Results Section, 212 5-3 EPICOT Format for Formulating Future Research Recommendations, 214