G
Expert Guidance for Chapter 5: Standards for Reporting Systemic Reviews



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 291
G Expert Guidance for Chapter 5: Standards for Reporting Systemic Reviews 291

OCR for page 291
TABLE G-1 Comparison of Chapter 5 Guidance on Conducting Systematic Reviews (SRs) of Comparative 292 Effectiveness Research Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 5.1 Prepare the final report Use a structured format that Quality of Reporting of Meta- Cochrane reviews all have the using a structured format adheres to the Evidence-based analyses (QUORUM)/Preferred same format, which is facilitated Practice Center (EPC) style Reporting Items for Systematic by RevMan. guide. Report must meet Reviews and Meta-Analyses Section 508 requirements for (PRISMA) are useful guides Cochrane has endorsed PRISMA, users with disabilities. for all authors of systematic and it will be incorporated into review reports. (NOTE: The the next version of the Cochrane next edition of the guidance handbook. will recommend adhering to PRISMA.) Commissioning bodies and journals usually have specific requirements regarding presentation and layout of the review. 5.1.1 Include a report title Required. Required. Required. 5.1.2 Include an abstract Required. Include a structured abstract Required. All full reviews must for reviews published as include an abstract of 400 words journal articles. or fewer. The abstract should primarily target healthcare decision makers.

OCR for page 291
5.1.3 Include an executive Required. An executive Include an executive summary Not mentioned. summary summary is published for reviews published as full- separately as well as with the length reports. full-length report. 5.1.4 Include a summary Required. Developed by the Not mentioned. Required. Plain-language written for the lay public Eisenberg Center. summaries provide findings in a straightforward style that can be understood by consumers. 5.1.5 Include an introduction Required in both full report as Required. Include a Required. (rationale and objectives) well as part of the executive background/introduction. summary. 5.1.6 Include a methods section. Describe the following: •  Research protocol Same elements in protocol are Description of the protocol is Review authors are encouraged to required in methods section. not mentioned, but the next cite their protocol. edition will recommend that reports indicate that a protocol was written and followed, and should report the protocol registration number. •    ligibility criteria  E Required in protocol and Required. Required. (criteria for including methods section. and excluding studies in the systematic review) 293 continued

OCR for page 291
TABLE G-1 Continued 294 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration •    nalytic framework and  A Required in protocol and Not mentioned. Not mentioned. key questions methods section of the full report. •    atabases and other  D Required in protocol and Required. The write-up of Required. information sources methods section. the search should include used to identify relevant information about the studies databases and interfaces searched. •    earch strategy S Required. Include a description Required. The search process Required. List all databases of the search methods. Full- should be documented in full, searched. Note the dates of the search strategy required in including information about last search for each database and appendix. This description the databases and interfaces the period searched. Note any should be detailed enough to searched (including the language or publication status allow replication of search. dates covered), full detailed restriction. List grey-literature search strategies (including sources. List individuals or any justifications for date or organizations contacted. List language restrictions), and the any journals and conference number of records retrieved proceedings specifically or details provided on where handsearched. List any other the strategy can be obtained. sources searched. An appendix documenting the search process should be included.

OCR for page 291
Study selection process Required in protocol and Required. State the method used to apply the methods section. selection criteria. •    ata extraction process D Required in protocol and Required. Describe the methods for data methods section. collection. •    ethods for handling  M Required in protocol and Required. Describe the strategies for dealing missing information methods. with missing data. •    nformation to be  I Required in protocol and Required. Not mentioned. extracted from included methods section. studies •    ethods to appraise the  M Required. Protocol and Required. Describe the methods used to quality of individual methods section should assess risk of bias. studies describe methods to assess risk of bias. •    ummary measures (e.g.,  S Required in protocol and Required. The effect measures of choice risk ratio, difference in methods section. should be stated. means) •    ationale for pooling (or  R Required in protocol and Not mentioned. Approach to determining whether not pooling) of included methods section. a meta-analysis is considered studies appropriate should be included. •    ethods of synthesizing  M Required in protocol and Required. The choice of meta-analysis the evidence (qualitative methods section. Describing method should be stated, including and meta-analysis) methods for grading of whether a fixed-effect or a random- strength of evidence in effects model is used. Approaches general and of each domain is to addressing clinical heterogeneity recommended. should be described. Method for identifying statistical heterogeneity should be stated (e.g., visually, using I2, using a chi-squared test). 295 continued

OCR for page 291
TABLE G-1 Continued 296 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration •    dditional analyses, if  A Required in protocol and Any secondary analyses All planned subgroup analyses done, indicating which methods section. (sensitivity analyses, etc.). should be listed (or independent were prespecified variables for meta-regression). Any other methods for investigating heterogeneity of effects should be described. 5.1.7 Include a results Organize presentation of The results of all analyses The results section should directly section. Organize the results in logical format. This should be considered as a address the objectives of the presentation of results is typically done around key whole, and overall coherence review. around key questions. questions. discussed. Describe the following (repeat for each key question): •    tudy selection process S Required. Flow chart is Required. Describe the details The results sections should start required documenting excluded of included and excluded with a summary of the results studies. studies. of the search (e.g., how many references were retrieved by the electronic searches, and how many were considered as potentially eligible after screening?). It is essential that the number of included studies is clearly stated.

OCR for page 291
•    ist of excluded studies  L Excluded studies are included Required. Studies that may appear to meet and reasons for their in references/appendix. the eligibility criteria, but which exclusion were excluded, should be listed and the reason for exclusion should be given. •    ppraisal of individual  A Required. Required. Required. A risk of bias table is studies’ quality strongly recommended. •    ualitative synthesis Q Required. Describe the findings Required. Both quantitative Not required. However, the of the review. and narrative synthesis should final report should include begin by constructing a clear a characteristics of included The study characteristics of descriptive summary of the studies table and Grading of eligible studies are usually included studies. An indication Recommendations Assessment, included in both a text of study quality or risk of bias Development and Evaluation summary and a summary table, may also be given. (GRADE) evidence tables. It should and sometimes in an evidence also summarize the general risk map as well. Where possible, results of of bias in results of the included individual studies should be studies, its variability across Highlight where evidence presented graphically, most studies, and any important flaws indicates that benefits, harms, commonly using a forest in individual studies. and trade-offs are different for plot that illustrates the effect distinct patient groups. estimates from individual studies. The justification for grade and domains are required, usually Consider how the relative provided in a grading table, effects may translate into sometimes in appendix. different absolute effects for people with differing underlying prognoses. 297 continued

OCR for page 291
TABLE G-1 Continued Agency for Healthcare 298 Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration •    eta-analysis of results,  M Required, if appropriate. Required. Consistency across Required. A summary of findings if performed (explain Describe the findings of the studies should be considered. table may be included to present rationale for doing one) review. the main findings of a review in a tabular format. •    dditional analyses, if  A Required as appropriate. Required. Include any Not mentioned. done, indicating which secondary analyses. were prespecified •    ables and figures T Include tables summarizing Where possible, results should Required. Tables that may the studies and quantitative be shown graphically. The most be included in a review: a syntheses. commonly used graphic is the characteristics of included studies forest plot. table, a risk of bias table, a characteristics of excluded studies Synthesis should usually table, a characteristics of studies include tabulated details about awaiting classification table, a study type, interventions, characteristics of ongoing studies number of participants, a table, and a summary of findings summary of participant table. Figures that may be included characteristics, outcomes, and in a review: forest plot, funnel outcome measures. plot, risk of bias graph, risk of bias summary, and other figures. 5.1.8 Include a discussion section. Include the following: •    ummary of the evidence S Required, though usually in the Suggests a statement of Required. conclusions section. principal findings.

OCR for page 291
•    trengths and limitations  S Recommended. Describe Required. Describe the Describe the quality of evidence, of the systematic review strengths and weaknesses strengths and weaknesses potential biases in the review of systematic review and of of the review. Appraise the process, and agreements/ studies. methodological quality of the disagreements with other studies review, and the relation to or reviews. other reviews. •    onclusions for each key  C Required. Present the benefits Should include practical Review authors should not make question and harms in a manner that implications for clinicians and recommendations for clinical helps decision makers. Express policy makers. practice. May highlight different benefits in absolute terms, actions that might be consistent rather than relative terms. with particular patterns of values and preferences. •    aps in evidence G Description of gaps in evidence Gaps in evidence should be Describe the completeness and is required as a separate section highlighted. applicability of evidence to the (does not necessarily need to review question. be in discussion section). •    uture research needs F Required. Some reports will The report should describe Describe the implications for also require this as a more fully any unanswered questions research. prioritized and fleshed-out and implications for further separate paper. research. 5.1.9 Include a section This is done automatically in Required. Required. describing funding sources the editing process. and COI 5.2 Peer review the draft Identify peer reviewers The advisory group The editorial team of the Cochrane report to ensure independent, should review the draft Review Group is ultimately unconflicted input from report for scientific quality responsible for the decision to persons with particular clinical, and completeness. The publish a Cochrane review on methodological, and statistical commissioning body may also its module. The decision is made expertise and submit a draft organize an independent peer after peer review and appropriate 299 report to these individuals. review of the draft report. revisions by the review authors. continued

OCR for page 291
TABLE G-1 Continued 300 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 5.2.1 Use a third party to Use an editorial review process Not mentioned. Peer review process is explicitly manage the peer review that provides for independent managed by the Cochrane Review process judgment of the adequacy of an Group. EPC’s response to public and peer review comments. 5.2.2 Provide a public Must post a draft report. Public comment period not Indefinite comment period: A comment period for the mentioned. formal feedback mechanism is in report and publicly report Public report on disposition place. on disposition of comments is posted 3 months after final A record of the comments and report posted. the way in which they were The review authors are required to dealt with should be kept with respond to feedback on a review the archive of the review. (usually within one month of receiving the feedback). 5.3 Publish the final report Systematic reviews are posted The review findings need to Reviews are published in the in a manner that ensures Cochrane Database of Systematic on the relevant AHRQ website. be effectively communicated free public access to practitioners and policy Reviews. makers. NOTE: Some information on AHRQ-, CRD-, and Cochrane-recommended methods was provided via personal communication with Stephanie Chang, EPC Program Task Order Officer, AHRQ (October 5, 2010); Lesley Stewart, Director, CRD (October 14, 2010); and Julian Higgins, Senior Statistician, MRC Biostatistics Unit, Institute of Public Health, University of Cambridge (October 4, 2010).

OCR for page 291
301 APPENDIX G REFERENCES Atkins, D., S. Chang, G. Gartlehner, D. I. Buckley, E. P. Whitlock, E. Berliner, and D. Matchar. 2010. Assessing the applicability of studies when comparing medical interventions. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effectivehealthcare. ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?productid=603& pageaction=displayproduct (accessed January 19, 2011). Chou, R., N. Aronson, D. Atkins, A. S. Ismaila, P. Santaguida, D. H. Smith, E. Whitlock, T. J. Wilt, and D. Moher. 2010. AHRQ series paper 4: Assessing harms when com- paring medical interventions: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):502–512. CRD (Centre for Reviews and Dissemination). 2009. Systematic reviews: CRD’s guidance for undertaking reviews in health care. York, UK: York Publishing Services, Ltd. Fu, R., G. Gartlehner, M. Grant, T. Shamliyan, A. Sedrakyan, T. J. Wilt, L. Griffith, M. Oremus, P. Raina, A. Ismaila, P. Santaguida, J. Lau, and T. A. Trikalinos. 2010. Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. In Methods guide for compara- tive effectiveness reviews, edited by Agency for Healthcare Research and Qual- ity. http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides- reviews-and-reports/?pageaction=displayProduct&productID=554 (accessed January 19, 2011). Helfand, M., and H. Balshem. 2010. AHRQ series paper 2: Principles for develop - ing guidance: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):484–490. Higgins, J. P. T., and S. Green, eds. 2008. Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons Norris, S., D. Atkins, W. Bruening, S. Fox, E. Johnson, R. Kane, S. C. Morton, M. Oremus, M. Ospina, G. Randhawa, K. Schoelles, P. Shekelle, and M. Viswanathan. 2010. Selecting observational studies for comparing medical interventions. In Methods guide for comparative effectiveness reviews, edited by Agency for Health- care Research and Quality. http://www.effectivehealthcare.ahrq.gov/index. cfm/search-for-guides-reviews-and-reports/?pageaction=displayProduct& productID=454 (accessed January 19, 2011). Owens, D. K., K. N. Lohr, D. Atkins, J. R. Treadwell, J. T. Reston, E. B. Bass, S. Chang, and M. Helfand. 2010. AHRQ series paper 5: Grading the strength of a body of evidence when comparing medical interventions: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):513–523. Relevo, R., and H. Balshem. 2011. Finding evidence for comparing medical interven - tions. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/ index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayProduct &productID=605 (accessed January 19, 2011). Slutsky, J., D. Atkins, S. Chang, and B. A. Collins Sharp. 2010. AHRQ series paper 1: Comparing medical interventions: AHRQ and the Effective Health Care Pro - gram. Journal of Clinical Epidemiology 63(5):481–483.

OCR for page 291
302 FINDING WHAT WORKS IN HEALTH CARE White, C. M., S. Ip, M. McPheeters, T. S. Carey, R. Chou, K. N. Lohr, K. Robinson, K. McDonald, and E. Whitlock. 2009. Using existing systematic reviews to replace de novo processes in CERs. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effective- healthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?page action=displayProduct&productID=329 (accessed January 19, 2011). Whitlock, E. P., S. A. Lopez, S. Chang, M. Helfand, M. Eder, and N. Floyd. 2010. AHRQ series paper 3: Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):491–501.