Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 249
D Expert Guidance for Chapter 2: Standards for Initiating a Systematic Review 249

OCR for page 249
TABLE D-1 Comparison of Chapter 2 Guidance on Conducting Systematic Reviews (SRs) of Comparative 250 Effectiveness Research Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 2.1 Establish a team with Provides guidance on Provides guidance on Provides guidance on establishing a appropriate expertise and establishing a review team (see establishing a review team (see review team (see below). experience to conduct the below). below). systematic review 2.1.1 Include expertise in Must include an individual Ideally includes an individual Must include expertise in the topic pertinent clinical content with relevant clinical expertise, with knowledge of the relevant area being reviewed. areas and when indicated, access clinical/topic area. to specialists with relevant expertise. 2.1.2 Include expertise in Must include an individual Ideally includes an individual Must include, or have access to, systematic review methods with expertise in conducting with expertise in SR methods, expertise in SR methodology. SRs. and/or qualitative research methods where appropriate. 2.1.3 Include expertise Must include an individual Ideally includes an individual Either a Trials Search Coordinator in searching for relevant with library expertise. with information retrieval does the search, or a Trials Search evidence skills. Coordinator or librarian should be consulted. 2.1.4 Include expertise in Must include an individual Ideally includes an individual Must include, or have access to, quantitative methods with statistical expertise. with expertise in statistics and statistical expertise. health economics. 2.1.5 Include other expertise Not mentioned. Includes a range of skills. Review authors are encouraged to as appropriate seek and incorporate the views of users and stakeholders.

OCR for page 249
2.2 Manage bias and Provides guidance on Provides guidance on Provides guidance on managing conflict of interest (COI) managing bias and COI in the managing bias and COI in the bias and COI in the review team of the team conducting the review team (see below). review team (see below). (see below). systematic review 2.2.1 Require each team Must disclose relevant COI should be noted early in Financial COI should be avoided, member to disclose financial, business, and the process and steps taken to but must be disclosed if there is potential COI and professional interests. ensure that these do not impact any. Also, any secondary interests professional or intellectual the review. that might unduly influence bias judgments made in a review should be disclosed. All authors must sign declarations of interest. 2.2.2 Exclude individuals Evidence-based Practice Recommends that all biases are Reviews should be free of real or with a clear financial Center (EPC) core team and declared. Does not specifically perceived bias or COI. Reviews conflict any authors on the reports address the issue of whether cannot be sponsored by commercial are barred from having any it is appropriate to include sources. significant competing interests. individuals with vested financial interests in the review team. 2.2.3 Exclude individuals EPC core team and any authors Any COI, including Personal conflicts must be whose professional or on the reports are barred professional or intellectual bias, disclosed. intellectual bias would from having any significant should be declared. diminish the credibility of competing interests. the review in the eyes of the intended users continued 251

OCR for page 249
TABLE D-1 Continued 252 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 2.3 Ensure user and Engage a range of stakeholders There may be a number of It may be useful to form an stakeholder input as the across various sectors in the individuals or groups who advisory group of people, review is designed and United States. Categories are consulted at various including representation of relevant conducted include clinicians; consumer/ stages, including healthcare stakeholders, to ensure that authors patients (including professionals, patient address the questions of importance representative organizations); representatives, service to stakeholders. employers and business users, and experts in research groups; federal and state methods. Some funding bodies partners; healthcare industry may require the establishment representatives; payers; health of an advisory group who will plans and policy makers; and comment on the protocol and researchers. final report and provide input to ensure that the review has practical relevance to likely end-users. 2.3.1 Protect the The name of the EPC Not mentioned. Sponsorship by any commercial independence of the review conducting an SR is not sources with financial interests team to make the final provided to the public until in the conclusions of Cochrane decisions about the design, the draft report the to protect reviews is prohibited. The sponsor analysis, and reporting of authors from external influence. should not be allowed to delay or the review Key informant and technical prevent publication of a review, or experts have no role in writing, interfere with the independence of analyzing, or drafting paper. the authors of reviews. Peer reviewers are selected to have no significant COI.

OCR for page 249
2.4 Manage bias and COI Provides guidance on Not mentioned. Not mentioned. for individuals providing managing bias and COI for input into the systematic individuals providing input review into the SR (see below). 2.4.1 Require individuals to Participants, consultant, The next edition of the Not mentioned. disclose potential COI and subcontracts, and other guidance will make explicit professional or intellectual technical experts must disclose that all biases must be declared bias in writing any financial, early in process and steps business, and professional taken to ensure that these do interests that are related to the not impact on the review. subject matter of a review. 2.4.2 Exclude input from Advisors or experts are not The next edition of the Not mentioned. individuals whose COI or automatically excluded if guidance will note that bias would diminish the there are conflicts, particularly professional and intellectual credibility of the review in for topic development and bias must be declared, but the eyes of the intended refinement. When an individual should not preclude being part user has a potential conflict and is of an advisory group. providing input as part of a group, the conflicts must be disclosed and balanced. Experts may be excluded for conflicts depending on the stage of the review and how input is provided. 2.5 Formulate the topic for Provides guidance on topic Provides guidance on topic Provides guidance on topic the systematic review formulation (see below). formulation (see below). formulation (see below). 253 continued

OCR for page 249
TABLE D-1 Continued 254 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 2.5.1 Confirm the need for a AHRQ has specific criteria Must check if there is an Many Cochrane Review Groups new review to ensure the need for a existing or ongoing review have developed priorities for new review. Topics should on topic to see if new review reviews of importance. Topics have strong potential for justified. Search Database suggested by review authors must significantly improving health of Abstracts of Reviews of be approved by the appropriate outcomes or for reducing Effects, Cochrane Database Cochrane Review Group. The unnecessary care or cost of Systematic Reviews, and background section of the protocol and also concern important others. If there is an existing should clearly state the rationale decisions for consumers or for review, assess for quality for the review and should explain one or more of these groups: (using CRD critical appraisal). why the questions being asked are patients, clinicians, health If high quality, see if update important. It might also mention system leaders, purchasers, is justified. If completed some why this review was undertaken payers, and policy makers. time ago, an update may be and how it might relate to a wider Should consider the available justified. review of a general problem. research basis for a topic and if current, high-quality research is available or underway. Try to reduce duplication of existing reviews. 2.5.2 Develop an analytic Develop an analytic framework Communicate key contextual Not mentioned. framework that clearly lays that portrays relevant clinical factors and conceptual issues out the chain of logic that concepts and the clinical relevant to review question. links the health intervention logic underlying beliefs about Explain why review is required to the outcomes of interest the mechanism by which and rationale for focus of the and defines the key clinical interventions may improve review. questions to be addressed health outcomes. by the systematic review

OCR for page 249
2.5.3 Use a standard format Topics selected for comparative The protocol should include The protocol should include a well- to articulate each clinical effectiveness SRs are focused the review questions framed formulated question. Questions question of interest into research questions using using PICOS: population, are stated broadly as review the PICOTS mnemonic: intervention, comparator, “objectives” and specified in detail population, intervention, outcome, and setting. The as “criteria for considering studies comparator, outcome, timing, review question may be for this review.” The clinical and setting. presented in general terms, or, question should address all of the more often the actual question elements in PICO: population, is discussed by the review intervention, comparator, and team and an objective, or series outcome. of objectives, framed by the PICOS format is agreed. 2.5.4 State the rationale for Fully explain the rationale State the objectives for The background section of the each clinical question for formulating each clinical undertaking the review. protocol should clearly state the question. rationale for the review and should explain why the questions being asked are important. Each review group has a title registration process. Some of their forms require statement of motivation for doing the review. 2.5.5 Refine each question Topic refinement requires Engaging stakeholders who If present, an advisory group may based on user and input from stakeholders (key are likely to be involved be involved in making and refining stakeholder input informants) that represent the in implementing the decisions about the interventions broad-based constituencies of recommendations of the review of interest, the populations to be the EHC (Effective Health Care helps to ensure that the review included, priorities for outcomes, Program) and for the particular is relevant to their needs. and possibly subgroup analyses. topic area. The form of involvement depends on the project. Where reviews have strict time constraints, consultation may 255 be impossible. continued

OCR for page 249
TABLE D-1 Continued 256 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 2.6 Develop a systematic Requires a protocol (see below). Requires a protocol (see below). Requires a protocol (see below). review protocol 2.6.1 Describe the context Fully explain the rationale Explain why the review The background section should and rationale for the review for formulating each clinical is required and provide a address the context for the review from both a decision- question. rationale for the inclusion question based on an already- making and research criteria and focus of the review. formed body of knowledge, the perspective rationale for the review, and why the questions being asked are important. 2.6.2 Describe the study Protocol should include Inclusion criteria should be set Include the criteria for selecting screening and selection detailed explanation and out in the protocol. Specify the studies for the review, including criteria (inclusion/exclusion justification for inclusion/ process by which decision on the types of studies, types of criteria) exclusion criteria. the selection of studies will be participants, types of interventions, made, including the number and types of outcome measures. of researchers who will screen titles and abstracts and then full papers, and the method for resolving disagreements about study eligibility. 2.6.3 Describe precisely Define the outcome measures. Specify the comparators and Review authors should consider which outcome measures, interventions that are eligible how outcomes may be measured, time points, interventions, for the review, the defined set both in terms of the type of scale and comparison groups will of relevant outcomes, and the likely to be used and the timing of be addressed timing of outcome assessment. measurement.

OCR for page 249
Specify the interventions of interest and the interventions against which these will be compared. When specifying drug interventions, factors such as the drug preparation, route of administration, dose, duration, and frequency should be considered. 2.6.4 Describe the search Describe the search strategy Include the preliminary search Search methods must be described. strategy for identifying (including databases and strategy for identifying relevant Inclusion of the search strategy is relevant evidence search terms) in the protocol. results. This should specify optional. the databases and additional sources that will be searched, and also the likely search terms to be used. 2.6.5 Describe the Provide plans to assess Inclusion criteria should be set Include the methods used to apply procedures for study evidence against inclusion/ out in the protocol. Specify the the selection criteria. selection exclusion criteria. process by which decision on the selection of studies will be made, including the number of researchers who will screen titles and abstracts and then full papers, and the method for resolving disagreements about study eligibility. 257 continued

OCR for page 249
TABLE D-1 Continued Agency for Healthcare 258 Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 2.6.6 Describe the data Describe how the data are Include the data extraction Include the methods used to extract extraction strategy extracted from each study and strategy. The data collected will or obtain data from published methods for collecting and depend on the type of question reports or from original researchers. managing the information. being addressed, and the types Identify key characteristics of study available. Describe that might be necessary for the data to be extracted and evidence synthesis due to their provide details on the software role in effect modification of used for recording data. the intervention–treatment Specify if authors of primary association and thus limit the studies will be contacted to applicability of findings. provide missing or additional data. If using foreign language papers, may need to specify translation arrangements. 2.6.7 Describe the process Describe how discrepancies Describe how discrepancies Include a process for identifying for identifying and among researchers will be between researchers will be and resolving disagreement resolving disagreement resolved in the protocol. resolved. between researchers in study between researchers in selection, data extraction, and study selection and data assessment of risk of bias in extraction decisions included studies. 2.6.8 Describe the approach Describe the approach for Specify the method of study Include the method used to assess to critically appraising assessing study quality in the appraisal, including the details risk of bias in individual studies. individual studies protocol. of how the study appraisal is to be used (e.g., will inform sensitivity analyses). Also specify the process for conducting the appraisal, the number of researchers involved, and how disagreements will be resolved.

OCR for page 249
2.6.9 Describe the method Discuss how evidence will Specify the data synthesis Address whether a meta-analysis for evaluating the body be summarized in a clinically strategy. State how hetero- is intended, whether to prespecify of evidence, including the relevant manner either as geneity will be explored an effect measure, how to handle quantitative and qualitative a narrative or using meta- and quantified, under what heterogeneity, whether to assume synthesis strategies analysis. Clearly state circumstances a meta- a fixed-effect or random-effects plans for meta-analysis and analysis would be considered model, how to incorporate risk of predefine clinical groups appropriate, and whether bias, how to handle missing data, that are too heterogeneous a fixed or random-effects how to address reporting biases. to allow for meta-analysis or model or both would be used. clinical groups for which the Describe any planned subgroup A qualitative synthesis strategy is qualitative analysis will be or sensitivity analyses or not required. presented separately. Identify, investigation of publication a priori, subgroups that will be bias. explored to explain potential heterogeneity. The approach to narrative synthesis should be outlined. Describe how the report will Should specify the outcomes present findings, including the of interest and what effect ordering of outcomes or other measures will be used. categorization scheme. Describe methods for prioritizing or selecting the most important outcomes to be presented in tables or summary key results (i.e., did key informants or the Technical Expert Panel help inform prioritization of outcomes?). 259 continued

OCR for page 249
TABLE D-1 Continued 260 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration Describe how criteria will be used to determine overall strength of the body of evidence for each key question and which outcomes will be graded. 2.6.10 Describe and justify Clearly state plans for meta- Any planned subgroup Authors should, whenever possible, any planned analyses analysis and predefine analyses should also be prespecify characteristics in the of differential treatment clinical groups that are too described in the data synthesis protocol that later will be subject effects according to heterogeneous to allow for section. to subgroup analyses or meta- patient subgroups, how an meta-analysis or clinical groups regression. intervention is delivered, for which the qualitative or how an outcome is analysis will be presented measured separately. Identify, a priori, subgroups that will be explored to explain potential heterogeneity. 2.6.11 Describe the proposed EPCs are to include time line Not mentioned. Protocols include a date when the timetable for conducting the in their workplan to AHRQ. review is expected. Reviews must review AHRQ posts approximate final be completed within 2 years, or completion date. they may be withdrawn from the Cochrane Database of Systematic Reviews.

OCR for page 249
2.7 Submit the protocol for Not mentioned. Some commissioning or Protocols should go out for external peer review funding bodies may require peer review. that they formally approve the protocol, and will provide input to the draft protocol. In addition, other stakeholders, such as clinical and methodological experts, patient groups, and service users, may be consulted in approving the protocol. 2.7.1 Provide a public Public comment period is Advocate where evidence base Approved protocols are comment period for the provided for key questions, but is contested: create dedicated, automatically published on the Cochrane Database of Systematic protocol and publicly report not protocol. Protocol is posted publicly accessible websites on disposition of comments publicly, but not available for that provide information Reviews. The Cochrane Library peer or public comment. The about all aspects of the Feedback tool allows users to protocol is developed with review. These websites enable provide comments on and feedback input by and reviewed by a external scrutiny of the review of Cochrane reviews and protocols Technical Expert Panel. process, and include feedback in The Cochrane Library. If accepted, facilities for interested parties the feedback will be published. to comment, ask questions, or submit evidence for consideration. continued 261

OCR for page 249
TABLE D-1 Continued 262 Agency for Healthcare Research and Quality (AHRQ) Centre for Reviews and Standards and Elements Effective Health Care Program Dissemination (CRD) The Cochrane Collaboration 2.8 Make the final protocol Key questions are available Not mentioned. The next Approved protocols are publicly available, and add for public review and edition will recommend automatically published on the any amendments to the Cochrane Database of Systematic comment prior to start of making the protocol publicly protocol in a timely fashion review. Protocol is made available. Modifications to Reviews. Any changes to the publicly available, but not for the protocol should be clearly protocol must be documented and comment after start of a review. documented and justified. reported in the completed review. Modifications to protocol should be clearly documented and justified, then posted and available to the public. NOTE: Some information on methods recommended by AHRQ, CRD, and Cochrane was provided via personal communication with Stephanie Chang, EPC Program Task Order Officer, AHRQ (October 5, 2010); Lesley Stewart, Director, CRD (October 14, 2010); and Julian Higgins, Senior Statistician, MRC Biostatistics Unit, Institute of Public Health, University of Cambridge (October 4, 2010).

OCR for page 249
263 APPENDIX D REFERENCES Atkins, D., S. Chang, G. Gartlehner, D. I. Buckley, E. P. Whitlock, E. Berliner, and D. Matchar. 2010. Assessing the applicability of studies when comparing medical interventions. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effectivehealthcare. ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?productid=603& pageaction=displayproduct (accessed January 19, 2011). Chou, R., N. Aronson, D. Atkins, A. S. Ismaila, P. Santaguida, D. H. Smith, E. Whitlock, T. J. Wilt, and D. Moher. 2010. AHRQ series paper 4: Assessing harms when com- paring medical interventions: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):502–512. CRD (Centre for Reviews and Dissemination). 2009. Systematic reviews: CRD’s guidance for undertaking reviews in health care. York, UK: York Publishing Services, Ltd. Fu, R., G. Gartlehner, M. Grant, T. Shamliyan, A. Sedrakyan, T. J. Wilt, L. Griffith, M. Oremus, P. Raina, A. Ismaila, P. Santaguida, J. Lau, and T. A. Trikalinos. 2010. Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. In Methods guide for compara- tive effectiveness reviews, edited by Agency for Healthcare Research and Qual- ity. http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides- reviews-and-reports/?pageaction=displayProduct&productID=554 (accessed January 19, 2011). Helfand, M., and H. Balshem. 2010. AHRQ series paper 2: Principles for develop - ing guidance: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):484–490. Higgins, J. P. T., and S. Green, eds. 2008. Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons. Norris, S., D. Atkins, W. Bruening, S. Fox, E. Johnson, R. Kane, S. C. Morton, M. Oremus, M. Ospina, G. Randhawa, K. Schoelles, P. Shekelle, and M. Viswanathan. 2010. Selecting observational studies for comparing medical inter- ventions. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/ index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayProduct &productID=454 (accessed January 19, 2011). Owens, D. K., K. N. Lohr, D. Atkins, J. R. Treadwell, J. T. Reston, E. B. Bass, S. Chang, and M. Helfand. 2010. AHRQ series paper 5: Grading the strength of a body of evidence when comparing medical interventions: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):513–523. Relevo, R., and H. Balshem. 2011. Finding evidence for comparing medical interven - tions. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/ index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayProduct &productID=605 (accessed January 19, 2011). Slutsky, J., D. Atkins, S. Chang, and B. A. Collins Sharp. 2010. AHRQ series paper 1: Comparing medical interventions: AHRQ and the Effective Health Care Pro - gram. Journal of Clinical Epidemiology 63(5):481–483. White, C. M., S. Ip, M. McPheeters, T. S. Carey, R. Chou, K. N. Lohr, K. Robinson, K. McDonald, and E. Whitlock. 2009. Using existing systematic reviews to replace de novo processes in CERs. In Methods guide for comparative effectiveness reviews, edited by Agency for Healthcare Research and Quality. http://www.effective- healthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?page action=displayProduct&productID=329 (accessed January 19, 2011).

OCR for page 249
264 FINDING WHAT WORKS IN HEALTH CARE Whitlock, E. P., S. A. Lopez, S. Chang, M. Helfand, M. Eder, and N. Floyd. 2010. AHRQ series paper 3: Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology 63(5):491–501.