Systematic Reviews: Systematic Review Process

The following steps outline how to conduct a systematic review (from Wright et al., 2007)

1. Defining the Research Question

To begin, researchers define the focus of the review using a research question. Research questions are usually created using the PICO components. Beginning with a precise, well-reasoned research question will reduce the amount of time required to retrieve relevant literature citations and will also reduce bias compared to approaches in which the question is shaped in response to literature already reviewed by the authors. The type of question (e.g. Therapy, Prevention, Prognosis, Test/Diagnosis, Etiology) will often lend itself to a specific type of article (e.g. a Randomized Clinical Trial, cohort studies, Case Control Trial). Reviewers should identify what level of evidence is appropriate for the type of question they are asking.

2. Create a Research Protocol

Next, the reviewers should outline the methods they will use for the search. A written document should outline what methods and terms will be used to search the literature, extract data, and the inclusion and exclusion criteria. The quality of the literature that is selected for inclusion will impact the quality of the final review. For example, cohort and case studies are considered a much lower quality of evidence than RCTs (see EBM Pyramid of Evidence in EBM Resources). Your search strategy may use indexing terms or keywords.

3. Search the Literature

Reviewers should now conduct a literature search. This is an exhaustive process, requiring the use of several search strategies, that results in a comprehensive list of potentially relevant studies. The use of a bibliographic management software (e.g. EndNote) can be crucial for managing the results of this step. Processes appropriate for a literature search include:

• Search multiple, general and specific bibliographic databases. At a minimum, search MEDLINE (PubMed or Ovid MEDLINE), Cochrane Databases, and EMBASE. When appropriate include other databases, such as CINAHL, PsycINFO, and Current Contents Connect.
• Use Web of Science (Cited References) to see which studies have cited an article (or study)
• Hand search specific journals, manufacturers
• Scan bibliographies of review articles, selected studies, book chapters, clinical practice guidelines, and theses
• Use the PubMed “Related Articles” feature

4. Extract Data

Once the studies are obtained, data from each study is put into tabular format to manage the range of studies included and provide raw data for statistical calculations (in step 6). The items extracted often include:

• Reference (citation)
• Objective of the study
• Type of Trial
• Population
• Intervention
• Control or Comparison
• Outcome
• Comments

5. Appraise the Quality of the Data

Next, reviewers should evaluate the quality of the studies found for quality and applicability. Quality factors may include, but are not limited to:

• Patient numbers and characteristics at baseline
• Study design/type of trial
• Randomization
• Blinding
• Recruitment
• Follow-up duration and numbers
• Generalizability
• Outcomes and how they are defined and assessed
• Risk factors
• Adverse effects
• Year of publication

6. Analysis and Statistical Confirmation

Using both common sense and statistical computations, decide whether the data are similar enough to be combined statistically and mathematically. If the data are not similar enough to combine, no further statistical analysis should be done. If the data can be combined, the systematic review article becomes a “meta-analysis”. (A meta-analysis is a subcategory of a systematic review.)

7. Present the Results

Last, reviewers will need to display their results in a coherent manner. Base your conclusions on the best evidence that is available from your review. Systematic reviews include data in several different formats. The raw data are presented in a table form and are accompanied with text. You will also want to make recommendations for future studies, particularly by identifying areas which have not had sufficient study.