Research Services: Evidence Synthesis Support

An evidence synthesis protocol states your rationale, hypothesis, and planned methodology. 

• Evidence Synthesis Protocol Template

  Use this document as a template to prepare a protocol for a range of evidence synthesis methodologies (such as systematic reviews, scoping reviews, or systematic maps).

• Planning Worksheet for Structured Literature Reviews

  Writing a literature review for a research paper or as part of your thesis? Even if you’re not performing a full evidence synthesis, completing the items on this checklist and keeping them as record of your planned work (like a study protocol) ensures reproducibility, transparency, and reduction of bias.

To help formulate your research question, some research question frameworks are listed below.

PICO for Quantitative Studies

• P       Population/Problem
• I        Intervention/Exposure
• C       Comparison
• O      Outcome

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

While PICO is a helpful framework for clinical research questions, it may not be the best choice for other types of research questions, especially outside the health sciences.  Here are a few others (for a comprehensive, but concise, overview of the almost 40 different types of research question frameworks, see this review from the British Medical Journal: Rapid review of existing question formulation frameworks)

PICo for Qualitative Studies

• P       Population/Problem
• I         Phenomenon of Interest 
• Co    Context

​​SPICE

• S    Setting
• P   Perspective (for whom)
• I    Intervention/Exposure
• C   Comparison
• E   Evaluation

SPIDER

• S     Sample
• PI   Phenomenon of Interest
• D    Design
• E     Evaluation
• R    Study Type

Inclusion and Exclusion Criteria

Inclusion and exclusion criteria are developed after a research question is finalized but before a search is carried out. They determine the limits for the evidence synthesis and are typically reported in the methods section of the publication. For unfamiliar or unclear concepts, a definition may be necessary to adequately describe the criterion for readers. 

Graph developed by: http://unimelb.libguides.com/c.php?g=492361&p=3368110

Librarians can recommend databases and other sources to search for a systematic review. The sources you choose will depend on your research question and the disciplines in which relevant research may be conducted. Below are some examples of scholarly databases.  Check the library's database list for a full list of available sources across all disciplines.

A search strategy should aim to be exhaustive, encompass multiple databases, when appropriate include grey literature, and be reproducible. PRISMA guidelines state that the full search strategy for at least one major database should be reported in an appendix and published along with the review ( http://www.prisma-statement.org/).

Most databases use a controlled vocabulary (a certain way words and phrases are indexed). This may require using different terms for different databases. Consult with a librarian if you have questions or would like assistance. 

Grey literature is produced outside of traditional publishing and distribution norms. This can include, among other things, white papers, government publications, working papers, preprints, unpublished trial data, and conference proceedings and abstracts. Grey literature can be found in some citation databases, as well as databases dedicated to grey literature.

Some databases dedicated to grey literature include:

• Grey Literature Report

  A report from the NY Academy of Medicine of gray literature published between 1999 - 2016

• Open Grey

  "Open access to 700.000 bibliographical references of grey literature (paper) produced in Europe and allows you to export records and locate the documents"

• GreySource

  "A selection of web-based resources on grey literature"

• Grey Matters

  Canadian Agency for Drugs and Technologies in Health provides a practical tool for searching health-related grey literature

It is recommended that you register your protocol prior to conducting your review. This will improve transparency and reproducibility, reduce bias, and will also ensure that other research teams do not duplicate your efforts. 

Open Science Framework An open source, multidisciplinary web application that connects and supports the research workflow. Researchers use the OSF to collaborate, document, archive, share, and register research projects, materials, and data. OSF can be used to pre-register a systematic review protocol and to share documents such as a Zotero library, search strategies, and data extraction forms.

Cochrane

"Our mission is to promote evidence-informed health decision-making by producing high-quality, relevant, accessible systematic reviews and other synthesized research evidence. Our work is internationally recognized as the benchmark for high-quality information about the effectiveness of health care."
Disciplines: Healthcare

Campbell Collaboration

"The Campbell Collaboration promotes positive social and economic change through the production and use of systematic reviews and other evidence synthesis for evidence-based policy and practice."
Disciplines: Business and Management, Crime and Justice, Disability, Education, International Development, Knowledge Translation and Implementation, Methods, Nutrition, and Social Welfare

Collaboration for Environmental Evidence

"An open community of stakeholders working towards a sustainable global environment and the conservation of biodiversity. CEE seeks to promote and deliver evidence syntheses on issues of greatest concern to environmental policy and practice as a public service."
Disciplines: Environmental issues

PROSPERO

An international database of prospectively registered systematic reviews in health and social care. Key features from the review protocol are recorded and maintained as a permanent record. (Does not accept scoping reviews)
Disciplines: Health and Social Care, Welfare, Public Health, Education, Crime, Justice, and International Development

Evidence synthesis methods require authors to search multiple databases, and not all databases accept the same search "syntax." Each individual database requires use of specialized search syntax, and therefore evidence synthesis search strategies must be 'translated' between databases. 

For example, a search for lung cancer[Title/Abstract] in PubMed will show you all citations with the phrase "lung cancer" in the title, abstract, or keywords, but a search for lung cancer[Title/Abstract] in Scopus will not work at all. 

It is very important to keep an accurate record of your searches so that they can be included in your report, enabling someone to reproduce them if necessary.

• Keep a record of the date you searched, the name of the database, and the platform.
• Document your search strategy for each resource.

Start with a title/abstract screening to remove studies that are clearly not related to your topic. Use your inclusion/exclusion criteria to screen the full-text of studies. It is highly recommended that two independent reviewer screen all studies, resolving areas of disagreement by consensus.

Use a spreadsheet, or systematic review software, to extract all relevant data from each included study. It is recommended that you pilot your data extraction tool, to determine if other fields should be included or existing fields clarified.

Use a Risk of Bias tool (such as the Cochrane RoB Tool) to assess the potential biases of studies in regards to study design and other factors. You can adapt existing tools to best meet the needs of your review, depending on the types of studies included.