Systematic Reviews - Social Sciences

Conducting a preliminary search:

• It will give you an idea of what literature is available and if your keywords map to the scientific literature. This is a good time to involve a librarian who can give you tips on searching for (limiting or expanding) subject headings.  
• And, you will want to know if the research/question that you are thinking about investigating has already been addressed.  

Recognizing and reducing bias in a systematic review is a big deal. Document all of your steps starting now. 

A protocol is the plan (or roadmap) for the research.  A complete plan should include the conceptual background, the research question or questions, objectives, the scope or extent of the review, the methods for searching (including the resources and the search strategies), screening processes specifying the inclusion and exclusion criteria, data extraction, quality appraisal, and synthesis.  Many protocols are published. The protocol establishes transparency, rigor, and is helpful to other researchers. Importantly when the criteria is established in advance it minimizes bias in the screening and selection process. 

Examples of protocol papers are found on the following sites:

BMJ Open Science

Social Science Protocols Journal 

PubMed - search for protocol papers

Examples of research question formulation guides:

There are numerous frameworks to assist with formulation research questions.  See the supplementary file from: 

Booth A, Noyes J, Flemming K, et al. Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ Global Health 2019;4:e001107. https://gh.bmj.com/content/4/Suppl_1/e001107

It is a good idea to search databases of systematic reviews, such as Cochrane and Campbell. Cline Library has access to Cochrane, which focuses on health interventions. While it isn't always relevant for research questions in the social sciences it can serve as a good place to look at examples.  You will at least get an idea of how time consuming a systematic review can be (months to years).  

A minimum of two databases is necessary. However the number of databases and which ones to search will depend on your research questions. Another good time to ask your librarian for help, if you have questions about which ones to include. 

Check out the links on the Evidence synthesis database box to the left. 

Cline Library offers support for many citation management tools including Mendeley and Zotero.  These tools are especially helpful when you need to export a large number of records, or if you are searching multiple databases that may have duplicative content.  Citation tools can find and merge duplicate records.  

At this stage you  (and one or two other reviewers) will identify articles that meet the eligibility criteria ( i.e. inclusion and exclusion). Have at least two researchers screening the articles. A third reviewer can settle any discrepancies. 

Screening happens in two steps:

1. Title and abstract level screening to exclude studies that are ineligible according to the inclusion and exclusion criteria. 
2. Full text screening. 

See: Polanin, J. R., Pigott, T. D., Espelage, D. L., & Grotpeter, J. K. (2019). Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses. Research Synthesis Methods, 10(3), 330–342. https://doi.org/10.1002/jrsm.1354

Methodological quality assessment  - there are several tools available to review the methodological quality (risk of bias) described in articles.