Psychology, Yuma: Thinking Critically

Evaluation Tools

The Appraisal of Guidelines for Research & Evaluation Instrument (AGREE II) addresses the variability in the quality of practice guidelines.

Centre for Evidence-Based Medicine (CEBM). Critical Appraisal Tools states it "contains useful tools and downloads for the critical appraisal of different types of medical evidence. Example appraisal sheets are provided together with several helpful examples."

Critical Appraisal Skills Programme (CASP) Checklists for many different study types are provided.

Critical Review Form for Qualitative Studies

Version 2, developed out of McMaster University

Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS)

Downes, M. J., Brennan, M. L., Williams, H. C., & Dean, R. S. (2016). Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ open, 6(12), e011458. https://doi.org/10.1136/bmjopen-2016-011458

Downs & Black Checklist for Assessing Studies

Downs, S. H., & Black, N. (1998). The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of epidemiology and community health, 52(6), 377–384. https://doi.org/10.1136/jech.52.6.377

GRADE = Grading of Recommendations Assessment, Development and Evaluation (GRADE). It is a  working group which "has developed a common, sensible and transparent approach to grading quality (or certainty) of evidence and strength of recommendations."

Grade Handbook discusses the GRADE method for grading quality of evidence.

MAGIC (Making GRADE the Irresistible choice) - Clear succinct guidance in how to use GRADE

Joanna Briggs Institute. Critical Appraisal Tools - "JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers." It includes checklists for 13 types of articles.

Latitudes Network is a searchable library of validity assessment tools for use in evidence syntheses. This website also provides access to training on the process of validity assessment.

Mixed Methods Appraisal Tool is used to appraise a mix of studies that are included in a systematic review - qualitative research, RCTs, non-randomized studies, quantitative studies, mixed methods studies.

RoB 2 Tool

Higgins JPT, Sterne JAC, Savović J, Page MJ, Hróbjartsson A, Boutron I, Reeves B, Eldridge S. A revised tool for assessing risk of bias in randomized trials In: Chandler J, McKenzie J, Boutron I, Welch V (editors). Cochrane Methods. Cochrane Database of Systematic Reviews 2016, Issue 10 (Suppl 1). dx.doi.org/10.1002/14651858.CD201601

ROBINS-I Risk of Bias for non-randomized (observational) studies or cohorts of interventions

Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., Carpenter, J. R., Chan, A. W., Churchill, R., Deeks, J. J., Hróbjartsson, A., Kirkham, J., Jüni, P., Loke, Y. K., Pigott, T. D., Ramsay, C. R., … Higgins, J. P. (2016). ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ (Clinical research ed.), 355, i4919. https://doi.org/10.1136/bmj.i4919

Scottish Intercollegiate Guidelines Network. Critical Appraisal Notes and Checklists

"Methodological assessment of studies selected as potential sources of evidence is based on a number of criteria that focus on those aspects of the study design that research has shown to have a significant effect on the risk of bias in the results reported and conclusions drawn. These criteria differ between study types, and a range of checklists is used to bring a degree of consistency to the assessment process."

The TREND Statement (CDC)

Des Jarlais, D. C., Lyles, C., Crepaz, N., & TREND Group (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. American journal of public health, 94(3), 361–366. https://doi.org/10.2105/ajph.94.3.361