Systematic Review Toolbox

Critical Appraisal Questions

  1. Is the study question relevant?
  2. Does the study add anything new?
  3. What type of research question is being asked?
  4. Was the study design appropriate for the research question?
  5. Did the study methods address the most important potential sources of bias?
  6. Was the study performed according to the original protocol?
  7. Does the study test a stated hypothesis?
  8. Were the statistical analyses performed correctly?
  9. Do the data justify the conclusions?
  10. Are there any conflicts of interest?

The University of Sydney Library, Systematic Reviews: Assessment Tools and Critical Appraisal

Taylor, P., Hussain, J. A., & Gadoud, A. (2013). How to appraise a systematic review. British Journal of Hospital Medicine, 74(6), 331-334. doi:10.12968/hmed.2013.74.6.331

Young, J. M., & Solomon, M. J. (2009). How to critically appraise an article. Nature Clinical Practice Gastroenterology and Hepatology, 6(2), 82-91. doi:10.1038/ncpgasthep1331

Quality Assessment

Assessing the quality of evidence contained within a systematic review is as important as analyzing the data within. Results from a poorly conducted study can be skewed by biases from the research methodology and should be interpreted with caution. Such studies should be acknowledged as such in the systematic review or outright excluded. Selecting an appropriate tool to help analyze strength of evidence and imbedded biases within each paper is also essential. If using a systematic review manuscript development tool (e.g., RevMan), a checklist may be built into the software. Other software (e.g., Rayyan) may help with screening search results and discarding irrelevant studies. The following tools/checklists may help with study assessment and critical appraisal.

Cochrane RoB 2 (RCT)

Rayyan for Study Selection

JBI SUMARI (Multiple)

QUADAS-2 (Diagnostic Test Accuracy)

Requesting Research Consultation

The Health Sciences Library provides consultation services for University of Hawaiʻi-affiliated students, staff, and faculty. The John A. Burns School of Medicine Health Sciences Library does not have staffing to conduct or assist researchers unaffiliated with the University of Hawaiʻi. Please utilize the publicly available guides and support pages that address research databases and tools.

icon   Follow us
icon   Follow us
icon   Follow us