JBI offers a suite of critical appraisal instruments that are freely available to systematic reviewers and researchers investigating the methodological limitations of primary research studies. The JBI instruments are designed to be study-specific and are presented as questions in a checklist. The JBI instruments have existed in a checklist-style format for approximately 20 years; however, as the field of research synthesis expands, many of the tools offered by JBI have become outdated. The JBI critical appraisal tools for quantitative studies (eg, randomized controlled trials, quasi-experimental studies) must be updated to reflect the current methodologies in this field. Cognizant of this and the recent developments in risk-of-bias science, the JBI Effectiveness Methodology Group was tasked with updating the current quantitative critical appraisal instruments. This paper details the methods and rationale that the JBI Effectiveness Methodology Group followed when updating the JBI critical appraisal instruments for quantitative study designs. We detail the key changes made to the tools and highlight how these changes reflect current methodological developments in this field.

Centre for Health Informatics Australian Institute of Health Innovation Macquarie University Sydney NSW Australia

Czech National Centre for Evidence Based Healthcare and Knowledge Translation Faculty of Medicine Institute of Biostatistics and Analyses Masaryk University Brno Czech Republic

JBI Faculty of Health and Medical Sciences The University of Adelaide Adelaide SA Australia

Queen's Collaboration for Health Care Quality Queen's University Kingston ON Canada

The Nottingham Centre for Evidence Based Healthcare A JBI Centre of Excellence School of Medicine University of Nottingham Nottingham UK

Zobrazit více v PubMed

Porritt K, Gomersall J, Lockwood C. JBI's systematic reviews: study selection and critical appraisal. Am J Nurs 2014;114(6):47–52.

JBI. Critical appraisal tools [internet]. Adelaide, JBI; n.d. [cited 2022 Nov 29]. Available from: https://jbi.global/critical-appraisal-tools .

Aromataris E, Munn Z. Aromataris E, Munn Z Chapter 1: JBI Systematic Reviews JBI Manual for Evidence Synthesis [internet]. Adelaide, JBI; 2020 [cited 2022 Nov 29]. Available from: https://synthesismanual.jbi.global .

Tufanaru C, Munn Z, Aromataris E, Campbell J, Hopp L Aromataris E, Munn Z. Chapter 3: Systematic reviews of effectiveness JBI Manual for Evidence Synthesis [internet]. Adelaide, JBI; 2020 [cited 2022 Nov 29]. Available from: https://synthesismanual.jbi.global .

Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A, et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. JBI Evid Synth 2020;18(10):2127–2133.

Munn Z, Moola S, Riitano D, Lisy K. The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence. Int J Health Policy Manag 2014;3(3):123.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, et al. Aromataris E, Munn Z. Chapter 2: Systematic reviews of qualitative evidence JBI Manual for Evidence Synthesis [internet]. Adelaide, JBI; 2020 [cited 2022 Nov 29]. Available from: https://synthesismanual.jbi.global .

McArthur A, Klugarova J, Yan H, Florescu S. Aromataris E, Munn Z Chapter 4: Systematic reviews of text and opinion JBI Manual for Evidence Synthesis [internet]. Adelaide, JBI; 2020 [cited 2022 Nov 29]. Available from: https://synthesismanual.jbi.global .

Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998;52(6):377–384.

Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.

JBI. QARI user manual. Adelaide, JBI; 2003.

Stone JC, Glass K, Clark J, Munn Z, Tugwell P, Doi SAR. A unified framework for bias assessment in clinical research. Int J Evid Based Healthc 2019;17(2):106–120.

Stone JC, Gurunathan U, Aromataris E, Glass K, Tugwell P, Munn Z, et al. Bias assessment in outcomes research: the role of relative versus absolute approaches. Value Health 2021;24(8):1145–1149.

Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane handbook for systematic reviews of interventions version 6.2 [internet]. Cochrane; 2021 [cited 2022 Nov 29]. Available from: https://training.cochrane.org/handbook .

Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021;372:n160.

Group GW. Grading quality of evidence and strength of recommendations. BMJ 2004;328(7454):1490.

Munn Z, Stone J, Aromataris E, Klugar M, Sears K, Leonardi-Bee J, et al. Assessing the risk of bias of quantitative analytical studies: introducing the vision for critical appraisal within JBI systematic reviews. JBI Evid Synth. 2023;21(3):467–71.

Moher D. Quality assessment of clinical trials. Wiley Encyclopedia of Clinical Trials; 2007.

Slack MK, Draugalis JR. Establishing the internal and external validity of experimental studies. Am J Health Syst Pharm. 2001;58(22):2173–2181.

Cook TD, Campbell DT. Quasi-experimentation: design and analysis issues for field settings. Houghton Mifflin; 1979.

García-Pérez MA. Statistical conclusion validity: some common threats and simple remedies. Front Psychol. 2012;3:325.

Tufanaru C, Huang WJ, Tsay S-F, Chou S-S. Statistics for systematic review authors. Lippincott Williams & Wilkins; 2012.

Hyman R. Quasi-experimentation: design and analysis issues for field settings (book). J Person Assess 1982;46(1):96–97.

Cook TD, Campbell DT, Shadish W. Experimental and quasi-experimental designs for generalized causal inference: Houghton Mifflin 2002.

Tikka C, Verbeek J, Ijaz S, Hoving JL, Boschman J, Hulshof C, et al. Quality of reporting and risk of bias: a review of randomised trials in occupational health. Occup Environ Med 2021;78(9):691–696.

Horsley T, Galipeau J, Petkovic J, Zeiter J, Hamstra SJ, Cook DA. Reporting quality and risk of bias in randomised trials in health professions education. Med Educ 2017;51(1):61–71.

Suri H. Zawacki-Richter O, Kerres M, Bedenlier S, Bond M, Buntins K Ethical considerations of conducting systematic reviews in educational research Systematic reviews in educational research: methodology, perspectives and application. Springer Fachmedien Wiesbaden:Wiesbaden; 2020. 41–54p.

Ferguson L. External validity, generalizability, and knowledge utilization. J Nurs Scholarsh 2004;36(1):16–22.

Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.

Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919.

Bero L, Chartres N, Diong J, Fabbri A, Ghersi D, Lam J, et al. The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures. Syst Rev 2018;7(1):242.

Morgan R. The ROBINS-E tool (Risk Of Bias In Non-randomized Studies-of Exposures) [internet]. University of Bristol; 2017 [cited 2022 Nov 29]. Available from: https://www.bristol.ac.uk/population-health-sciences/centres/cresyda/barr/riskofbias/robins-e/ .

Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Churchill R, Chandler J, Cumpston MS, editors. Cochrane handbook for systematic reviews of interventions [internet]. Cochrane; 2017 [cited 2022 Nov 29]. Available from: https://training.cochrane.org/handbook .

Steiner PM, Cook TD, Shadish WR, Clark MH. The importance of covariate selection in controlling for selection bias in observational studies. Psychol Methods 2010;15(3):250.

Nunan D, Aronson J, Bankhead C. Catalogue of bias: attrition bias. BMJ Evid Based Med 2018;23(1):21–22.

Morgan RL, Thayer KA, Bero L, Bruce N, Falck-Ytter Y, Ghersi D, et al. GRADE: assessing the quality of evidence in environmental and occupational health. Environ Int 2016;92:611–616.

Weinberg CR. Toward a clearer definition of confounding. Am J Epidemiol 1993;137(1):1–8.

Skelly AC, Dettori JR, Brodt ED. Assessing bias: the importance of considering confounding. Evid Based Spine Care J 2012;3(01):9–12.

Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.