Research Pearls: Checklists and Flowcharts to Improve Research Quality


      To instill quality in published clinical research, reporting guidelines, consisting of checklists and flowcharts, were developed to protect against reporting poorly designed research, and researchers should be aware of the available instruments and their appropriate use. With the popularity of synthetic reviews, meta-analyses, and systematic reviews, there is a greater need to assess risk of bias and study quality. This review highlights the most frequently used guidelines and checklists, risk-of-bias scales, and quality rating scales that can assist researchers with improving their research and its eventual publication.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Djulbegovic B.
        • Guyatt G.H.
        Progress in evidence-based medicine: A quarter century on.
        Lancet. 2017; 390: 415-423
        • Ogrinc G.
        • Mooney S.
        • Estrada C.
        • et al.
        The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: Explanation and elaboration.
        BMJ Qual Saf. 2008; 17: i13-i32
        • Von Elm E.
        • Altman D.G.
        • Egger M.
        • et al.
        The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies.
        PLoS Med. 2007; 4: e296
        • Altman D.G.
        • Simera I.
        A history of the evolution of guidelines for reporting medical research: The long road to the EQUATOR network.
        J R Soc Med. 2016; 109: 67-77
        • Bossuyt P.M.
        • Reitsma J.B.
        • Bruns D.E.
        • et al.
        The STARD statement for reporting studies of diagnostic accuracy: Explanation and elaboration.
        Ann Intern Med. 2003; 138: W1-W12
        • EQUATOR network
        (Published 2018. Accessed June 6, 2019.)
        • Pannucci C.J.
        • Wilkins E.G.
        Identifying and avoiding bias in research.
        Plast Reconstr Surg. 2010; 126: 619
        • Gerhard T.
        Bias: Considerations for research practice.
        Am J Health Syst Pharm. 2008; 65: 2159-2168
        • Simundić A.-M.
        Bias in research.
        Biochem Med (Zagreb). 2013; 23: 12-15
        • Gardenier J.
        • Resnik D.
        The misuse of statistics: Concepts, tools, and a research agenda.
        Account Res. 2002; 9: 65-74
        • Kim S.Y.
        • Park J.E.
        • Lee Y.J.
        • et al.
        Testing a tool for assessing the risk of bias for nonrandomized studies showed moderate reliability and promising validity.
        J Clin Epidemiol. 2013; 66: 408-414
        • Sterne J.A.
        • Hernán M.A.
        • Reeves B.C.
        • et al.
        ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions.
        BMJ. 2016; 355: i4919
        • Moher D.
        • Cook D.J.
        • Eastwood S.
        • Olkin I.
        • Rennie D.
        • Stroup D.F.
        Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement.
        Rev Esp Salud Pub. 2000; 74 ([in Spanish]): 597-602
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement.
        PLoS Med. 2009; 6e1000097
        • Stroup D.F.
        • Berlin J.A.
        • Morton S.C.
        • et al.
        Meta-analysis of observational studies in epidemiology: A proposal for reporting.
        JAMA. 2000; 283: 2008-2012
        • Begg C.
        • Cho M.
        • Eastwood S.
        • et al.
        Improving the quality of reporting of randomized controlled trials: The CONSORT statement.
        JAMA. 1996; 276: 637-639
        • Moher D.
        • Jones A.
        • Lepage L.
        • CONSORT Group (Consolidated Standards for Reporting of Trials)
        Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation.
        JAMA. 2001; 285: 1992-1995
        • Schulz K.F.
        • Altman D.G.
        • Moher D.
        CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials.
        BMC Med. 2010; 8: 18
        • Bossuyt P.M.
        • Reitsma J.B.
        • Bruns D.E.
        • et al.
        STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies.
        BMJ. 2015; 351: h5527
        • Chan A.-W.
        • Tetzlaff J.M.
        • Altman D.G.
        • et al.
        SPIRIT 2013 statement: Defining standard protocol items for clinical trials.
        Ann Intern Med. 2013; 158: 200-207
        • Hohmann E.
        • Brand J.C.
        • Rossi M.J.
        • et al.
        Expert opinion is necessary: Delphi Panel methodology facilitates a scientific approach to consensus.
        Arthroscopy. 2018; 34: 349-351
        • Hohmann E.
        • Cote M.P.
        • Brand J.C.
        Research pearls: Expert consensus based evidence using the Delphi method.
        Arthroscopy. 2018; 34: 3278-3282
        • Husereau D.
        • Drummond M.
        • Petrou S.
        • et al.
        Consolidated health economic evaluation reporting standards (CHEERS) statement.
        Cost Eff Resour Alloc. 2013; 11: 6
        • Davidoff F.
        • Batalden P.
        Toward stronger evidence on quality improvement. Draft publication guidelines: The beginning of a consensus project.
        BMJ Qual Saf. 2005; 14: 319-325
        • Davies L.
        • Batalden P.
        • Davidoff F.
        • Stevens D.
        • Ogrinc G.
        The SQUIRE guidelines: An evaluation from the field, 5 years post release.
        BMJ Qual Saf. 2015; 24: 769-775
        • SQUIRE
        (Published 2017. Accessed June 23, 2019.)
        • Terrace L.
        Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: The AGREE project.
        Qual Saf Health Care. 2003; 12: 18-23
        • Brouwers M.C.
        • Kho M.E.
        • Browman G.P.
        • et al.
        AGREE II: Advancing guideline development, reporting and evaluation in health care.
        CMAJ. 2010; 182: E839-E842
        • O’Brien B.C.
        • Harris I.B.
        • Beckman T.J.
        • Reed D.A.
        • Cook D.A.
        Standards for reporting qualitative research: A synthesis of recommendations.
        Acad Med. 2014; 89: 1245-1251
        • Gagnier J.J.
        • Kienle G.
        • Altman D.G.
        • Moher D.
        • Sox H.
        • Riley D.
        The CARE guidelines: Consensus-based clinical case reporting guideline development.
        J Med Case Rep. 2013; 7: 223
      1. Arthroscopy: Journal of Arthroscopic and Related Surgery.
        (Published 2019. Accessed June 25, 2019.)
        • Luchini C.
        • Stubbs B.
        • Solmi M.
        • Veronese N.
        Assessing the quality of studies in meta-analyses: Advantages and limitations of the Newcastle Ottawa Scale.
        World J Metaanal. 2017; 5: 80-84
        • Higgins J.P.
        • Altman D.G.
        • Gøtzsche P.C.
        • et al.
        The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.
        BMJ. 2011; 343: d5928
        • Whiting P.
        • Savović J.
        • Higgins J.P.
        • et al.
        ROBIS: A new tool to assess risk of bias in systematic reviews was developed.
        J Clin Epidemiol. 2016; 69: 225-234
        • Wells G.A.
        • Shea B.
        • O’Connell D.
        • et al.
        The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses.
        (Published 2001. Accessed June 5, 2019.)
        • Sim Y.
        • Horner N.S.
        • de Sa D.
        • Simunovic N.
        • Karlsson J.
        • Ayeni O.R.
        Reporting of non-hip score outcomes following femoroacetabular impingement surgery: A systematic review.
        J Hip Preserv Surg. 2015; 2: 224-241
        • Maher C.G.
        • Sherrington C.
        • Herbert R.D.
        • Moseley A.M.
        • Elkins M.
        Reliability of the PEDro scale for rating quality of randomized controlled trials.
        Phys Ther. 2003; 83: 713-721
        • Schünemann H.
        The GRADE handbook.
        Cochrane Collaboration, 2013
        • Coleman B.
        • Khan K.
        • Maffulli N.
        • Cook J.
        • Wark J.
        Studies of surgical outcome after patellar tendinopathy: Clinical significance of methodological deficiencies and guidelines for future studies.
        Scand J Med Sci Sports. 2000; 10: 2-11
        • Tallon C.
        • Coleman B.D.
        • Khan K.M.
        • Maffulli N.
        Outcome of surgery for chronic Achilles tendinopathy: A critical review.
        Am J Sports Med. 2001; 29: 315-320
        • Mokkink L.B.
        • Terwee C.B.
        • Patrick D.L.
        • et al.
        The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study.
        Qual Life Res. 2010; 19: 539-549
        • Slim K.
        • Nini E.
        • Forestier D.
        • Kwiatkowski F.
        • Panis Y.
        • Chipponi J.
        Methodological index for non-randomized studies (MINORS): Development and validation of a new instrument.
        ANZ J Surg. 2003; 73: 712-716
        • Downs S.H.
        • Black N.
        The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions.
        J Epidemiol Community Health. 1998; 52: 377-384
        • Jadad A.R.
        • Moore R.A.
        • Carroll D.
        • et al.
        Assessing the quality of reports of randomized clinical trials: Is blinding necessary?.
        Control Clin Trials. 1996; 17: 1-12
        • PEDro scale
        (Published 1999. Accessed June 27, 2019.)
        • Shea B.J.
        • Reeves B.C.
        • Wells G.
        • et al.
        AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
        BMJ. 2017; 358: j4008
        • Atkins D.
        • Best D.
        • Briss P.A.
        • et al.
        Grading quality of evidence and strength of recommendations.
        BMJ. 2004; 328: 1490
        • McCormick F.
        • Cvetanovich G.L.
        • Kim J.M.
        • et al.
        An assessment of the quality of rotator cuff randomized controlled trials: Utilizing the Jadad score and CONSORT criteria.
        J Shoulder Elbow Surg. 2013; 22: 1180-1185
        • Sorensen A.A.
        • Wojahn R.D.
        • Manske M.C.
        • Calfee R.P.
        Using the STROBE statement to assess reporting of observational trials in hand surgery.
        J Hand Surg. 2013; 38: 1584
        • Tunis A.S.
        • McInnes M.D.
        • Hanna R.
        • Esmail K.
        Association of study quality with completeness of reporting: Have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement?.
        Radiology. 2013; 269: 413-426
        • Kunkel S.
        • Sabatino M.
        • Moschetti W.
        • Jevsevar D.
        Systematic reviews and meta-analyses in the orthopedic literature: Assessment of the current state of quality and proposal of a new rating strategy.
        Clin Res Orthop. 2018; 1: 1-7
        • Plint A.C.
        • Moher D.
        • Morrison A.
        • et al.
        Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review.
        Med J Aust. 2006; 185: 263
        • Smidt N.
        • Rutjes A.
        • Van der Windt D.
        • et al.
        The quality of diagnostic accuracy studies since the STARD statement: Has it improved?.
        Neurology. 2006; 67: 792-797
        • Cui Q.
        • Tian J.
        • Song X.
        • Yang K.
        Does the CONSORT checklist for abstracts improve the quality of reports of randomized controlled trials on clinical pathways?.
        J Eval Clin Pract. 2014; 20: 827-833
      2. Altman DG, I. Simera I. Using reporting guidelines effectively to ensure good reporting of health research [Chapter 4]. In: Moher D, Altman DG, Schulz KF, Simera I, Wager E, eds. Guidelines for reporting health research: A user's manual. Ed 1. Hoboken, NJ: John Wiley and Sons, Ltd.; 2014.