Research Article|Articles in Press

The Majority of Sports Medicine and Arthroscopy Related Randomized-Controlled Trials Reporting Non-Significant Results are Statistically Fragile



      To evaluate the robustness of sports medicine and arthroscopy related randomized controlled trials (RCTs) reporting non-significant results by calculating the reverse fragility index (RFI) and reverse fragility quotient (RFQ).


      All sports medicine and arthroscopic related RCTs from January 1, 2010 through August 3, 2021 were identified. Randomized-controlled trials comparing dichotomous variables with a reported p-value ≥ 0.05 were included. Study characteristics, such as publication year and sample size, as well as loss to follow-up and number of outcome events were recorded. The RFI at a threshold of P < 0.05 and respective RFQ were calculated for each study. Coefficients of determination were calculated to determine the relationships between RFI and the number of outcome events, sample size, and number of patients lost to follow-up. The number of RCTs in which the loss to follow-up was greater than the RFI was determined.


      Fifty-four studies and 4,638 patients were included in this analysis. The mean sample size and loss to follow-up were 85.9 patients and 12.5 patients, respectively. The mean RFI was 3.7, signifying that a change of 3.7 events in one arm was needed to flip the results of the study from non-significant to significant (p<0.05). Of the 54 studies investigated, 33 (61%) had a loss to follow-up greater than their calculated RFI. The mean RFQ was 0.05. A significant correlation between RFI with sample size (R2=0.10, p=0.02) and the total number of observed events (R2=0.13, p<0.01) was found. No significant correlation existed between RFI and loss to follow-up in the lesser arm (R2=0.01, p=0.41).


      The RFI and RFQ are statistical tools that allow the fragility of studies reporting non-significant results to be appraised. Utilizing this methodology, we found that the majority of sports medicine and arthroscopy related RCTs reporting non-significant results are fragile.

      Clinical Relevance

      RFI and RFQ serve as tools that can be used to assess the validity of RCT results and provide additional context for appropriate conclusions.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Parisien RL
        • Trofa DP
        • Dashe J
        • et al.
        Statistical Fragility and the Role of P Values in the Sports Medicine Literature.
        J Am Acad Orthop Surg. 2019; 27: e324-e329
        • Pocock SJ
        • McMurray JJ
        • Collier TJ
        Making Sense of Statistics in Clinical Trial Reports: Part 1 of a 4-Part Series on Statistics for Clinical Trials.
        J Am Coll Cardiol. 2015; 66: 2536-2549
        • Ioannidis JP
        Contradicted and initially stronger effects in highly cited clinical research.
        JAMA. 2005; 294: 218-228
        • Kyriakides PW
        • Schultz BJ
        • Egol K
        • Leucht P
        The fragility and reverse fragility indices of proximal humerus fracture randomized controlled trials: a systematic review.
        Eur J Trauma Emerg Surg. 2022 Dec; 48 (Epub 2021 May 31. PMID: 34056677): 4545-4552
        • Walsh M
        • Srinathan SK
        • McAuley DF
        • et al.
        The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index.
        J Clin Epidemiol. 2014; 67: 622-628
        • Evaniew N
        • Files C
        • Smith C
        • et al.
        The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey.
        Spine J. 2015; 15: 2188-2197
        • Freiman JA
        • Chalmers TC
        • Smith Jr., H
        • Kuebler RR
        The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials.
        N Engl J Med. 1978; 299: 690-694
        • Akl EA
        • Briel M
        • You JJ
        • et al.
        Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review.
        BMJ. 2012; 344e2809
        • Docherty KF
        • Campbell RT
        • Jhund PS
        • Petrie MC
        • McMurray JJV
        How robust are clinical trials in heart failure?.
        Eur Heart J. 2017; 38: 338-345
        • Tignanelli CJ
        • Napolitano LM
        The Fragility Index in Randomized Clinical Trials as a Means of Optimizing Patient Care.
        JAMA Surg. 2019; 154: 74-79
        • Mazzinari G
        • Ball L
        • Serpa Neto A
        • et al.
        The fragility of statistically significant findings in randomised controlled anaesthesiology trials: systematic review of the medical literature.
        Br J Anaesth. 2018; 120: 935-941
        • Matics TJ
        • Khan N
        • Jani P
        • Kane JM
        The Fragility Index in a Cohort of Pediatric Randomized Controlled Trials.
        J Clin Med. 2017; 6
        • Brown J
        • Lane A
        • Cooper C
        • Vassar M
        The Results of Randomized Controlled Trials in Emergency Medicine Are Frequently Fragile.
        Ann Emerg Med. 2019; 73: 565-576
        • Checketts JX
        • Scott JT
        • Meyer C
        • Horn J
        • Jones J
        • Vassar M
        The Robustness of Trials That Guide Evidence-Based Orthopaedic Surgery.
        J Bone Joint Surg Am. 2018; 100: e85
        • Wayant C
        • Meyer C
        • Gupton R
        • Som M
        • Baker D
        • Vassar M
        The Fragility Index in a Cohort of HIV/AIDS Randomized Controlled Trials.
        J Gen Intern Med. 2019; 34: 1236-1243
        • Ehlers CB
        • Curley AJ
        • Fackler NP
        • et al.
        The Statistical Fragility of Single-Bundle vs Double-Bundle Autografts for ACL Reconstruction: A Systematic Review of Comparative Studies.
        Orthop J Sports Med. 2021; 923259671211064626
        • Khan M
        • Evaniew N
        • Gichuru M
        • et al.
        The Fragility of Statistically Significant Findings From Randomized Trials in Sports Surgery: A Systematic Survey.
        Am J Sports Med. 2017; 45: 2164-2170
        • Parisien RL
        • Ehlers C
        • Cusano A
        • Tornetta 3rd, P
        • Li X
        • Wang D
        The Statistical Fragility of Platelet-Rich Plasma in Rotator Cuff Surgery: A Systematic Review and Meta-analysis.
        Am J Sports Med. 2021; 49: 3437-3442
        • Ehlers CB
        • Curley AJ
        • Fackler NP
        • Minhas A
        • Chang ES
        The Statistical Fragility of Hamstring Versus Patellar Tendon Autografts for Anterior Cruciate Ligament Reconstruction: A Systematic Review of Comparative Studies.
        Am J Sports Med. 2021; 49: 2827-2833
        • Khan MS
        • Fonarow GC
        • Friede T
        • et al.
        Application of the Reverse Fragility Index to Statistically Nonsignificant Randomized Clinical Trial Results.
        JAMA Netw Open. 2020; 3e2012469
        • Ahmed W
        • Fowler RA
        • McCredie VA
        Does Sample Size Matter When Interpreting the Fragility Index?.
        Crit Care Med. 2016; 44: e1142-e1143
        • Lee BG
        • Cho NS
        • Rhee YG
        Effect of two rehabilitation protocols on range of motion and healing rates after arthroscopic rotator cuff repair: aggressive versus limited early passive exercises.
        Arthroscopy. 2012; 28: 34-42
        • Pinski JM
        • Boakye LA
        • Murawski CD
        • Hannon CP
        • Ross KA
        • Kennedy JG
        Low Level of Evidence and Methodologic Quality of Clinical Outcome Studies on Cartilage Repair of the Ankle.
        Arthroscopy. 2016; 32: 214-222 e211
        • Harris JD
        • Erickson BJ
        • Abrams GD
        • et al.
        Methodologic quality of knee articular cartilage studies.
        Arthroscopy. 2013; 29: 1243-1252 e1245
        • Bhandari M
        • Montori VM
        • Schemitsch EH
        The undue influence of significant p-values on the perceived importance of study results.
        Acta Orthop. 2005; 76: 291-295
        • Bhandari M
        • Guyatt GH
        • Lochner H
        • Sprague S
        • Tornetta 3rd, P
        Application of the Consolidated Standards of Reporting Trials (CONSORT) in the Fracture Care Literature.
        J Bone Joint Surg Am. 2002; 84: 485-489
        • Mundi R
        • Bhandari M
        Cochrane in CORR ((R)): Double-bundle Versus Single-bundle Reconstruction for Anterior Cruciate Ligament Rupture in Adults (Review).
        Clin Orthop Relat Res. 2016; 474: 1099-1101
        • Evaniew N
        • Simunovic N
        • McKee MD
        • Schemitsch E
        Cochrane in CORR(R): Surgical versus conservative interventions for treating fractures of the middle third of the clavicle.
        Clin Orthop Relat Res. 2014; 472: 2579-2585
        • Chaudhry H
        • Ayeni OR
        The etiology of femoroacetabular impingement: what we know and what we don't.
        Sports Health. 2014; 6: 157-161
        • Maldonado DR
        • Go CC
        • Huang BH
        • Domb BG
        The Fragility Index of Hip Arthroscopy Randomized Controlled Trials: A Systematic Survey.
        Arthroscopy. 2021; 37: 1983-1989
        • Alderson P
        Absence of evidence is not evidence of absence.
        BMJ. 2004; 328: 476-477
        • Schober P
        • Bossers SM
        • Schwarte LA
        Statistical Significance Versus Clinical Importance of Observed Effect Sizes: What Do P Values and Confidence Intervals Really Represent?.
        Anesth Analg. 2018; 126: 1068-1072
        • El-Daly I
        • Ibraheim H
        • Rajakulendran K
        • Culpan P
        • Bates P
        Are patient-reported outcome measures in orthopaedics easily read by patients?.
        Clin Orthop Relat Res. 2016; 474: 246-255