ABSTRACT
Purpose
To evaluate the robustness of sports medicine and arthroscopy related randomized controlled
trials (RCTs) reporting non-significant results by calculating the reverse fragility
index (RFI) and reverse fragility quotient (RFQ).
Methods
All sports medicine and arthroscopic related RCTs from January 1, 2010 through August
3, 2021 were identified. Randomized-controlled trials comparing dichotomous variables
with a reported p-value ≥ 0.05 were included. Study characteristics, such as publication
year and sample size, as well as loss to follow-up and number of outcome events were
recorded. The RFI at a threshold of P < 0.05 and respective RFQ were calculated for
each study. Coefficients of determination were calculated to determine the relationships
between RFI and the number of outcome events, sample size, and number of patients
lost to follow-up. The number of RCTs in which the loss to follow-up was greater than
the RFI was determined.
Results
Fifty-four studies and 4,638 patients were included in this analysis. The mean sample
size and loss to follow-up were 85.9 patients and 12.5 patients, respectively. The
mean RFI was 3.7, signifying that a change of 3.7 events in one arm was needed to
flip the results of the study from non-significant to significant (p<0.05). Of the
54 studies investigated, 33 (61%) had a loss to follow-up greater than their calculated
RFI. The mean RFQ was 0.05. A significant correlation between RFI with sample size
(R2=0.10, p=0.02) and the total number of observed events (R2=0.13, p<0.01) was found. No significant correlation existed between RFI and loss
to follow-up in the lesser arm (R2=0.01, p=0.41).
Conclusion
The RFI and RFQ are statistical tools that allow the fragility of studies reporting
non-significant results to be appraised. Utilizing this methodology, we found that
the majority of sports medicine and arthroscopy related RCTs reporting non-significant
results are fragile.
Clinical Relevance
RFI and RFQ serve as tools that can be used to assess the validity of RCT results
and provide additional context for appropriate conclusions.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to ArthroscopyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
REFERENCES
- Statistical Fragility and the Role of P Values in the Sports Medicine Literature.J Am Acad Orthop Surg. 2019; 27: e324-e329
- Making Sense of Statistics in Clinical Trial Reports: Part 1 of a 4-Part Series on Statistics for Clinical Trials.J Am Coll Cardiol. 2015; 66: 2536-2549
- Contradicted and initially stronger effects in highly cited clinical research.JAMA. 2005; 294: 218-228
- The fragility and reverse fragility indices of proximal humerus fracture randomized controlled trials: a systematic review.Eur J Trauma Emerg Surg. 2022 Dec; 48 (Epub 2021 May 31. PMID: 34056677): 4545-4552https://doi.org/10.1007/s00068-021-01684-2
- The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index.J Clin Epidemiol. 2014; 67: 622-628
- The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey.Spine J. 2015; 15: 2188-2197
- The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 "negative" trials.N Engl J Med. 1978; 299: 690-694
- Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review.BMJ. 2012; 344e2809
- How robust are clinical trials in heart failure?.Eur Heart J. 2017; 38: 338-345
- The Fragility Index in Randomized Clinical Trials as a Means of Optimizing Patient Care.JAMA Surg. 2019; 154: 74-79
- The fragility of statistically significant findings in randomised controlled anaesthesiology trials: systematic review of the medical literature.Br J Anaesth. 2018; 120: 935-941
- The Fragility Index in a Cohort of Pediatric Randomized Controlled Trials.J Clin Med. 2017; 6
- The Results of Randomized Controlled Trials in Emergency Medicine Are Frequently Fragile.Ann Emerg Med. 2019; 73: 565-576
- The Robustness of Trials That Guide Evidence-Based Orthopaedic Surgery.J Bone Joint Surg Am. 2018; 100: e85
- The Fragility Index in a Cohort of HIV/AIDS Randomized Controlled Trials.J Gen Intern Med. 2019; 34: 1236-1243
- The Statistical Fragility of Single-Bundle vs Double-Bundle Autografts for ACL Reconstruction: A Systematic Review of Comparative Studies.Orthop J Sports Med. 2021; 923259671211064626
- The Fragility of Statistically Significant Findings From Randomized Trials in Sports Surgery: A Systematic Survey.Am J Sports Med. 2017; 45: 2164-2170
- The Statistical Fragility of Platelet-Rich Plasma in Rotator Cuff Surgery: A Systematic Review and Meta-analysis.Am J Sports Med. 2021; 49: 3437-3442
- The Statistical Fragility of Hamstring Versus Patellar Tendon Autografts for Anterior Cruciate Ligament Reconstruction: A Systematic Review of Comparative Studies.Am J Sports Med. 2021; 49: 2827-2833
- Application of the Reverse Fragility Index to Statistically Nonsignificant Randomized Clinical Trial Results.JAMA Netw Open. 2020; 3e2012469
- Does Sample Size Matter When Interpreting the Fragility Index?.Crit Care Med. 2016; 44: e1142-e1143
- Effect of two rehabilitation protocols on range of motion and healing rates after arthroscopic rotator cuff repair: aggressive versus limited early passive exercises.Arthroscopy. 2012; 28: 34-42
- Low Level of Evidence and Methodologic Quality of Clinical Outcome Studies on Cartilage Repair of the Ankle.Arthroscopy. 2016; 32: 214-222 e211
- Methodologic quality of knee articular cartilage studies.Arthroscopy. 2013; 29: 1243-1252 e1245
- The undue influence of significant p-values on the perceived importance of study results.Acta Orthop. 2005; 76: 291-295
- Application of the Consolidated Standards of Reporting Trials (CONSORT) in the Fracture Care Literature.J Bone Joint Surg Am. 2002; 84: 485-489
- Cochrane in CORR ((R)): Double-bundle Versus Single-bundle Reconstruction for Anterior Cruciate Ligament Rupture in Adults (Review).Clin Orthop Relat Res. 2016; 474: 1099-1101
- Cochrane in CORR(R): Surgical versus conservative interventions for treating fractures of the middle third of the clavicle.Clin Orthop Relat Res. 2014; 472: 2579-2585
- The etiology of femoroacetabular impingement: what we know and what we don't.Sports Health. 2014; 6: 157-161
- The Fragility Index of Hip Arthroscopy Randomized Controlled Trials: A Systematic Survey.Arthroscopy. 2021; 37: 1983-1989
- Absence of evidence is not evidence of absence.BMJ. 2004; 328: 476-477
- Statistical Significance Versus Clinical Importance of Observed Effect Sizes: What Do P Values and Confidence Intervals Really Represent?.Anesth Analg. 2018; 126: 1068-1072
- Are patient-reported outcome measures in orthopaedics easily read by patients?.Clin Orthop Relat Res. 2016; 474: 246-255
Article info
Publication history
Accepted:
February 16,
2023
Received in revised form:
February 14,
2023
Received:
July 19,
2022
Publication stage
In Press Journal Pre-ProofFootnotes
Ethical review committee statement: IRB approval was not required for this study.
This manuscript has not been published previously, is not under consideration for publication elsewhere, and has not been submitted to or presented as an abstract at a professional meeting. The manuscript has been read by all active authors and all endorse that the manuscript represents honest work. We have no conflicts of interest to disclose.
Identification
Copyright
© 2023 Published by Elsevier on behalf of the Arthroscopy Association of North America
