Purpose
To determine what subspecialties have applied machine learning (ML) to predict clinically
significant outcomes (CSOs) within orthopaedic surgery and to determine whether the
performance of these models was acceptable through assessing discrimination and other
ML metrics where reported.
Methods
The PubMed, EMBASE, and Cochrane Central Register of Controlled Trials databases were
queried for articles that used ML to predict achievement of the minimal clinically
important difference (MCID), patient acceptable symptomatic state (PASS), or substantial
clinical benefit (SCB) after orthopaedic surgical procedures. Data pertaining to demographic
characteristics, subspecialty, specific ML algorithms, and algorithm performance were
analyzed.
Results
Eighteen articles met the inclusion criteria. Seventeen studies developed novel algorithms,
whereas one study externally validated an established algorithm. All studies used
ML to predict MCID achievement, whereas 3 (16.7%) predicted SCB achievement and none
predicted PASS achievement. Of the studies, 7 (38.9%) concerned outcomes after spine
surgery; 6 (33.3%), after sports medicine surgery; 3 (16.7%), after total joint arthroplasty
(TJA); and 2 (11.1%), after shoulder arthroplasty. No studies were found regarding
trauma, hand, elbow, pediatric, or foot and ankle surgery. In spine surgery, concordance
statistics (C-statistics) ranged from 0.65 to 0.92; in hip arthroscopy, 0.51 to 0.94;
in TJA, 0.63 to 0.89; and in shoulder arthroplasty, 0.70 to 0.95. Most studies reported
C-statistics at the upper end of these ranges, although populations were heterogeneous.
Conclusions
Currently available ML algorithms can discriminate the propensity to achieve CSOs
using the MCID after spine, TJA, sports medicine, and shoulder surgery with a fair
to good performance as evidenced by C-statistics ranging from 0.6 to 0.95 in most
analyses. Less evidence is available on the ability of ML to predict achievement of
SCB, and no evidence is available for achievement of the PASS. Such algorithms may
augment shared decision-making practices and allow clinicians to provide more appropriate
patient expectations using individualized risk assessments. However, these studies
remain limited by variable reporting of performance metrics, CSO quantification methods,
and adherence to predictive modeling guidelines, as well as limited external validation.
Level of Evidence
Level III, systematic review of Level III studies.
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to ArthroscopyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Response to “Regarding ‘Editorial Commentary: Artificial intelligence in sports medicine diagnosis needs to improve’”.Arthroscopy. 2021; 37: 1367-1368
- Clinical and research medical applications of artificial intelligence.Arthroscopy. 2021; 37: 1694-1697
- Machine learning and artificial intelligence: Definitions, applications, and future directions.Curr Rev Musculoskelet Med. 2020; 13: 69-76
- Artificial intelligence for the orthopaedic surgeon: An overview of potential benefits, limitations, and clinical applications.J Am Acad Orthop Surg. 2021; 29: 235-243
- Artificial intelligence and orthopaedics: An introduction for clinicians.J Bone Joint Surg Am. 2020; 102: 830-840
- Can machine learning algorithms predict which patients will achieve minimally clinically important differences from total joint arthroplasty?.Clin Orthop Relat Res. 2019; 477: 1267-1279
- Application of machine learning algorithms to predict clinically meaningful improvement after arthroscopic anterior cruciate ligament reconstruction.Orthop J Sports Med. 2021; 9 (23259671211046575)
- Machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes.J Bone Joint Surg Am. 2021; 103: 1055-1062
- Development of supervised machine learning algorithms for prediction of satisfaction at 2 years following total shoulder arthroplasty.J Shoulder Elbow Surg. 2021; 30: e290-e299
- Development of machine learning algorithms to predict clinically meaningful improvement for the patient-reported health state after total hip arthroplasty.J Arthroplasty. 2020; 35: 2119-2123
- Development and internal validation of supervised machine learning algorithms for predicting clinically significant functional improvement in a mixed population of primary hip arthroscopy patients.Arthroscopy. 2021; 37: 1488-1497
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.PLoS Med. 2009; 6e1000097
- The Brier score does not evaluate the clinical utility of diagnostic tests or prediction models.Diagn Progn Res. 2017; 1: 19
- Use and misuse of the receiver operating characteristic curve in risk prediction.Circulation. 2007; 115: 928-935
- Comparing different supervised machine learning algorithms for disease prediction.BMC Med Inform Decis Mak. 2019; 19: 281
- Towards better clinical prediction models: seven steps for development and an ABCD for validation.Eur Heart J. 2014; 35: 1925-1931
- Assessing the performance of prediction models: a framework for traditional and novel measures.Epidemiology. 2010; 21: 128-138
- The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models.Diagn Progn Res. 2018; 2: 7
- Methodological index for non-randomized studies (MINORS): Development and validation of a new instrument.ANZ J Surg. 2003; 73: 712-716
- External validation of a prediction model for pain and functional outcome after elective lumbar spinal fusion.Eur Spine J. 2020; 29: 374-383
- Development and validation of a prediction model for pain and functional outcomes after lumbar spine surgery.JAMA Surg. 2018; 153: 634-642
- Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar discectomy: feasibility of center-specific modeling.Spine J. 2019; 19: 853-861
- Machine learning-based preoperative predictive analytics for lumbar spinal stenosis.Neurosurg Focus. 2019; 46: E5
- Development of prediction models for clinically meaningful improvement in PROMIS scores after lumbar decompression.Spine J. 2020; 21: 397-404
- Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy.PLoS One. 2019; 14e0215133
- Applied machine learning for spine surgeons: Predicting outcome for patients undergoing treatment for lumbar disc herniation using PRO data.Global Spine J. 2020; (2192568220967643)
- Machine learning algorithms predict clinically significant improvements in satisfaction after hip arthroscopy.Arthroscopy. 2021; 37: 1143-1151
- Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery.Am J Sports Med. 2020; 48: 415-423
- Radiographic indices are not predictive of clinical outcomes among 1735 patients indicated for hip arthroscopic surgery: A machine learning analysis.Am J Sports Med. 2020; 48: 2910-2918
- Association between preoperative mental health and clinically meaningful outcomes after osteochondral allograft for cartilage defects of the knee: A machine learning analysis.Am J Sports Med. 2021; 49: 948-957
- Can machine learning methods produce accurate and easy-to-use preoperative prediction models of one-year improvements in pain and functioning after knee arthroplasty?.J Arthroplasty. 2021; 36: 112-117.e6
- Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set.J Shoulder Elbow Surg. 2021; 30: e225-e236
- What Is the accuracy of three different machine learning techniques to predict clinical outcomes after shoulder arthroplasty?.Clin Orthop Relat Res. 2020; 478: 2351-2363
- Applied predictive modeling.Springer, New York2013
- Large heterogeneity among minimal clinically important differences for hip arthroscopy outcomes: A systematic review of reporting trends and quantification methods.Arthroscopy. 2021; 37: 1028-1037.e6
- Editorial: Opposites attract at CORR-machine learning and qualitative research.Clin Orthop Relat Res. 2020; 478: 2193-2196
Article Info
Publication History
Published online: December 26, 2021
Accepted:
December 20,
2021
Received:
September 27,
2021
See commentary on page 2106Footnotes
The authors report the following potential conflicts of interest or sources of funding: K.N.K. is on the editorial or governing board of Arthroscopy. B.U.N. owns stock or stock options in BICMD, outside the submitted work. J.C. is a paid consultant for ConMed Linvatec, Arthrex, Ossur, and Smith & Nephew, outside the submitted work, and is a board or committee member of American Orthopaedic Society for Sports Medicine, AANA, and International Society of Arthroscopy, Knee Surgery and Orthopaedic Sports Medicine. S.J.N. receives research support from Allosource, Arthrex, Athletico, DJ Orthopaedics, Linvatec, Miomed, and Stryker, outside the submitted work; receives intellectual property royalties from Ossur and Stryker, outside the submitted work; is a paid consultant for Stryker, outside the submitted work; receives publishing royalties and financial or material support from Springer, outside the submitted work; and is on the editorial or governing board of American Orthopaedic Associations, American Orthopaedic Society for Sports Medicine, and AANA. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
Identification
Copyright
© 2021 by the Arthroscopy Association of North America
