Advertisement

Meaningless Applications and Misguided Methodologies in Artificial Intelligence–Related Orthopaedic Research Propagates Hype Over Hope

      Abstract

      There exists great hope and hype in the literature surrounding applications of artificial intelligence (AI) to orthopaedic surgery. Between 2018 and 2021, a total of 178 AI-related articles were published in orthopaedics. However, for every 2 original research papers that apply AI to orthopaedics, a commentary or review is published (30.3%). AI-related research in orthopaedics frequently fails to provide use cases that offer the uninitiated an opportunity to appraise the importance of AI by studying meaningful questions, evaluating unknown hypotheses, or analyzing quality data. The hype perpetuates a feed-forward cycle that relegates AI to a meaningless buzzword by rewarding those with nascent understanding and rudimentary technical knowhow into committing several basic errors: (1) inappropriately conflating vernacular (“AI/machine learning”), (2) repackaging registry data, (3) prematurely releasing internally validated algorithms, (4) overstating the “black box phenomenon” by failing to provide weighted analysis, (5) claiming to evaluate AI rather than the data itself, and (6) withholding full model architecture code. Relevant AI-specific guidelines are forthcoming, but forced application of the original Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines designed for regression analyses is irrelevant and misleading. To safeguard meaningful use, AI-related research efforts in orthopaedics should be (1) directed toward administrative support over clinical evaluation and management, (2) require the use of the advanced model, and (3) answer a question that was previously unknown, unanswered, or unquantifiable.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Ramkumar P.N.
        • Kunze K.N.
        • Haeberle H.S.
        • et al.
        Clinical and research medical applications of artificial intelligence.
        Arthroscopy. 2021; 37: 1694-1697
        • Makhni E.C.
        • Makhni S.
        • Ramkumar P.N.
        Artificial intelligence for the orthopaedic surgeon: An overview of potential benefits, limitations, and clinical applications.
        J Am Acad Orthop Surg. 2021; 29: 235-243
        • Luu B.
        • Wright A.
        • Haeberle H.
        • et al.
        Machine learning outperforms logistic regression analysis to predict next season NHL player injury: An analysis of 2,322 players from 2007-2017.
        Orthop J Sport Med. 2020; 82325967120953404
        • Karnuta J.
        • Luu B.
        • Haeberle H.
        • et al.
        Machine learning outperforms regression analysis to predict next season MLB player injury: Epidemiology and validation of 13,982 player-years from performance and injury profile trends between 2000-17.
        Orthop J Sport Med. 2020; 82325967120963046
        • Ramkumar P.N.
        • Karnuta J.M.
        • Haeberle H.S.
        • et al.
        Radiographic indices are not predictive of clinical outcome among 1,735 patients indicated for hip arthroscopy: A machine learning analysis.
        Am J Sports Med. 2020; 48: 2910-2918
        • Fritz B.
        • Marbach G.
        • Civardi F.
        • Fucentese S.F.
        • Pfirrmann C.W.A.
        Deep convolutional neural network-based detection of meniscus tears: Comparison with radiologists and surgery as standard of reference.
        Skeletal Radiol. 2020; 49: 1207-1217
        • Chang P.D.
        • Wong T.T.
        • Rasiej M.J.
        Deep learning for detection of complete anterior cruciate ligament tear.
        J Digit Imaging. 2019; 32: 980-986
        • Karnuta J.M.
        • Haeberle H.S.
        • Luu B.C.
        • et al.
        Artificial intelligence to identify arthroplasty implants from radiographs of the hip.
        J Arthroplasty. 2021; 36: S290-S294
        • Karnuta J.M.
        • Luu B.C.
        • Roth A.L.
        • et al.
        Artificial intelligence to identify arthroplasty implants from radiographs of the knee.
        J Arthroplasty. 2021; 36: 935-940
        • Borjali A.
        • Chen A.F.
        • Muratoglu O.K.
        • Morid M.A.
        • Varadarajan K.M.
        Detecting total hip replacement prosthesis design on plain radiographs using deep convolutional neural network.
        J Orthop Res. 2020; 38: 1465-1471
        • Murphy M.
        • Killen C.
        • Burnham R.
        • Sarvari F.
        • Wu K.
        • Brown N.
        Artificial intelligence accurately identifies total hip arthroplasty implants: a tool for revision surgery.
        Hip Int. 2021 Jan 8; 1120700020987526
        • Nwachukwu B.U.
        • Beck E.C.
        • Lee E.K.
        • et al.
        Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery.
        Am J Sports Med. 2020; 48: 415-423
        • Karnuta J.M.
        • Churchill J.L.
        • Haeberle H.S.
        • et al.
        The value of artificial neural networks for predicting length of stay, discharge disposition and inpatient costs after anatomic and reverse shoulder arthroplasty.
        J Shoulder Elbow Surg. 2020; 29: 2385-2394
        • Ramkumar P.N.
        • Haeberle H.S.
        • Ramanathan D.
        • et al.
        Remote patient monitoring using mobile health for total knee arthroplasty: Validation of a wearable and machine learning–based surveillance platform.
        J Arthroplasty. 2019; 34: 2253-2259
        • Ramkumar P.N.
        • Haeberle H.S.
        • Bloomfield M.R.
        • et al.
        Artificial intelligence and arthroplasty at a single institution: Real-world applications of machine learning to big data, value-based care, mobile health, and remote patient monitoring.
        J Arthroplasty. 2019; 34: 2204-2209
      1. Kim JS, Vivas A, Arvind V, et al. Can natural language processing and artificial intelligence automate the generation of billing codes from operative note dictations [published online February 28, 2022]? Global Spine J. https://doi.org/10.1177/21925682211062831

        • Overhage J.M.
        • McCallie Jr., D.
        Physician time spent using the electronic health record during outpatient encounters: A descriptive study.
        Ann Intern Med. 2020; 172: 169-174
        • Navarro S.M.
        • Wang E.Y.
        • Haeberle H.S.
        • et al.
        Machine learning and primary total knee arthroplasty: Patient forecasting for a patient-specific payment model.
        J Arthroplasty. 2018; 33: 3617-3623
        • Ramkumar P.N.
        • Karnuta J.M.
        • Navarro S.M.
        • et al.
        Preoperative prediction of value metrics and a patient-specific payment model for primary total hip arthroplasty: Development and validation of a deep learning model.
        J Arthroplasty. 2019; 34: 2228-2234.e1
        • Choudhury A.
        • Perumalla S.
        Using machine learning to minimize delays caused by prior authorization: A brief report.
        Cogent Engineering. 2021; 8: 1
        • Polce E.M.
        • Kunze K.N.
        • Dooley M.S.
        • Piuzzi N.S.
        • Boettner F.
        • Sculco P.K.
        Efficacy and applications of artificial intelligence and machine learning analyses in total joint arthroplasty: A call for improved reporting.
        J Bone Joint Surg Am. 2022; 104: 821-832
      2. Rubinger L, Gazendam A, Ekhtiari S, Bhandari M. Machine learning and artificial intelligence in research and healthcare [published online February 1, 2022]. Injury. https://doi.org/10.1016/j.injury.2022.01.046

      3. Kunze KN, Polce EM, Patel A, Courtney PM, Sporer SM, Levine BR. Machine learning algorithms predict within one size of the final implant ultimately used in total knee arthroplasty with good-to-excellent accuracy [published online January 13, 2022]. Knee Surg Sports Traumatol Arthrosc. https://doi.org/10.1007/s00167-022-06866-y

        • Polce E.M.
        • Kunze K.N.
        • Paul K.M.
        • Levine B.R.
        Machine learning predicts femoral and tibial implant size mismatch for total knee arthroplasty.
        Arthroplast Today. 2021; 8: 268-277.e2
      4. Kunze KN, Krivicich LM, Clapp IM, et al. Machine learning algorithms predict achievement of clinically significant outcomes after orthopaedic surgery: A systematic review [published online December 27, 2021]. Arthroscopy. https://doi.org/10.1016/j.arthro.2021.12.030

        • Kunze K.N.
        • Polce E.M.
        • Alter T.D.
        • Nho S.J.
        Machine learning algorithms predict prolonged opioid use in opioid-naïve primary hip arthroscopy patients.
        J Am Acad Orthop Surg Glob Res Rev. 2021; 5: e21.00093-e21.00098
        • Kunze K.N.
        • Polce E.M.
        • Clapp I.
        • Nwachukwu B.U.
        • Chahla J.
        • Nho S.J.
        machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes.
        J Bone Joint Surg Am. 2021; 103: 1055-1062
        • Kunze K.N.
        • Polce E.M.
        • Rasio J.
        • Nho S.J.
        Machine learning algorithms predict clinically significant improvements in satisfaction after hip arthroscopy.
        Arthroscopy. 2021; 37: 1143-1151
        • González-Esteban Y.
        • Patrici Calvo E.
        Ethically governing artificial intelligence in the field of scientific research and innovation.
        Heliyon. 2022; 8e08946
        • Pua Y.H.
        • Kang H.
        • Thumboo J.
        • et al.
        Machine learning methods are comparable to logistic regression techniques in predicting severe walking limitation following total knee arthroplasty.
        Knee Surg Sports Traumatol Arthrosc. 2020; 28: 3207-3216
        • Ramkumar P.N.
        • Karnuta J.M.
        • Haeberle H.S.
        • Rodeo S.A.
        • Nwachukwu B.U.
        • Williams 3rd, R.J.
        Effect of preoperative imaging and patient factors on clinically meaningful outcomes and quality of life after osteochondral allograft transplantation: A machine learning analysis of cartilage defects of the knee.
        Am J Sports Med. 2021; 49: 2177-2186
        • Ramkumar P.N.
        • Karnuta J.M.
        • Haeberle H.S.
        • et al.
        Association between preoperative mental health and clinically meaningful outcomes after osteochondral allograft for cartilage defects of the knee: A machine learning analysis.
        Am J Sports Med. 2021; 49: 948-957
        • Ramkumar P.N.
        • Karnuta J.M.
        • Navarro S.M.
        • et al.
        Deep learning preoperatively predicts value metrics for primary total knee arthroplasty: Development and validation of an artificial neural network model.
        J Arthroplasty. 2019; 34: 2220-2227.e1
        • Collins G.S.
        • Reitsma J.B.
        • Altman D.G.
        • Moons K.G.
        Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement.
        Br J Surg. 2015; 102: 148-158
        • Collins G.S.
        • Dhiman P.
        • Andaur Navarro C.L.
        • et al.
        Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence.
        BMJ Open. 2021; 11e048008
        • Kunze K.N.
        • Orr M.
        • Krebs V.
        • Bhandari M.
        • Piuzzi N.S.
        Potential benefits, unintended consequences, and future roles of artificial intelligence in orthopaedic surgery research: A call to emphasize data quality and indications.
        Bone Joint Open. 2022; 3: 93-97
      5. Kunze KN, Manzi JE, Polce EM, Vadhera A, Bhandari M, Piuzzi NS. High social media attention scores are not reflective of study quality: An altmetrics-based content analysis [published online February 9, 2022]. Intern Emerg Med. https://doi.org/10.1007/s11739-022-02939-5