Advertisement

Meaningless Applications and Misguided Methodologies in Artificial Intelligence–Related Orthopaedic Research Propagates Hype Over Hope

      Abstract

      There exists great hope and hype in the literature surrounding applications of artificial intelligence (AI) to orthopaedic surgery. Between 2018 and 2021, a total of 178 AI-related articles were published in orthopaedics. However, for every 2 original research papers that apply AI to orthopaedics, a commentary or review is published (30.3%). AI-related research in orthopaedics frequently fails to provide use cases that offer the uninitiated an opportunity to appraise the importance of AI by studying meaningful questions, evaluating unknown hypotheses, or analyzing quality data. The hype perpetuates a feed-forward cycle that relegates AI to a meaningless buzzword by rewarding those with nascent understanding and rudimentary technical knowhow into committing several basic errors: (1) inappropriately conflating vernacular (“AI/machine learning”), (2) repackaging registry data, (3) prematurely releasing internally validated algorithms, (4) overstating the “black box phenomenon” by failing to provide weighted analysis, (5) claiming to evaluate AI rather than the data itself, and (6) withholding full model architecture code. Relevant AI-specific guidelines are forthcoming, but forced application of the original Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines designed for regression analyses is irrelevant and misleading. To safeguard meaningful use, AI-related research efforts in orthopaedics should be (1) directed toward administrative support over clinical evaluation and management, (2) require the use of the advanced model, and (3) answer a question that was previously unknown, unanswered, or unquantifiable.
      In the last 4 years, the intersection of artificial intelligence (AI) and orthopaedic surgery has been explored and extensively discussed in the peer-reviewed literature. Although the quality of the literature presently favors narrative and systematic reviews over meaningful original research, there is continued interest in publishing on the subject of “artificial intelligence and machine learning.” Rather than providing use cases that offer the uninitiated an opportunity to assess and appreciate the importance and impact of AI itself, a large portion of present-day research instead reduces the technology with low-quality data. In turn, this perpetuates a feed-forward cycle that relegates the topic to a meaningless buzzword and cultivates a false understanding of its potential. Worse, the hype props those with nascent understanding and rudimentary technical knowhow into developing and releasing internally validated algorithms that repackage extant registry data without addressing meaningful problems or answering new questions. Now that AI awareness has been sufficiently established in the orthopaedic literature as a prospective predictive modeling tool, relevant use case application with thoughtful methodologic execution is critical. The original Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines do not apply to AI research. Rather than propagating the misconception that orthopaedic applications surrounding AI should be directed toward clinical evaluation and management, attention should instead be redirected to improving the experience of the orthopaedic patient and surgeon to fully realize the expectations of AI.

      The Good

      Although the related technical aspects may be beyond the scope of the orthopaedic surgeon, AI is not difficult to appreciate or conceptualize, as we have lived in a world that has surreptitiously applied it to our existence for years. From targeted suggestions that offer us our next media segment to consume or the very diction that finishes our thought before we’re even done typing, AI has transformed numerous industries through newfound efficiencies and supportive decision-making. In brief, AI is computer automation. By building complex associations that are reinforced with positive and negative feedback, algorithms can be built to yield predictions that can iteratively improve with newly introduced data.
      • Ramkumar P.N.
      • Kunze K.N.
      • Haeberle H.S.
      • et al.
      Clinical and research medical applications of artificial intelligence.
      Unlike regression models that statically analyze finite relationships with relatively homogeneous data formats, AI is capable of dynamically analyzing disorganized data and prospectively predicting outcomes.
      • Ramkumar P.N.
      • Kunze K.N.
      • Haeberle H.S.
      • et al.
      Clinical and research medical applications of artificial intelligence.
      ,
      • Makhni E.C.
      • Makhni S.
      • Ramkumar P.N.
      Artificial intelligence for the orthopaedic surgeon: An overview of potential benefits, limitations, and clinical applications.
      With the exponential growth of computing power and subsequent decrease in cost to run these automated processes, AI remains ripe to transform health care and, specifically, orthopaedic surgery along all phases of the care continuum as we continue to aggregate more data than what a single individual or team of individuals can process. Recent applications and research efforts implementing AI in the field of orthopaedics have demonstrated great promise in predicting future injury risk,
      • Luu B.
      • Wright A.
      • Haeberle H.
      • et al.
      Machine learning outperforms logistic regression analysis to predict next season NHL player injury: An analysis of 2,322 players from 2007-2017.
      ,
      • Karnuta J.
      • Luu B.
      • Haeberle H.
      • et al.
      Machine learning outperforms regression analysis to predict next season MLB player injury: Epidemiology and validation of 13,982 player-years from performance and injury profile trends between 2000-17.
      interpreting advanced imaging,
      • Ramkumar P.N.
      • Karnuta J.M.
      • Haeberle H.S.
      • et al.
      Radiographic indices are not predictive of clinical outcome among 1,735 patients indicated for hip arthroscopy: A machine learning analysis.
      • Fritz B.
      • Marbach G.
      • Civardi F.
      • Fucentese S.F.
      • Pfirrmann C.W.A.
      Deep convolutional neural network-based detection of meniscus tears: Comparison with radiologists and surgery as standard of reference.
      • Chang P.D.
      • Wong T.T.
      • Rasiej M.J.
      Deep learning for detection of complete anterior cruciate ligament tear.
      identifying implants,
      • Karnuta J.M.
      • Haeberle H.S.
      • Luu B.C.
      • et al.
      Artificial intelligence to identify arthroplasty implants from radiographs of the hip.
      • Karnuta J.M.
      • Luu B.C.
      • Roth A.L.
      • et al.
      Artificial intelligence to identify arthroplasty implants from radiographs of the knee.
      • Borjali A.
      • Chen A.F.
      • Muratoglu O.K.
      • Morid M.A.
      • Varadarajan K.M.
      Detecting total hip replacement prosthesis design on plain radiographs using deep convolutional neural network.
      • Murphy M.
      • Killen C.
      • Burnham R.
      • Sarvari F.
      • Wu K.
      • Brown N.
      Artificial intelligence accurately identifies total hip arthroplasty implants: a tool for revision surgery.
      • Nwachukwu B.U.
      • Beck E.C.
      • Lee E.K.
      • et al.
      Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery.
      projecting patient-reported outcomes,
      • Karnuta J.M.
      • Churchill J.L.
      • Haeberle H.S.
      • et al.
      The value of artificial neural networks for predicting length of stay, discharge disposition and inpatient costs after anatomic and reverse shoulder arthroplasty.
      reporting value-based metrics,
      • Ramkumar P.N.
      • Haeberle H.S.
      • Ramanathan D.
      • et al.
      Remote patient monitoring using mobile health for total knee arthroplasty: Validation of a wearable and machine learning–based surveillance platform.
      and augmenting telehealth.
      • Ramkumar P.N.
      • Haeberle H.S.
      • Bloomfield M.R.
      • et al.
      Artificial intelligence and arthroplasty at a single institution: Real-world applications of machine learning to big data, value-based care, mobile health, and remote patient monitoring.
      ,

      Kim JS, Vivas A, Arvind V, et al. Can natural language processing and artificial intelligence automate the generation of billing codes from operative note dictations [published online February 28, 2022]? Global Spine J. https://doi.org/10.1177/21925682211062831

      Overall, AI remains an exciting prospective tool that applies data in real time to automate tasks that have the potential to fundamentally improve patient care, unburden administrative work, and translate once-qualitative insights into tangible quantitative metrics.
      The fundamental question surrounding the emphasis on AI in orthopaedics is “why should we care?” It was not long ago when surgeons felt the business of medicine was outside the scope of our expertise. Today, we find it vital to our survival and increasingly essential in educating trainees. How we understand, leverage, and interact with technology will similarly be critical to our day-to-day workflow. AI has the potential to dictate how we perform clinical activities, communicate in medicolegal terms, and allocate our time for optimal resource use. Consider the clinic encounter wherein documentation of multiple elements within the electronic health record (EHR) are necessary to appropriately bill. Oftentimes, we are faced to choose between documenting during the face-to-face encounter, limit our time with the patient, or run late. All options detract from the experience of both the surgeon and the patient. Through natural language processing, a subset of AI, autocompletion of documentation and clicks for standard encounters can rapidly save time. In one report by Kim et al.,

      Kim JS, Vivas A, Arvind V, et al. Can natural language processing and artificial intelligence automate the generation of billing codes from operative note dictations [published online February 28, 2022]? Global Spine J. https://doi.org/10.1177/21925682211062831

      billing codes were able to be automatically generated from operative note dictations. Today, administrative burden associated with the clinical encounter on EHRs directly contributes to burnout. For every 15-minute patient encounter, physicians spend over 16 minutes just on the EHR and 11% of that time documenting spills into the “after-hours” (weekends or outside of clinic business hours).
      • Overhage J.M.
      • McCallie Jr., D.
      Physician time spent using the electronic health record during outpatient encounters: A descriptive study.
      The importance of adopting AI-based techniques is misunderstood. Contrary to popular belief, AI is not intended to—nor should it ever be suggested to—replace routine patient-facing clinical activities of the orthopaedic surgeon. Overall, as orthopaedic surgeons we do a great job of taking care of patients. The actual “doctor” activities of diagnosis and management do not require reinvention. Instead, AI is better served redressing the areas that have contributed to a U.S. health care system many consider to be broken, from patient access and caregiver wellness to financial transparency. Numerous opportunities exist for the orthopaedic surgeon to leverage what many consider to be the fourth industrial revolution. In addition to personalizing medicine and decreasing administrative burnout, AI offers the ability to navigate the present pandemic by enabling remote patient monitoring for the orthopaedic patient.
      • Ramkumar P.N.
      • Haeberle H.S.
      • Ramanathan D.
      • et al.
      Remote patient monitoring using mobile health for total knee arthroplasty: Validation of a wearable and machine learning–based surveillance platform.
      ,
      • Ramkumar P.N.
      • Haeberle H.S.
      • Bloomfield M.R.
      • et al.
      Artificial intelligence and arthroplasty at a single institution: Real-world applications of machine learning to big data, value-based care, mobile health, and remote patient monitoring.
      Through pre-existing sensors on smartphones and wearable devices, physical examination metrics like knee range of motion and gait can be remotely ascertained in the perioperative period.
      • Ramkumar P.N.
      • Haeberle H.S.
      • Ramanathan D.
      • et al.
      Remote patient monitoring using mobile health for total knee arthroplasty: Validation of a wearable and machine learning–based surveillance platform.
      Remote patient monitoring offers unique convenience to both the surgeon and the patient while ensuring patient safety. When considering financial transparency and value-based care in orthopaedics, patient-specific payment models have been proposed that stratify patients by their preoperative comorbidities to normalize surgeon reimbursement to the preoperative surgical risk.
      • Navarro S.M.
      • Wang E.Y.
      • Haeberle H.S.
      • et al.
      Machine learning and primary total knee arthroplasty: Patient forecasting for a patient-specific payment model.
      ,
      • Ramkumar P.N.
      • Karnuta J.M.
      • Navarro S.M.
      • et al.
      Preoperative prediction of value metrics and a patient-specific payment model for primary total hip arthroplasty: Development and validation of a deep learning model.
      During a time where reimbursements continue to be devalued for procedural work, AI can be preoperatively leveraged to predict the risk profile of a patient before the incision to better arbitrate reimbursement fees. With more than 16 hours per week spent on haggling with prior authorization approval, this area remains ripe for automation.
      • Choudhury A.
      • Perumalla S.
      Using machine learning to minimize delays caused by prior authorization: A brief report.
      The use cases for AI in improving the experience of patients, orthopaedic surgeons, and their teams through automation are limitless and cause for continued hope in this domain of research.

      The Bad

      Despite the promise of AI in orthopaedics, the exponential growth in this realm of research has created several dilemmas that warrant acknowledgement and redressal. While the early acceptance from the orthopaedic community to learn a foreign concept was initially inspiring, AI has now been couched as a colossal buzzword that conveys emptiness and invites skepticism. Beginning with terminology, several articles refer to the topic as “AI and machine learning (ML)” when this is both redundant and unclear.
      • Polce E.M.
      • Kunze K.N.
      • Dooley M.S.
      • Piuzzi N.S.
      • Boettner F.
      • Sculco P.K.
      Efficacy and applications of artificial intelligence and machine learning analyses in total joint arthroplasty: A call for improved reporting.
      ,

      Rubinger L, Gazendam A, Ekhtiari S, Bhandari M. Machine learning and artificial intelligence in research and healthcare [published online February 1, 2022]. Injury. https://doi.org/10.1016/j.injury.2022.01.046

      Conceptually, AI is the umbrella term that enables a machine to simulate human behavior; ML is a narrower term and small subset of AI that performs specific tasks to yield predictions using pattern recognition. AI systems can apply ML, but ML cannot apply AI. When the nomenclature is described as a pair, this undermines the authority of the presented content and creates confusion.
      A large portion of the orthopaedic research concerning AI is portrayed and personified as if the readership is evaluating the merits as its judge, jury, and executioner. This is evident with literature entitled “Machine learning algorithms predict…” or “AI predicts….”.

      Kunze KN, Polce EM, Patel A, Courtney PM, Sporer SM, Levine BR. Machine learning algorithms predict within one size of the final implant ultimately used in total knee arthroplasty with good-to-excellent accuracy [published online January 13, 2022]. Knee Surg Sports Traumatol Arthrosc. https://doi.org/10.1007/s00167-022-06866-y

      • Polce E.M.
      • Kunze K.N.
      • Paul K.M.
      • Levine B.R.
      Machine learning predicts femoral and tibial implant size mismatch for total knee arthroplasty.

      Kunze KN, Krivicich LM, Clapp IM, et al. Machine learning algorithms predict achievement of clinically significant outcomes after orthopaedic surgery: A systematic review [published online December 27, 2021]. Arthroscopy. https://doi.org/10.1016/j.arthro.2021.12.030

      • Kunze K.N.
      • Polce E.M.
      • Alter T.D.
      • Nho S.J.
      Machine learning algorithms predict prolonged opioid use in opioid-naïve primary hip arthroscopy patients.
      • Kunze K.N.
      • Polce E.M.
      • Clapp I.
      • Nwachukwu B.U.
      • Chahla J.
      • Nho S.J.
      machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes.
      • Kunze K.N.
      • Polce E.M.
      • Rasio J.
      • Nho S.J.
      Machine learning algorithms predict clinically significant improvements in satisfaction after hip arthroscopy.
      This portrayal is closer to marketing than science and underscores a fundamentally misunderstood tenet: AI is only as good as the data that fuel its process. The bottleneck of AI-derived insights is not the analytic process, but rather the quality of the data. As with any peer-reviewed publication containing statistical analysis, the data drive the quality and generalizability of the conclusions. In most cases wherein authors claim to be evaluating AI-based processes, most are in fact simply evaluating the reflection of their data. When certain reports refute the ability of an AI-based process to supersede the predictive power of traditional regression, it occurs when the data are low volume or low quality. If a Ferrari and Prius were asked to race in rush hour traffic on the 101 freeway in Los Angeles, the horsepower of the Ferrari doesn’t help much, and one could conclude the Prius is just as fast as a Ferrari. While this is farcical, this is the exact phenomenon that is occurring when we attempt to compare deep learning models with archaic regression analyses in the setting of poor data.
      Not only are these studies failing to actually evaluate AI, an equivalent corollary would be to rename these articles “ANOVA shows the obese and morbidly obese have worse outcomes after knee replacement” or “Student t test demonstrates smokers are more likely to develop nonunion after fracture fixation.” Just as we are not evaluating t tests, we are not evaluating AI. Instead, we are using AI as a tool to evaluate the data and its conclusions as with any other research report. Taken one step further, some authors even attempt to caution or apply unique ethical principles to AI, as if it warrants a new playbook.
      • González-Esteban Y.
      • Patrici Calvo E.
      Ethically governing artificial intelligence in the field of scientific research and innovation.
      The ethics related to AI do not differ from the ethics of any other statistical process, which center on the data. The same principles that guide data storage, management, and analysis apply regardless of the statistical modeling process.
      • Pua Y.H.
      • Kang H.
      • Thumboo J.
      • et al.
      Machine learning methods are comparable to logistic regression techniques in predicting severe walking limitation following total knee arthroplasty.
      With respect to quality control for original research applying AI-based algorithms, there exist three points of emphasis with respect to the methods, results, and discussion. While guidelines surrounding AI-related research are not yet available and will be discussed in depth in the subsequent section, authors in should always provide the full code (i.e., GitHub URL) to maximize generalizability and transparency. Additionally, the “black box” of this realm of research exists but is presently overstated. With most analyses, there exists a way to cross-sectionally evaluate how the algorithm is performing with SHAP (SHapley Additive exPlanations) analysis
      • Ramkumar P.N.
      • Karnuta J.M.
      • Haeberle H.S.
      • Rodeo S.A.
      • Nwachukwu B.U.
      • Williams 3rd, R.J.
      Effect of preoperative imaging and patient factors on clinically meaningful outcomes and quality of life after osteochondral allograft transplantation: A machine learning analysis of cartilage defects of the knee.
      ,
      • Ramkumar P.N.
      • Karnuta J.M.
      • Haeberle H.S.
      • et al.
      Association between preoperative mental health and clinically meaningful outcomes after osteochondral allograft for cartilage defects of the knee: A machine learning analysis.
      for individual predictions or heatmaps
      • Karnuta J.M.
      • Haeberle H.S.
      • Luu B.C.
      • et al.
      Artificial intelligence to identify arthroplasty implants from radiographs of the hip.
      ,
      • Karnuta J.M.
      • Luu B.C.
      • Roth A.L.
      • et al.
      Artificial intelligence to identify arthroplasty implants from radiographs of the knee.
      for imaging feature identification. In addition, if the study was not evaluated with multiple populations, then the work needs external validation before releasing a makeshift online prediction tool (i.e., https://www.shinyapps.io/) available to the general public. Without external validation, the tool could confer premature generalizability and generate false predictions that compromise patient safety or misguide communication.

      The Ugly

      The potential surrounding AI has in turn left both journals and authors susceptible to abuse as we stand atop its hype cycle. AI confers the distinct advantage of automation to provide insights that were previously unknown, unanswered, or unquantifiable. When evaluating a study that boasts use of AI or ML, ask (1) if the question has ever been answered before and (2) if AI was necessary to answer the question. Although “publish or perish” remains a sad reality, the onus is as much on the authors as it is on the journals to avoid duping editorials boards and readers unfamiliar with AI into believing a study is unique simply because a trendy statistical model was applied or a popular buzzword increased pageviews. For those familiar with ML techniques, it is very easy to repackage previous studies using extant registry data—especially using unoriginal or previously answered questions—and generate a clinical prediction tool. Unless that tool undergoes external validation with multiple population data sources and is prospectively applied for evaluation, the study has done nothing more than to continue hollowing meaningful applications of AI and fortify its stature as a buzzword. Rather than indulging in the gratuitous application of AI, the most meaningful use for this technology is to apply it to clinical activities that are nondiagnostic and treatment-related. Physicians have spent over a decade trained in the art, science, and humanity of medical diagnosis and management; directing efforts to unburdening the clinical workforce from clinical medicine would be a misappropriation of resources. Instead, AI should be directed toward optimizing the health care experience by improving access, decreasing the documentation burden, and optimizing value-based communication. One way to optimize value-based communication is to take advantage of the evidence-based aspects of predictive AI to arbitrate risk-adjusted reimbursement models before elective orthopaedic surgery among surgeons, payers, and hospitals.
      • Ramkumar P.N.
      • Karnuta J.M.
      • Navarro S.M.
      • et al.
      Preoperative prediction of value metrics and a patient-specific payment model for primary total hip arthroplasty: Development and validation of a deep learning model.
      ,
      • Ramkumar P.N.
      • Karnuta J.M.
      • Navarro S.M.
      • et al.
      Deep learning preoperatively predicts value metrics for primary total knee arthroplasty: Development and validation of an artificial neural network model.
      Along the same vein of perpetuating hype over substance with regards to AI research in orthopaedics, Figure 1 illustrates the increased interest to publish on the topic. From 2018 to 2021, a total of 178 AI-related articles were published in orthopaedics; of these 178, 54 were narratives or review articles on the subject (30.3%). For every 2 original research papers that develop and apply AI techniques to orthopaedic applications, a commentary or update is published. A recent trend among reviews entails the forced application of the TRIPOD guidelines to grade AI research, which serves as an emblematic microcosm for the most pressing issue in this realm of research.
      • Collins G.S.
      • Reitsma J.B.
      • Altman D.G.
      • Moons K.G.
      Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement.
      Despite the fact that these guidelines were designed only for traditional multivariate models and specific AI guidelines are imminent,
      • Collins G.S.
      • Dhiman P.
      • Andaur Navarro C.L.
      • et al.
      Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence.
      editorial boards and authors either lack the familiarity or interest to question the coarse translation of antiquated guidelines in favor of publishing yet another hollow review article.
      • Polce E.M.
      • Kunze K.N.
      • Dooley M.S.
      • Piuzzi N.S.
      • Boettner F.
      • Sculco P.K.
      Efficacy and applications of artificial intelligence and machine learning analyses in total joint arthroplasty: A call for improved reporting.
      ,
      • Kunze K.N.
      • Orr M.
      • Krebs V.
      • Bhandari M.
      • Piuzzi N.S.
      Potential benefits, unintended consequences, and future roles of artificial intelligence in orthopaedic surgery research: A call to emphasize data quality and indications.
      To many researchers and journals, the orthopaedic literature on AI should focus on meaningful original research instead of misguided content intended to generate clicks.

      Kunze KN, Manzi JE, Polce EM, Vadhera A, Bhandari M, Piuzzi NS. High social media attention scores are not reflective of study quality: An altmetrics-based content analysis [published online February 9, 2022]. Intern Emerg Med. https://doi.org/10.1007/s11739-022-02939-5

      Figure thumbnail gr1
      Fig 1Year-over-year growth of artificial intelligence research in orthopaedics from 2018 to 2021 with respect to original and nonoriginal research.

      Conclusions

      Although AI has already demonstrated great promise to transform orthopaedics, research efforts should be directed toward optimizing the health care experience by improving access, decreasing the documentation burden, and optimizing value-based communication instead of clinical diagnostic and treatment support tools. When evaluating the quality of AI research in orthopaedics, nomenclature matters, and most studies are evaluating the data rather than the actual analytic technique. Fundamental principles of sharing the full code instead of prematurely releasing publicly available and internally validated prediction tools will be more useful for researchers. The hype surrounding AI in orthopaedic research has generated a review article for every 2 original reports; forced application of the original TRIPOD guidelines are not applicable to AI research and relevant guidelines are forthcoming. To ensure quality AI research in orthopaedics, the question must be previously unknown, unanswered, or unquantifiable and the advanced model must be necessary.

      Supplementary Data

      References

        • Ramkumar P.N.
        • Kunze K.N.
        • Haeberle H.S.
        • et al.
        Clinical and research medical applications of artificial intelligence.
        Arthroscopy. 2021; 37: 1694-1697
        • Makhni E.C.
        • Makhni S.
        • Ramkumar P.N.
        Artificial intelligence for the orthopaedic surgeon: An overview of potential benefits, limitations, and clinical applications.
        J Am Acad Orthop Surg. 2021; 29: 235-243
        • Luu B.
        • Wright A.
        • Haeberle H.
        • et al.
        Machine learning outperforms logistic regression analysis to predict next season NHL player injury: An analysis of 2,322 players from 2007-2017.
        Orthop J Sport Med. 2020; 82325967120953404
        • Karnuta J.
        • Luu B.
        • Haeberle H.
        • et al.
        Machine learning outperforms regression analysis to predict next season MLB player injury: Epidemiology and validation of 13,982 player-years from performance and injury profile trends between 2000-17.
        Orthop J Sport Med. 2020; 82325967120963046
        • Ramkumar P.N.
        • Karnuta J.M.
        • Haeberle H.S.
        • et al.
        Radiographic indices are not predictive of clinical outcome among 1,735 patients indicated for hip arthroscopy: A machine learning analysis.
        Am J Sports Med. 2020; 48: 2910-2918
        • Fritz B.
        • Marbach G.
        • Civardi F.
        • Fucentese S.F.
        • Pfirrmann C.W.A.
        Deep convolutional neural network-based detection of meniscus tears: Comparison with radiologists and surgery as standard of reference.
        Skeletal Radiol. 2020; 49: 1207-1217
        • Chang P.D.
        • Wong T.T.
        • Rasiej M.J.
        Deep learning for detection of complete anterior cruciate ligament tear.
        J Digit Imaging. 2019; 32: 980-986
        • Karnuta J.M.
        • Haeberle H.S.
        • Luu B.C.
        • et al.
        Artificial intelligence to identify arthroplasty implants from radiographs of the hip.
        J Arthroplasty. 2021; 36: S290-S294
        • Karnuta J.M.
        • Luu B.C.
        • Roth A.L.
        • et al.
        Artificial intelligence to identify arthroplasty implants from radiographs of the knee.
        J Arthroplasty. 2021; 36: 935-940
        • Borjali A.
        • Chen A.F.
        • Muratoglu O.K.
        • Morid M.A.
        • Varadarajan K.M.
        Detecting total hip replacement prosthesis design on plain radiographs using deep convolutional neural network.
        J Orthop Res. 2020; 38: 1465-1471
        • Murphy M.
        • Killen C.
        • Burnham R.
        • Sarvari F.
        • Wu K.
        • Brown N.
        Artificial intelligence accurately identifies total hip arthroplasty implants: a tool for revision surgery.
        Hip Int. 2021 Jan 8; 1120700020987526
        • Nwachukwu B.U.
        • Beck E.C.
        • Lee E.K.
        • et al.
        Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery.
        Am J Sports Med. 2020; 48: 415-423
        • Karnuta J.M.
        • Churchill J.L.
        • Haeberle H.S.
        • et al.
        The value of artificial neural networks for predicting length of stay, discharge disposition and inpatient costs after anatomic and reverse shoulder arthroplasty.
        J Shoulder Elbow Surg. 2020; 29: 2385-2394
        • Ramkumar P.N.
        • Haeberle H.S.
        • Ramanathan D.
        • et al.
        Remote patient monitoring using mobile health for total knee arthroplasty: Validation of a wearable and machine learning–based surveillance platform.
        J Arthroplasty. 2019; 34: 2253-2259
        • Ramkumar P.N.
        • Haeberle H.S.
        • Bloomfield M.R.
        • et al.
        Artificial intelligence and arthroplasty at a single institution: Real-world applications of machine learning to big data, value-based care, mobile health, and remote patient monitoring.
        J Arthroplasty. 2019; 34: 2204-2209
      1. Kim JS, Vivas A, Arvind V, et al. Can natural language processing and artificial intelligence automate the generation of billing codes from operative note dictations [published online February 28, 2022]? Global Spine J. https://doi.org/10.1177/21925682211062831

        • Overhage J.M.
        • McCallie Jr., D.
        Physician time spent using the electronic health record during outpatient encounters: A descriptive study.
        Ann Intern Med. 2020; 172: 169-174
        • Navarro S.M.
        • Wang E.Y.
        • Haeberle H.S.
        • et al.
        Machine learning and primary total knee arthroplasty: Patient forecasting for a patient-specific payment model.
        J Arthroplasty. 2018; 33: 3617-3623
        • Ramkumar P.N.
        • Karnuta J.M.
        • Navarro S.M.
        • et al.
        Preoperative prediction of value metrics and a patient-specific payment model for primary total hip arthroplasty: Development and validation of a deep learning model.
        J Arthroplasty. 2019; 34: 2228-2234.e1
        • Choudhury A.
        • Perumalla S.
        Using machine learning to minimize delays caused by prior authorization: A brief report.
        Cogent Engineering. 2021; 8: 1
        • Polce E.M.
        • Kunze K.N.
        • Dooley M.S.
        • Piuzzi N.S.
        • Boettner F.
        • Sculco P.K.
        Efficacy and applications of artificial intelligence and machine learning analyses in total joint arthroplasty: A call for improved reporting.
        J Bone Joint Surg Am. 2022; 104: 821-832
      2. Rubinger L, Gazendam A, Ekhtiari S, Bhandari M. Machine learning and artificial intelligence in research and healthcare [published online February 1, 2022]. Injury. https://doi.org/10.1016/j.injury.2022.01.046

      3. Kunze KN, Polce EM, Patel A, Courtney PM, Sporer SM, Levine BR. Machine learning algorithms predict within one size of the final implant ultimately used in total knee arthroplasty with good-to-excellent accuracy [published online January 13, 2022]. Knee Surg Sports Traumatol Arthrosc. https://doi.org/10.1007/s00167-022-06866-y

        • Polce E.M.
        • Kunze K.N.
        • Paul K.M.
        • Levine B.R.
        Machine learning predicts femoral and tibial implant size mismatch for total knee arthroplasty.
        Arthroplast Today. 2021; 8: 268-277.e2
      4. Kunze KN, Krivicich LM, Clapp IM, et al. Machine learning algorithms predict achievement of clinically significant outcomes after orthopaedic surgery: A systematic review [published online December 27, 2021]. Arthroscopy. https://doi.org/10.1016/j.arthro.2021.12.030

        • Kunze K.N.
        • Polce E.M.
        • Alter T.D.
        • Nho S.J.
        Machine learning algorithms predict prolonged opioid use in opioid-naïve primary hip arthroscopy patients.
        J Am Acad Orthop Surg Glob Res Rev. 2021; 5: e21.00093-e21.00098
        • Kunze K.N.
        • Polce E.M.
        • Clapp I.
        • Nwachukwu B.U.
        • Chahla J.
        • Nho S.J.
        machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes.
        J Bone Joint Surg Am. 2021; 103: 1055-1062
        • Kunze K.N.
        • Polce E.M.
        • Rasio J.
        • Nho S.J.
        Machine learning algorithms predict clinically significant improvements in satisfaction after hip arthroscopy.
        Arthroscopy. 2021; 37: 1143-1151
        • González-Esteban Y.
        • Patrici Calvo E.
        Ethically governing artificial intelligence in the field of scientific research and innovation.
        Heliyon. 2022; 8e08946
        • Pua Y.H.
        • Kang H.
        • Thumboo J.
        • et al.
        Machine learning methods are comparable to logistic regression techniques in predicting severe walking limitation following total knee arthroplasty.
        Knee Surg Sports Traumatol Arthrosc. 2020; 28: 3207-3216
        • Ramkumar P.N.
        • Karnuta J.M.
        • Haeberle H.S.
        • Rodeo S.A.
        • Nwachukwu B.U.
        • Williams 3rd, R.J.
        Effect of preoperative imaging and patient factors on clinically meaningful outcomes and quality of life after osteochondral allograft transplantation: A machine learning analysis of cartilage defects of the knee.
        Am J Sports Med. 2021; 49: 2177-2186
        • Ramkumar P.N.
        • Karnuta J.M.
        • Haeberle H.S.
        • et al.
        Association between preoperative mental health and clinically meaningful outcomes after osteochondral allograft for cartilage defects of the knee: A machine learning analysis.
        Am J Sports Med. 2021; 49: 948-957
        • Ramkumar P.N.
        • Karnuta J.M.
        • Navarro S.M.
        • et al.
        Deep learning preoperatively predicts value metrics for primary total knee arthroplasty: Development and validation of an artificial neural network model.
        J Arthroplasty. 2019; 34: 2220-2227.e1
        • Collins G.S.
        • Reitsma J.B.
        • Altman D.G.
        • Moons K.G.
        Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement.
        Br J Surg. 2015; 102: 148-158
        • Collins G.S.
        • Dhiman P.
        • Andaur Navarro C.L.
        • et al.
        Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence.
        BMJ Open. 2021; 11e048008
        • Kunze K.N.
        • Orr M.
        • Krebs V.
        • Bhandari M.
        • Piuzzi N.S.
        Potential benefits, unintended consequences, and future roles of artificial intelligence in orthopaedic surgery research: A call to emphasize data quality and indications.
        Bone Joint Open. 2022; 3: 93-97
      5. Kunze KN, Manzi JE, Polce EM, Vadhera A, Bhandari M, Piuzzi NS. High social media attention scores are not reflective of study quality: An altmetrics-based content analysis [published online February 9, 2022]. Intern Emerg Med. https://doi.org/10.1007/s11739-022-02939-5