Advertisement

The Arthroscopic Surgery Skill Evaluation Tool Global Rating Scale is a Valid and Reliable Adjunct Measure of Performance on a Virtual Reality Simulator for Hip Arthroscopy

Published:February 01, 2021DOI:https://doi.org/10.1016/j.arthro.2021.01.046

      Purpose

      The purpose of this study is to further evaluate the construct validity and interobserver reliability of a hip arthroscopy virtual simulator using the Arthroscopic Surgery Skill Evaluation Tool (ASSET) global rating scale.

      Methods

      Thirty participants (23 male/7 female) completed a diagnostic arthroscopy and a loose body retrieval simulation on the VirtaMed Arthros Hip Simulator (Zurich, Switzerland) twice at a minimum of 1 week apart. Subjects consisted of 12 novices (medical students, postgraduate year [PGY] 1-2), 5 intermediate trainees (PGY3-4), 9 senior trainees (PGY5 and fellows), and 4 attending faculty. Simulator metrics were recorded and then compiled to generate a total simulator score (TSS). The loose body retrieval was graded using the ASSET scoring tool. Inter-rater and intrarater reliability for the ASSET for 2 blinded raters and construct validity of the ASSET and the TSS were calculated. Correlation between the TSS, ASSET and individual simulator metrics was determined.

      Results

      Prior simulation experience (P ≤ 0.01) correlated with higher TSS and higher ASSET, while video game experience correlated with higher TSS on the diagnostic module only (P = 0.004). There was a significant difference in ASSET score among all experience groups (P < 0.04). Novices had the lowest mean ASSET whereas experts had the highest mean ASSET with a difference of 17.4 points. Overall performance on the surgical module significantly correlated with the ASSET score (r = 0.444, P = 0.016). There was a significant positive correlation among higher ASSET and number of loose bodies retrieved, operation time, camera path and grasper path length, and percentage of cartilage injury. ASSET demonstrated excellent intrarater reliability and showed substantial or better inter-reliability in 8 of 9 domains.

      Conclusion

      The VirtaMed hip arthroscopy simulator demonstrated good construct validity and excellent reliability for simulator-based metrics and ASSET score. Use of both simulator metrics and ASSET offers a more comprehensive performance assessment on hip arthroscopy simulation than either measure alone.

      Clinical Relevance

      As virtual reality simulation for arthroscopy becomes more commonplace in orthopaedic training, evaluation of the most effective objective and subjective measures of performance is necessary to optimize simulation training.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Camp C.L.
        • Krych A.J.
        • Stuart M.J.
        • Regnier T.D.
        • Mills K.M.
        • Turner N.S.
        Improving resident performance in knee arthroscopy: a prospective value assessment of simulators and cadaveric skills laboratories.
        J Bone Joint Surg Am. 2016; 98: 220-225
        • Rebolledo B.J.
        • Hammann-Scala J.
        • Leali A.
        • Ranawat A.S.
        Arthroscopy skills development with a surgical simulator: a comparative study in orthopaedic surgery residents.
        Am J Sports Med. 2015; 43: 1526-1529
        • Roberts C.S.
        • Hunter A.M.
        Flying blind in American orthopaedic surgery: the urgent need for an educational curriculum in orthopaedic surgery residency training.
        Injury. 2014; 45: 465-466
        • Dougherty P.J.
        • Marcus R.E.
        ACGME and ABOS changes for the orthopaedic surgery PGY-1 (intern) year.
        Clin Orthop Relat Res. 2013; 471: 3412-3416
        • Tay C.
        • Khajuria A.
        • Gupte C.
        Simulation training: a systematic review of simulation in arthroscopy and proposal of a new competency-based training framework.
        Int J Surg. 2014; 12: 626-633
        • Frank R.M.
        • Wang K.C.
        • Davey A.
        • et al.
        Utility of modern arthroscopic simulator training models: a meta-analysis and updated systematic review.
        Arthroscopy. 2018; 34: 1650-1677
        • Gomoll A.H.
        • O'Toole R.V.
        • Czarnecki J.
        • Warner J.J.
        Surgical experience correlates with performance on a virtual reality simulator for shoulder arthroscopy.
        Am J Sports Med. 2007; 35: 883-888
        • Koehler R.J.
        • Amsdell S.
        • Arendt E.A.
        • et al.
        The Arthroscopic Surgical Skill Evaluation Tool (ASSET).
        Am J Sports Med. 2013; 41: 1229-1237
        • Van Nortwick S.S.
        • Lendvay T.S.
        • Jensen A.R.
        • Wright A.S.
        • Horvath K.D.
        • Kim S.
        Methodologies for establishing validity in surgical simulation studies.
        Surgery. 2010; 147: 622-630
        • Koehler R.J.
        • Nicandri G.T.
        Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.
        J Bone Joint Surg Am. 2013; 95: e1871-e1876
        • Raja A.
        • Thomas P.
        • Harrison A.
        • Tompkins M.
        • Braman J.
        Validation of assessing arthroscopic skill using the ASSET evaluation.
        J Surg Educ. 2019; 76: 1640-1644
        • Modi C.S.
        • Morris G.
        • Mukherjee R.
        Computer-simulation training for knee and shoulder arthroscopic surgery.
        Arthroscopy. 2010; 26: 832-840
        • Srivastava S.
        • Youngblood P.L.
        • Rawn C.
        • Hariri S.
        • Heinrichs W.L.
        • Ladd A.L.
        Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity.
        J Shoulder Elbow Surg. 2004; 13: 196-205
        • Henn 3rd, R.F.
        • Shah N.
        • Warner J.J.
        • Gomoll A.H.
        Shoulder arthroscopy simulator training improves shoulder arthroscopy performance in a cadaveric model.
        Arthroscopy. 2013; 29: 982-985
        • Dunn J.C.
        • Belmont P.J.
        • Lanzi J.
        • et al.
        Arthroscopic shoulder surgical simulation training curriculum: transfer reliability and maintenance of skill over time.
        J Surg Educ. 2015; 72: 1118-1123
        • Bauer D.E.
        • Wieser K.
        • Aichmair A.
        • Zingg P.O.
        • Dora C.
        • Rahm S.
        Validation of a virtual reality-based hip arthroscopy simulator.
        Arthroscopy. 2019; 35: 789-795
        • Rahm S.
        • Wieser K.
        • Bauer D.E.
        • et al.
        Efficacy of standardized training on a virtual reality simulator to advance knee and shoulder arthroscopic motor skills.
        BMC Musculoskelet Disord. 2018; 19: 150
        • Ode G.
        • Loeffler B.
        • Chadderdon R.C.
        • et al.
        Wrist arthroscopy: can we gain proficiency through knee arthroscopy simulation?.
        J Surg Educ. 2018; 75: 1664-1672
        • Maradit Kremers H.
        • Schilz S.R.
        • Van Houten H.K.
        • et al.
        Trends in utilization and outcomes of hip arthroscopy in the United States between 2005 and 2013.
        J Arthroplasty. 2017; 32: 750-755
        • Westermann R.W.
        • Day M.A.
        • Duchman K.R.
        • Glass N.A.
        • Lynch T.S.
        • Rosneck J.T.
        Trends in hip arthroscopic labral repair: an American Board of Orthopaedic Surgery Database Study.
        Arthroscopy. 2019; 35: 1413-1419
        • Mehta N.
        • Chamberlin P.
        • Marx R.G.
        • et al.
        Defining the learning curve for hip arthroscopy: a threshold analysis of the volume-outcomes relationship.
        Am J Sports Med. 2018; 46: 1284-1293
        • Howells N.R.
        • Auplish S.
        • Hand G.C.
        • Gill H.S.
        • Carr A.J.
        • Rees J.L.
        Retention of arthroscopic shoulder skills learned with use of a simulator. Demonstration of a learning curve and loss of performance level after a time delay.
        J Bone Joint Surg Am. 2009; 91: 1207-1213
        • Howells N.R.
        • Gill H.S.
        • Carr A.J.
        • Price A.J.
        • Rees J.L.
        Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study.
        J Bone Joint Surg Br. 2008; 90: 494-499
        • Jackson W.F.
        • Khan T.
        • Alvand A.
        • et al.
        Learning and retaining simulated arthroscopic meniscal repair skills.
        J Bone Joint Surg Am. 2012; 94: e132
        • McCarthy A.D.
        • Moody L.
        • Waterworth A.R.
        • Bickerstaff D.R.
        Passive haptics in a knee arthroscopy simulator: is it valid for core skills training?.
        Clin Orthop Relat Res. 2006; 442: 13-20
        • Pedowitz R.A.
        • Esch J.
        • Snyder S.
        Evaluation of a virtual reality simulator for arthroscopy skills development.
        Arthroscopy. 2002; 18: E29
        • Strom P.
        • Kjellin A.
        • Hedman L.
        • Wredmark T.
        • Fellander-Tsai L.
        Training in tasks with different visual-spatial components does not improve virtual arthroscopy performance.
        Surg Endosc. 2004; 18: 115-120
        • Tashiro Y.
        • Miura H.
        • Nakanishi Y.
        • Okazaki K.
        • Iwamoto Y.
        Evaluation of skills in arthroscopic training based on trajectory and force data.
        Clin Orthop Relat Res. 2009; 467: 546-552
        • Bartlett J.D.
        • Lawrence J.E.
        • Khanduja V.
        Virtual reality hip arthroscopy simulator demonstrates sufficient face validity.
        Knee Surg Sports Traumatol Arthrosc. 2019; 27: 3162-3167
        • Bartlett J.D.
        • Lawrence J.E.
        • Yan M.
        • et al.
        The learning curves of a validated virtual reality hip arthroscopy simulator.
        Arch Orthop Trauma Surg. 2020; 140: 761-767
        • Gallagher K.
        • Bahadori S.
        • Antonis J.
        • Immins T.
        • Wainwright T.W.
        • Middleton R.
        Validation of the hip arthroscopy module of the VirtaMed Virtual Reality Arthroscopy Trainer.
        Surg Technol Int. 2019; 34: 430-436
        • Khanduja V.
        • Lawrence J.E.
        • Audenaert E.
        Testing the construct validity of a virtual reality hip arthroscopy simulator.
        Arthroscopy. 2017; 33: 566-571
        • Phillips L.
        • Cheung J.J.H.
        • Whelan D.B.
        • et al.
        Validation of a dry model for assessing the performance of arthroscopic hip labral repair.
        Am J Sports Med. 2017; 45: 2125-2130
        • Pollard T.C.
        • Khan T.
        • Price A.J.
        • Gill H.S.
        • Glyn-Jones S.
        • Rees J.L.
        Simulated hip arthroscopy skills: learning curves with the lateral and supine patient positions: a randomized trial.
        J Bone Joint Surg Am. 2012; 94: e68
        • Howse E.A.
        • Botros D.B.
        • Mannava S.
        • Stone A.V.
        • Stubbs A.J.
        Basic hip arthroscopy: anatomic establishment of arthroscopic portals without fluoroscopic guidance.
        Arthrosc Tech. 2016; 5: e247-e250
        • Buyukdogan K.
        • Utsunomiya H.
        • Bolia I.
        • et al.
        Right versus left hip arthroscopy for surgeons on the learning curve.
        Arthrosc Tech. 2017; 6: e1837-e1844