Purpose
The purpose of this study is to further evaluate the construct validity and interobserver
reliability of a hip arthroscopy virtual simulator using the Arthroscopic Surgery
Skill Evaluation Tool (ASSET) global rating scale.
Methods
Thirty participants (23 male/7 female) completed a diagnostic arthroscopy and a loose
body retrieval simulation on the VirtaMed Arthros Hip Simulator (Zurich, Switzerland)
twice at a minimum of 1 week apart. Subjects consisted of 12 novices (medical students,
postgraduate year [PGY] 1-2), 5 intermediate trainees (PGY3-4), 9 senior trainees
(PGY5 and fellows), and 4 attending faculty. Simulator metrics were recorded and then
compiled to generate a total simulator score (TSS). The loose body retrieval was graded
using the ASSET scoring tool. Inter-rater and intrarater reliability for the ASSET
for 2 blinded raters and construct validity of the ASSET and the TSS were calculated.
Correlation between the TSS, ASSET and individual simulator metrics was determined.
Results
Prior simulation experience (P ≤ 0.01) correlated with higher TSS and higher ASSET, while video game experience
correlated with higher TSS on the diagnostic module only (P = 0.004). There was a significant difference in ASSET score among all experience
groups (P < 0.04). Novices had the lowest mean ASSET whereas experts had the highest mean ASSET
with a difference of 17.4 points. Overall performance on the surgical module significantly
correlated with the ASSET score (r = 0.444, P = 0.016). There was a significant positive correlation among higher ASSET and number
of loose bodies retrieved, operation time, camera path and grasper path length, and
percentage of cartilage injury. ASSET demonstrated excellent intrarater reliability
and showed substantial or better inter-reliability in 8 of 9 domains.
Conclusion
The VirtaMed hip arthroscopy simulator demonstrated good construct validity and excellent
reliability for simulator-based metrics and ASSET score. Use of both simulator metrics
and ASSET offers a more comprehensive performance assessment on hip arthroscopy simulation
than either measure alone.
Clinical Relevance
As virtual reality simulation for arthroscopy becomes more commonplace in orthopaedic
training, evaluation of the most effective objective and subjective measures of performance
is necessary to optimize simulation training.
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to ArthroscopyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Improving resident performance in knee arthroscopy: a prospective value assessment of simulators and cadaveric skills laboratories.J Bone Joint Surg Am. 2016; 98: 220-225
- Arthroscopy skills development with a surgical simulator: a comparative study in orthopaedic surgery residents.Am J Sports Med. 2015; 43: 1526-1529
- Flying blind in American orthopaedic surgery: the urgent need for an educational curriculum in orthopaedic surgery residency training.Injury. 2014; 45: 465-466
- ACGME and ABOS changes for the orthopaedic surgery PGY-1 (intern) year.Clin Orthop Relat Res. 2013; 471: 3412-3416
- Simulation training: a systematic review of simulation in arthroscopy and proposal of a new competency-based training framework.Int J Surg. 2014; 12: 626-633
- Utility of modern arthroscopic simulator training models: a meta-analysis and updated systematic review.Arthroscopy. 2018; 34: 1650-1677
- Surgical experience correlates with performance on a virtual reality simulator for shoulder arthroscopy.Am J Sports Med. 2007; 35: 883-888
- The Arthroscopic Surgical Skill Evaluation Tool (ASSET).Am J Sports Med. 2013; 41: 1229-1237
- Methodologies for establishing validity in surgical simulation studies.Surgery. 2010; 147: 622-630
- Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.J Bone Joint Surg Am. 2013; 95: e1871-e1876
- Validation of assessing arthroscopic skill using the ASSET evaluation.J Surg Educ. 2019; 76: 1640-1644
- Computer-simulation training for knee and shoulder arthroscopic surgery.Arthroscopy. 2010; 26: 832-840
- Initial evaluation of a shoulder arthroscopy simulator: establishing construct validity.J Shoulder Elbow Surg. 2004; 13: 196-205
- Shoulder arthroscopy simulator training improves shoulder arthroscopy performance in a cadaveric model.Arthroscopy. 2013; 29: 982-985
- Arthroscopic shoulder surgical simulation training curriculum: transfer reliability and maintenance of skill over time.J Surg Educ. 2015; 72: 1118-1123
- Validation of a virtual reality-based hip arthroscopy simulator.Arthroscopy. 2019; 35: 789-795
- Efficacy of standardized training on a virtual reality simulator to advance knee and shoulder arthroscopic motor skills.BMC Musculoskelet Disord. 2018; 19: 150
- Wrist arthroscopy: can we gain proficiency through knee arthroscopy simulation?.J Surg Educ. 2018; 75: 1664-1672
- Trends in utilization and outcomes of hip arthroscopy in the United States between 2005 and 2013.J Arthroplasty. 2017; 32: 750-755
- Trends in hip arthroscopic labral repair: an American Board of Orthopaedic Surgery Database Study.Arthroscopy. 2019; 35: 1413-1419
- Defining the learning curve for hip arthroscopy: a threshold analysis of the volume-outcomes relationship.Am J Sports Med. 2018; 46: 1284-1293
- Retention of arthroscopic shoulder skills learned with use of a simulator. Demonstration of a learning curve and loss of performance level after a time delay.J Bone Joint Surg Am. 2009; 91: 1207-1213
- Transferring simulated arthroscopic skills to the operating theatre: a randomised blinded study.J Bone Joint Surg Br. 2008; 90: 494-499
- Learning and retaining simulated arthroscopic meniscal repair skills.J Bone Joint Surg Am. 2012; 94: e132
- Passive haptics in a knee arthroscopy simulator: is it valid for core skills training?.Clin Orthop Relat Res. 2006; 442: 13-20
- Evaluation of a virtual reality simulator for arthroscopy skills development.Arthroscopy. 2002; 18: E29
- Training in tasks with different visual-spatial components does not improve virtual arthroscopy performance.Surg Endosc. 2004; 18: 115-120
- Evaluation of skills in arthroscopic training based on trajectory and force data.Clin Orthop Relat Res. 2009; 467: 546-552
- Virtual reality hip arthroscopy simulator demonstrates sufficient face validity.Knee Surg Sports Traumatol Arthrosc. 2019; 27: 3162-3167
- The learning curves of a validated virtual reality hip arthroscopy simulator.Arch Orthop Trauma Surg. 2020; 140: 761-767
- Validation of the hip arthroscopy module of the VirtaMed Virtual Reality Arthroscopy Trainer.Surg Technol Int. 2019; 34: 430-436
- Testing the construct validity of a virtual reality hip arthroscopy simulator.Arthroscopy. 2017; 33: 566-571
- Validation of a dry model for assessing the performance of arthroscopic hip labral repair.Am J Sports Med. 2017; 45: 2125-2130
- Simulated hip arthroscopy skills: learning curves with the lateral and supine patient positions: a randomized trial.J Bone Joint Surg Am. 2012; 94: e68
- Basic hip arthroscopy: anatomic establishment of arthroscopic portals without fluoroscopic guidance.Arthrosc Tech. 2016; 5: e247-e250
- Right versus left hip arthroscopy for surgeons on the learning curve.Arthrosc Tech. 2017; 6: e1837-e1844
Article info
Publication history
Published online: February 01, 2021
Accepted:
January 17,
2021
Received:
May 24,
2020
See commentary on page 1867Footnotes
Full ICMJE author disclosure forms are available for this article online, as supplementary material.
Identification
Copyright
© 2021 by the Arthroscopy Association of North America
