Purpose
To create and determine face validity and content validity of arthroscopic rotator
cuff repair (ARCR) performance metrics, to confirm construct validity of the metrics
coupled with a cadaveric shoulder, and to establish a performance benchmark for the
procedure on a cadaveric shoulder.
Methods
Five experienced arthroscopic shoulder surgeons created step, error, and sentinel
error metrics for an ARCR. Fourteen shoulder arthroscopy faculty members from the
Arthroscopy Association of North America formed the modified Delphi panel to assess
face and content validity. Eight Arthroscopy Association of North America shoulder
arthroscopy faculty members (experienced group) were compared with 9 postgraduate
year 4 or 5 orthopaedic residents (novice group) in their ability to perform an ARCR.
Instructions were given to perform a diagnostic arthroscopy and a 2-anchor, 4–simple
suture repair of a 2-cm supraspinatus tear. The procedure was videotaped in its entirety
and independently scored in blinded fashion by trained, paired reviewers.
Results
Delphi panel consensus for 42 steps and 66 potential errors was obtained. Overall
performance assessment showed a mean inter-rater reliability of 0.93. Novice surgeons
completed 17% fewer steps (32.1 vs 37.5, P = .001) and enacted 2.5 times more errors than the experienced group (6.21 vs 2.5,
P = .012). Fifty percent of the experienced group members and none of the novice group
members achieved the proficiency benchmark of a minimum of 37 steps completed with
3 or fewer errors.
Conclusions
Face validity and content validity for the ARCR metrics, along with construct validity
for the metrics and cadaveric shoulder, were verified. A proficiency benchmark was
established based on the mean performance of an experienced group of arthroscopic
shoulder surgeons.
Clinical Relevance
Validated procedural metrics combined with the use of a cadaveric shoulder can be
used to accurately assess the performance of an ARCR.
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to ArthroscopyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Metric-based simulation training to proficiency in medical education:-What it is and how to do it.Ulster Med J. 2012; 81: 107-113
- Virtual reality simulation for the operating room: Proficiency-based training as a paradigm shift in surgical skills training.Ann Surg. 2005; 241: 364-372
- Chapter 12 - Proficiency-Based Progression Simulation Training: A To-Do List for Medicine.in: Fundamentals of surgical simulation: Principles and practices. Springer, London2012: 329
- Metric development for an arthroscopic Bankart procedure: Assessment of face and content validity.Arthroscopy. 2015; 3: 1430-1440
- Review of methods for objective surgical skill evaluation.Surg Endosc. 2011; 25: 356-366
- Motion analysis: A validated method for showing skill levels in arthroscopy.Arthroscopy. 2008; 243: 335-342
- Magellan and Copernicus: Arthroscopy Association of North America seeking excellence in education.Arthroscopy. 2015; 31: 1428-1429
- The Bankart performance metrics combined with a shoulder model simulator create a precise and accurate training tool for measuring surgeon skill.Arthroscopy. 2015; 31: 1639-1654
- The Bankart performance metrics combined with a cadaveric shoulder create a precise and accurate assessment tool for measuring surgeon skill.Arthroscopy. 2015; 31: 1655-1670
- A proficiency-based progression training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic Bankart skill set.Arthroscopy. 2015; 31: 1854-1871
- Analysis of errors in laparoscopic surgical procedures.Surg Endosc. 2004; 18: 592-595
- What do master surgeons think of surgical competence and revalidation?.Am J Surg. 2001; 182: 110-116
- Virtual reality training improves operating room performance: Results of a randomized, double-blinded study.Ann Surg. 2002; 236: 458-464
- Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies.Am J Surg. 2007; 193: 797-804
- Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying.J Am Coll Surg. 2008; 207: 560-568
- Outlier experienced surgeon’s performances impact on benchmark for technical surgical skills training.ANZ J Surg. 2018; 88: E412-E417
- Standards for educational and psychological testing.American Educational Research Association, Washington, D.C.1999
- Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures.J Am Coll Surg. 2003; 197: 479-488
- The knee arthroscopy learning curve: Quantitative assessment of surgical skills.Arthroscopy. 2014; 30: 613-621
- Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.J Bone Joint Surg Am. 2013; 95: e187
- Editorial Commentary: “A brush too broad” (validation, global rating scales, and the like . . . what do they really mean?).Arthroscopy. 2017; 33: 1004-1006
- Objective structured assessment of technical skills and checklist scales reliability compared for high stakes assessments.ANZ J Surg. 2014; 84: 568-573
- Fundamental principles of validation, and reliability: Rigorous science for the assessment of surgical education and training.Surg Endosc Intervent Tech. 2003; 17: 1525-1529
- A global assessment tool for evaluation of intraoperative laparoscopic skills.Am J Surg. 2005; 190: 107-113
- Prospective, randomised and blinded comparison of proficiency-based progression full-physics virtual reality simulator training versus invasive vascular experience for learning carotid artery angiography by very experienced operators.BMJ Simul Technol Enhanc Learn. 2016; 2: 1-5
- Proficiency-based progression training: An ‘end to end’ model for decreasing error applied to achievement of effective epidural analgesia during labour: A randomised control study.BMJ. 2018; 8: e020099
- Surgical skill and complication rates after bariatric surgery.N Engl J Med. 2013; 369: 1434-1442
Article info
Publication history
Accepted:
July 3,
2019
Received:
January 20,
2019
Footnotes
The authors report the following potential conflicts of interest or sources of funding: J.T. is a paid consultant for Conmed Medical and has patents and stock or stock options with KFX Medical. P.S.P. is on the AANA Board of Directors and is a consultant for and receives payment for lectures and royalties from DJO Surgical and Zimmer Biomet. G.R. is a consultant for Arthrex and is a consultant for and receives payment for lectures from Stryker. I.V. is a consultant for and receives payment for lectures from Zimmer Biomet, Arthrex, Smith and Nephew, and Arthrosurface; receives royalties from Inomed; and receives payment for development of educational presentations from Zimmer, Arthrex, and Smith and Nephew. R.K.N.R. is a consultant for Medbridge; receives payment for lectures from Mitek/Smith and Nephew; and receives royalties from the AANA. L.M. is a board member for AANA, receives support for himself and his institution for consultancy from Smith and Nephew, OrthoSpacer, and Active Implants, is employed by Northwell Health, has grants/grants pending from Flexion Therapeutics, and receives support for lectures including service on speakers bureaus from Smith and Nephew; A.G.G. receives consulting fees for expert advice on the methodology. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
Identification
Copyright
© 2019 by the Arthroscopy Association of North America
