Advertisement

Arthroscopic Rotator Cuff Repair Metrics: Establishing Face, Content, and Construct Validity in a Cadaveric Model

      Purpose

      To create and determine face validity and content validity of arthroscopic rotator cuff repair (ARCR) performance metrics, to confirm construct validity of the metrics coupled with a cadaveric shoulder, and to establish a performance benchmark for the procedure on a cadaveric shoulder.

      Methods

      Five experienced arthroscopic shoulder surgeons created step, error, and sentinel error metrics for an ARCR. Fourteen shoulder arthroscopy faculty members from the Arthroscopy Association of North America formed the modified Delphi panel to assess face and content validity. Eight Arthroscopy Association of North America shoulder arthroscopy faculty members (experienced group) were compared with 9 postgraduate year 4 or 5 orthopaedic residents (novice group) in their ability to perform an ARCR. Instructions were given to perform a diagnostic arthroscopy and a 2-anchor, 4–simple suture repair of a 2-cm supraspinatus tear. The procedure was videotaped in its entirety and independently scored in blinded fashion by trained, paired reviewers.

      Results

      Delphi panel consensus for 42 steps and 66 potential errors was obtained. Overall performance assessment showed a mean inter-rater reliability of 0.93. Novice surgeons completed 17% fewer steps (32.1 vs 37.5, P = .001) and enacted 2.5 times more errors than the experienced group (6.21 vs 2.5, P = .012). Fifty percent of the experienced group members and none of the novice group members achieved the proficiency benchmark of a minimum of 37 steps completed with 3 or fewer errors.

      Conclusions

      Face validity and content validity for the ARCR metrics, along with construct validity for the metrics and cadaveric shoulder, were verified. A proficiency benchmark was established based on the mean performance of an experienced group of arthroscopic shoulder surgeons.

      Clinical Relevance

      Validated procedural metrics combined with the use of a cadaveric shoulder can be used to accurately assess the performance of an ARCR.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Gallagher A.G.
        Metric-based simulation training to proficiency in medical education:-What it is and how to do it.
        Ulster Med J. 2012; 81: 107-113
        • Gallagher A.G.
        • Ritter E.M.
        • Champion H.
        • et al.
        Virtual reality simulation for the operating room: Proficiency-based training as a paradigm shift in surgical skills training.
        Ann Surg. 2005; 241: 364-372
        • Gallagher A.G.
        • O’Sullivan G.C.
        Chapter 12 - Proficiency-Based Progression Simulation Training: A To-Do List for Medicine.
        in: Fundamentals of surgical simulation: Principles and practices. Springer, London2012: 329
        • Angelo R.L.
        • Ryu R.K.
        • Pedowitz R.A.
        • Gallagher A.G.
        Metric development for an arthroscopic Bankart procedure: Assessment of face and content validity.
        Arthroscopy. 2015; 3: 1430-1440
        • Reiley C.E.
        • Lin H.C.
        • Yuh D.D.
        • Hager G.D.
        Review of methods for objective surgical skill evaluation.
        Surg Endosc. 2011; 25: 356-366
        • Howells N.R.
        • Brinsden M.D.
        • Gill R.S.
        • Carr A.J.
        • Rees J.L.
        Motion analysis: A validated method for showing skill levels in arthroscopy.
        Arthroscopy. 2008; 243: 335-342
        • Angelo R.L.
        Magellan and Copernicus: Arthroscopy Association of North America seeking excellence in education.
        Arthroscopy. 2015; 31: 1428-1429
        • Angelo R.L.
        • Pedowitz R.A.
        • Ryu R.K.
        • Gallagher A.G.
        The Bankart performance metrics combined with a shoulder model simulator create a precise and accurate training tool for measuring surgeon skill.
        Arthroscopy. 2015; 31: 1639-1654
        • Angelo R.L.
        • Ryu R.K.
        • Pedowitz R.A.
        • Gallagher A.G.
        The Bankart performance metrics combined with a cadaveric shoulder create a precise and accurate assessment tool for measuring surgeon skill.
        Arthroscopy. 2015; 31: 1655-1670
        • Angelo R.L.
        • Ryu R.K.
        • Pedowitz R.A.
        • et al.
        A proficiency-based progression training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic Bankart skill set.
        Arthroscopy. 2015; 31: 1854-1871
        • Seymour N.E.
        • Gallagher A.G.
        • Roman S.A.
        • O’Brien M.K.
        • Andersen D.K.
        • Satava R.M.
        Analysis of errors in laparoscopic surgical procedures.
        Surg Endosc. 2004; 18: 592-595
        • Cuschieri A.
        • Francis N.
        • Crosby J.
        • Hanna G.B.
        What do master surgeons think of surgical competence and revalidation?.
        Am J Surg. 2001; 182: 110-116
        • Seymour N.E.
        • Gallagher A.G.
        • Roman S.A.
        • et al.
        Virtual reality training improves operating room performance: Results of a randomized, double-blinded study.
        Ann Surg. 2002; 236: 458-464
        • Ahlberg G.
        • Enochsson L.
        • Gallagher A.G.
        • et al.
        Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies.
        Am J Surg. 2007; 193: 797-804
        • Van Sickle K.R.
        • Ritter E.M.
        • Baghai M.
        • et al.
        Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying.
        J Am Coll Surg. 2008; 207: 560-568
        • Gallagher A.G.
        • Henn P.J.
        • Neary P.C.
        • et al.
        Outlier experienced surgeon’s performances impact on benchmark for technical surgical skills training.
        ANZ J Surg. 2018; 88: E412-E417
        • American Educational Research Association, American Psychological Association, National Council on Measurement in Education
        Standards for educational and psychological testing.
        American Educational Research Association, Washington, D.C.1999
        • Gallagher A.G.
        • Smith C.D.
        • Bowers S.P.
        • et al.
        Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures.
        J Am Coll Surg. 2003; 197: 479-488
        • Hodgins J.L.
        • Veillette C.
        • Biau D.
        • Sonnadara R.
        The knee arthroscopy learning curve: Quantitative assessment of surgical skills.
        Arthroscopy. 2014; 30: 613-621
        • Koehler R.J.
        • Nicandri G.T.
        Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.
        J Bone Joint Surg Am. 2013; 95: e187
        • Angelo R.L.
        Editorial Commentary: “A brush too broad” (validation, global rating scales, and the like . . . what do they really mean?).
        Arthroscopy. 2017; 33: 1004-1006
        • Gallagher A.G.
        • O’Sullivan G.C.
        • Leonard G.
        • Bunting B.P.
        • McGlade K.J.
        Objective structured assessment of technical skills and checklist scales reliability compared for high stakes assessments.
        ANZ J Surg. 2014; 84: 568-573
        • Gallagher A.
        • Ritter E.
        • Satava R.
        Fundamental principles of validation, and reliability: Rigorous science for the assessment of surgical education and training.
        Surg Endosc Intervent Tech. 2003; 17: 1525-1529
        • Vassiliou M.C.
        • Feldman L.S.
        • Andrew C.G.
        A global assessment tool for evaluation of intraoperative laparoscopic skills.
        Am J Surg. 2005; 190: 107-113
        • Cates C.U.
        • Lönn L.
        • Gallagher A.G.
        Prospective, randomised and blinded comparison of proficiency-based progression full-physics virtual reality simulator training versus invasive vascular experience for learning carotid artery angiography by very experienced operators.
        BMJ Simul Technol Enhanc Learn. 2016; 2: 1-5
        • Srinivasan K.K.
        • Gallagher A.
        • O’Brien N.
        • et al.
        Proficiency-based progression training: An ‘end to end’ model for decreasing error applied to achievement of effective epidural analgesia during labour: A randomised control study.
        BMJ. 2018; 8: e020099
        • Birkmeyer J.D.
        • Finks J.F.
        • O’Reilly A.
        • et al.
        Surgical skill and complication rates after bariatric surgery.
        N Engl J Med. 2013; 369: 1434-1442