Advertisement

The Knee Arthroscopy Learning Curve: Quantitative Assessment of Surgical Skills

      Purpose

      To assess orthopaedic trainees performing diagnostic knee arthroscopies and evaluate procedural competence using a novel statistical method, the Cumulative Summation Test for Learning Curve (LC-CUSUM).

      Methods

      Twenty orthopaedic trainees in postgraduate year (PGY) 1 through 5 performed diagnostic knee arthroscopy and were evaluated intraoperatively with a validated 10-point knee task-specific checklist (TSCL) and 50-point global rating scale (GRS). A score of 40 points or greater (of 50 points) for the GRS and 8 points or greater (of 10 points) for the knee TSCL was considered to indicate a successful procedure. For the LC-CUSUM analysis, adequate performance was defined as a 10% failure rate, inadequate performance was defined as a 30% failure rate, and an acceptable deviation from adequate performance was defined as 10%. A limit h equal to 1.6 was selected to give a true-discovery rate of 90% and a false-discovery rate of 10% over 50 procedures.

      Results

      A total of 340 consecutive procedures were performed by 20 trainees during the 12-month study period. The cumulative number of arthroscopic procedures performed by trainees before study start increased with increasing PGY. The median number of arthroscopic procedures performed per trainee was 16.5 (interquartile range, 14 to 21.75). Competency in knee arthroscopy for the TSCL was achieved by 8 trainees (40%), after a median of 16 procedures (interquartile range, 13 to 20), and for the GRS by only a single trainee (5%), after 14 procedures. Threshold-adjusted curves stratified by PGY level enabled multiple trainees to achieve competency for both the TSCL and GRS.

      Conclusions

      The LC-CUSUM can be successfully applied to knee arthroscopy to provide an individualized assessment of performance and quantitatively demonstrate competency for basic arthroscopic tasks.

      Clinical Relevance

      The LC-CUSUM is an effective method to evaluate procedure competence in arthroscopic training and can provide objective feedback and benchmarks in the learning phase.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Farnworth L.R.
        • Lemay D.E.
        • Wooldridge T.
        • et al.
        A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: The financial impact of orthopaedic surgical training in the operating room.
        Iowa Orthop J. 2001; 21: 31-35
        • Mabrey J.D.
        • Gillogly S.D.
        • Kasser J.R.
        • et al.
        Virtual reality simulation of arthroscopy of the knee.
        Arthroscopy. 2002; 18: E28
        • Pedowitz R.A.
        • Esch J.
        • Snyder S.
        Evaluation of a virtual reality simulator for arthroscopy skills development.
        Arthroscopy. 2002; 18: E29
        • Irani J.L.
        • Mello M.M.
        • Ashley S.W.
        • Whang E.E.
        • Zinner M.J.
        • Breen E.
        Surgical residents' perceptions of the effects of the ACGME duty hour requirements 1 year after implementation.
        Surgery. 2005; 138: 246-253
        • Zuckerman J.D.
        • Kubiak E.N.
        • Immerman I.
        • Dicesare P.
        The early effects of code 405 work rules on attitudes of orthopaedic residents and attending surgeons.
        J Bone Joint Surg Am. 2005; 87: 903-908
        • O'Neill P.J.
        • Cosgarea A.J.
        • Freedman J.A.
        • Queale W.S.
        • McFarland E.G.
        Arthroscopic proficiency: A survey of orthopaedic sports medicine fellowship directors and orthopaedic surgery department chairs.
        Arthroscopy. 2002; 18: 795-800
        • van Hove P.D.
        • Tuijthof G.J.
        • Verdaasdonk E.G.
        • Stassen L.P.
        • Dankelman J.
        Objective assessment of technical surgical skills.
        Br J Surg. 2010; 97: 972-987
        • Darzi A.
        • Smith S.
        • Taffinder N.
        Assessing operative skill. Needs to become more objective.
        BMJ. 1999; 318: 887-888
        • Elliott M.J.
        • Caprise P.A.
        • Henning A.E.
        • Kurtz C.A.
        • Sekiya J.K.
        Diagnostic knee arthroscopy: A pilot study to evaluate surgical skills.
        Arthroscopy. 2012; 28: 218-224
        • Insel A.
        • Carofino B.
        • Leger R.
        • Arciero R.
        • Mazzocca A.D.
        The development of an objective model to assess arthroscopic performance.
        J Bone Joint Surg Am. 2009; 91: 2287-2295
        • Sonnadara R.R.
        • Van Vliet A.
        • Safir O.
        • et al.
        Orthopedic boot camp: Examining the effectiveness of an intensive surgical skills course.
        Surgery. 2011; 149: 745-749
        • Dessolle L.
        • Fréour T.
        • Barrière P.
        • et al.
        How soon can I be proficient in embryo transfer? Lessons from the cumulative summation test for learning curve (LC-CUSUM).
        Hum Reprod. 2010; 25: 380-386
        • Jowell P.S.
        • Baillie J.
        • Branch M.S.
        • Affronti J.
        • Browning C.L.
        • Bute B.P.
        Quantitative assessment of procedural competence. A prospective study of training in endoscopic retrograde cholangiopancreatography.
        Ann Intern Med. 1996; 125: 983-989
        • Ramesha P.
        • Biau D.
        • Mann L.
        • Johnson A.
        • Moise Jr., K.J.
        Use of the Learning Curve–Cumulative Summation test for quantitative and individualized assessment of competency of a surgical procedure in obstetrics and gynecology: Fetoscopic laser ablation as a model.
        Am J Obstet Gynecol. 2011; 204: 218.e1-218.e9
        • Salowi M.A.
        • Choong Y.F.
        • Goh P.P.
        • Ismail M.
        • Lim T.O.
        CUSUM: A dynamic tool for monitoring competency in cataract surgery performance.
        Br J Ophthalmol. 2010; 94: 445-449
        • Biau D.J.
        • Porcher R.
        A method for monitoring a process from an out of control to an in control state: Application to the learning curve.
        Stat Med. 2010; 29: 1900-1909
        • Biau D.J.
        • Williams S.M.
        • Schlup M.
        • Nizard R.S.
        • Porcher R.
        Quantitative and individualized assessment of the learning curve using LC-CUSUM.
        Br J Surg. 2008; 95: 925-929
        • Balsyte D.
        • Schäffer L.
        • Burkhardt T.
        • Wisser J.
        • Zimmermann R.
        • Kurmanavicius J.
        Continuous independent quality control for fetal ultrasound biometry provided by the cumulative summation technique.
        Ultrasound Obstet Gynecol. 2010; 35: 449-455
        • Trajkovski T.
        • Veillette C.
        • Backstein D.
        • Wadey V.M.
        • Kraemer B.
        Resident self-assessment of operative experience in primary total knee and total hip arthroplasty: Is it accurate?.
        Can J Surg. 2012; 55: S153-S157
        • Slade Shantz J.A.
        • Leiter J.R.
        • Collins J.B.
        • Macdonald P.B.
        Validation of a global assessment of arthroscopic skills in a cadaveric knee model.
        Arthroscopy. 2013; 29: 106-112
        • Gomoll A.H.
        • Pappas G.
        • Forsythe B.
        • Warner J.J.
        Individual skill progression on a virtual reality simulator for shoulder arthroscopy: A 3-year follow-up study.
        Am J Sports Med. 2008; 36: 1139-1142
        • Howells N.R.
        • Gill H.S.
        • Carr A.J.
        • Price A.J.
        • Rees J.L.
        Transferring simulated arthroscopic skills to the operating theatre: A randomised blinded study.
        J Bone Joint Surg Br. 2008; 90: 494-499
        • Martin K.D.
        • Belmont P.J.
        • Schoenfeld A.J.
        • Todd M.
        • Cameron K.L.
        • Owens B.D.
        Arthroscopic basic task performance in shoulder simulator model correlates with similar task performance in cadavers.
        J Bone Joint Surg Am. 2011; 93: e1271-e1275
        • McCarthy A.
        • Harley P.
        • Smallwood R.
        Virtual arthroscopy training: Do the “virtual skills” developed match the real skills required?.
        Stud Health Technol Inform. 1999; 62: 221-227
        • Michelson J.D.
        Simulation in orthopaedic education: An overview of theory and practice.
        J Bone Joint Surg Am. 2006; 88: 1405-1411
        • Srivastava S.
        • Youngblood P.L.
        • Rawn C.
        • Hariri S.
        • Heinrichs W.L.
        • Ladd A.L.
        Initial evaluation of a shoulder arthroscopy simulator: Establishing construct validity.
        J Shoulder Elbow Surg. 2004; 13: 196-205
        • Koehler R.J.
        • Nicandri G.T.
        Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.
        J Bone Joint Surg Am. 2013; 95: e1871-e1876
        • Jackson W.F.
        • Khan T.
        • Alvand A.
        • Al-Ali S.
        • Gill H.S.
        • Price A.J.
        • et al.
        Learning and retaining simulated arthroscopic meniscal repair skills.
        J Bone Joint Surg Am. 2012; 94: e132
        • Alvand A.
        • Logishetty K.
        • Middleton R.
        • et al.
        Validating a global rating scale to monitor individual resident learning curves during arthroscopic knee meniscal repair.
        Arthroscopy. 2013; 29: 906-912
        • Hall M.P.
        • Kaplan K.M.
        • Gorczynski C.T.
        • Zuckerman J.D.
        • Rosen J.E.
        Assessment of arthroscopic training in U.S. orthopedic surgery residency programs—A resident self-assessment.
        Bull NYU Hosp Jt Dis. 2010; 68: 5-10
        • Alvand A.
        • Auplish S.
        • Gill H.
        • Rees J.
        Innate arthroscopic skills in medical students and variation in learning curves.
        J Bone Joint Surg Am. 2011; 93 (e115(1-9))
        • Alvand A.
        • Auplish S.
        • Khan T.
        • Gill H.S.
        • Rees J.L.
        Identifying orthopaedic surgeons of the future: The inability of some medical students to achieve competence in basic arthroscopic tasks despite training: A randomised study.
        J Bone Joint Surg Br. 2011; 93: 1586-1591
        • Alvand A.
        • Khan T.
        • Al-Ali S.
        • Jackson W.F.
        • Price A.J.
        • Rees J.L.
        Simple visual parameters for objective assessment of arthroscopic skill.
        J Bone Joint Surg Am. 2012; 94: e97
        • Olson T.
        • Koehler R.
        • Butler A.
        • Amsdell S.
        • Nicandri G.
        Is there a valid and reliable assessment of diagnostic knee arthroscopy skill?.
        Clin Orthop Relat Res. 2013; 471: 1670-1676
        • Koehler R.J.
        • Amsdell S.
        • Arendt E.A.
        • et al.
        The Arthroscopic Surgical Skill Evaluation Tool (ASSET).
        Am J Sports Med. 2013; 41: 1229-1237