Advertisement

Editorial Commentary: Proficiency-Based Progression Surgical Simulation Training Is an Efficient Adjunct to High-Volume Clinical Experience

      Abstract

      The optimal way to train a future surgeon has been debated for years, with strategies ranging from the well-known “see one, do one, teach one” approach to more novel approaches that rely on metrics and proficiency. Recent research shows that surgical training with a proficiency-based progression curriculum is an efficient strategy for teaching arthroscopy procedural skills, and, further, may improve patient safety by reducing the technical errors that might otherwise occur before proficiency is achieved. While every surgical specialty has its nuances that must be mastered to provide safe, effective, and efficient care, for a variety of reasons, the skills needed to perform arthroscopy are incredibly difficult to learn, let alone achieve proficiency or master. “On-the-job” training for orthopaedic residents has become more difficult in today’s fast-paced, work hour–limited, volume-rewarded society. Proficiency-based progression is a piece of the puzzle, but for now, it is not a complete substitute for high-volume, clinical experience and exposure to the countless variables that may affect a "real-life" surgical procedure.
      The optimal way to train a future surgeon has been debated for years, with strategies ranging from the well-known “see one, do one, teach one” approach to more novel approaches that rely on metrics and proficiency. In this month’s issue, Angelo, St. Pierre, and Tauro, and Gallagher attempt to analyze the impact of a proficiency-based progression (PBP) curriculum on surgical trainee ability to demonstrate proficiency for performing shoulder arthroscopy procedures in their article, “A Proficiency-Based Progression Simulation Training Curriculum to Acquire the Skills Needed in Performing Arthroscopic Bankart and Rotator Cuff Repairs—Implementation and Impact.”
      • Angelo R.L.
      • St. Pierre P.
      • Tauro J.
      • Gallagher A.G.
      A proficiency-based progression simulation training curriculum to acquire the skills needed in performing arthroscopic Bankart and rotator cuff repairs— implementation and impact.
      The authors conducted a prospective, randomized study involving 18 orthopaedic trainees (16 fellows, 2 senior residents) undergoing a PBP training curriculum and assessed their ability to perform an arthroscopic Bankart repair (ABR) and arthroscopic rotator cuff repair (ARCR). All trainees underwent a pre-course after being randomly assigned to cadaveric ABR (Bankart subgroup, N = 6), cadaveric ARCR (cuff subgroup, N = 6), or basic skills on a shoulder simulator (N = 6). After going through the PBP curriculum over the course of 2.5 days at the Orthopedic Learning Center (Rosemont, IL), subjects were assessed on their ability to perform cadaveric ABR and ARCR in real-time by trained orthopaedic surgeons using previously validated metrics.
      The authors found that both the Bankart and cuff subgroups made significantly fewer errors at the completion of the course compared with baseline assessments (Bankart subgroup: 58% fewer; cuff subgroup: 58% fewer). In addition, the authors found that proficiency was achieved by 89% (ABR) and 83% (ARCR) of the trainees by the end of the course. The authors concluded that surgical training with a PBP curriculum is an efficient strategy for teaching shoulder arthroscopy procedural skills, and, further, may improve patient safety by reducing the technical errors that might otherwise occur before proficiency is achieved.
      Over the past decade, it has been extremely challenging, if not downright impossible, for surgical trainees to keep up with the ever-evolving field of arthroscopic surgery, particularly given the rapid advancements in techniques, implants, and approaches. While every surgical specialty has its nuances that must be mastered to provide safe, effective, and efficient care, for a variety of reasons, the skills needed to perform arthroscopy are incredibly difficult to learn, let alone achieve proficiency or master. “On-the-job” training for orthopaedic residents has become more difficult in today’s fast-paced, volume-rewarded society, and in 2020, hands-on arthroscopic learning essentially came to halt with the shutdown of elective procedures due to the coronavirus disease 2019 (COVID-19) pandemic.
      Surgical educators have been searching for strategies to optimize surgical training for residents/fellows for years, with efforts focused on patient safety and surgical efficiency. While work in this area goes back several decades,
      • Ahlberg G.
      • Enochsson L.
      • Gallagher A.G.
      • et al.
      Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies.
      • Seymour N.E.
      • Gallagher A.G.
      • Roman S.A.
      • et al.
      Virtual reality training improves operating room performance: Results of a randomized, double-blinded study.
      • Van Sickle K.R.
      • Ritter E.M.
      • Baghai M.
      • et al.
      Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying.
      • Van Sickle K.R.
      • Ritter E.M.
      • McClusky 3rd, D.A.
      • et al.
      Attempted establishment of proficiency levels for laparoscopic performance on a national scale using simulation: The results from the 2004 SAGES Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) learning center study.
      in medicine, a renewed research focus on improving surgical training with simulation really took off after implementation of the 2003 Accreditation Council for Graduate Medical Education 80-hour work week restrictions for residents, and specific to orthopaedics, after the implementation of the 2013 Accreditation Council for Graduate Medical Education–mandated surgical skills curriculum.
      • Dougherty P.J.
      • Marcus R.E.
      ACGME and ABOS changes for the orthopaedic surgery PGY-1 (intern) year.
      Along with basic surgical skill development, arthroscopic skills became a natural area of focus, likely the result of arthroscopy being somewhat similar to laparoscopy (with an already-developed surgical skillset curriculum in general surgery
      • Sroka G.
      • Feldman L.S.
      • Vassiliou M.C.
      • Kaneva P.A.
      • Fayez R.
      • Fried G.M.
      Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial.
      ), as well as the availability and interest in arthroscopic simulators
      While substantial attention has been given to determining which simulator models are most effective,
      • Frank R.M.
      • Erickson B.
      • Frank J.M.
      • et al.
      Utility of modern arthroscopic simulator training models.
      ,
      • Frank R.M.
      • Wang K.C.
      • Davey A.
      • et al.
      Utility of modern arthroscopic simulator training models: A meta-analysis and updated systematic review.
      over the last decade, Angelo et al. have focused their efforts more on the curriculum itself, as opposed to the specific simulator. Specifically, in a variety of studies conducted under the Copernicus Initiative, Angelo et al. have taken a stepwise approach to developing a PBP curriculum for teaching shoulder arthroscopy skills, as well validating the metrics used in such a program
      • Angelo R.L.
      • Pedowitz R.A.
      • Ryu R.K.
      • Gallagher A.G.
      The Bankart performance metrics combined with a shoulder model simulator create a precise and accurate training tool for measuring surgeon skill.
      • Angelo R.L.
      • Ryu R.K.
      • Pedowitz R.A.
      • et al.
      A proficiency-based progression training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic Bankart skill set.
      • Angelo R.L.
      • Ryu R.K.
      • Pedowitz R.A.
      • Gallagher A.G.
      Metric development for an arthroscopic Bankart procedure: Assessment of face and content validity.
      • Angelo R.L.
      • Ryu R.K.
      • Pedowitz R.A.
      • Gallagher A.G.
      The Bankart performance metrics combined with a cadaveric shoulder create a precise and accurate assessment tool for measuring surgeon skill.
      • Pedowitz R.A.
      • Nicandri G.T.
      • Angelo R.L.
      • Ryu R.K.
      • Gallagher A.G.
      Objective assessment of knot-tying proficiency with the fundamentals of arthroscopic surgery training program workstation and knot tester.
      In brief, a PRP program differs from traditional apprenticeship programs, as trainees are required to demonstrate mastery of specific skills before progressing to more advanced skills.
      • Angelo R.L.
      Magellan and Copernicus: Arthroscopy Association of North America seeking excellence in education.
      In their most recent study, Angelo et al. determined that a PBP for shoulder arthroscopy skills, specifically Bankart repair and rotator cuff repair, is effective with respect to reducing technical errors and developing surgical proficiency. The amount of planning that went into such a study, along with the investigators being able to successfully operationalize the protocol in a single setting over the course of 2.5 days, is to be commended. The findings are incredibly impactful—if a PBP curriculum can be successfully implemented over the course of 2.5 days (or fewer), perhaps these methods and metrics can be applied to other arthroscopic skills. Certainly, the ability to send surgical trainees to a course, or better yet, have the simulator and PBP curriculum/materials/simulator available at the trainee’s home program, and have the trainee demonstrate proficiency in a skill before being allowed to perform that skill on an actual patient in the operating room, is attractive. Not only does this have the potential to improve surgical efficiency, but also, improve patient safety.
      While everyone can agree that improving efficiency while improving patient safety is ideal, certain questions still remain. In theory, PBP programs would not require a minimum number of reps, hours, or cases for trainees to demonstrate proficiency—the trainees simply need to pass the test. That test, in this study, was performing the skill on a cadaveric specimen. While certainly operating on a cadaveric specimen is the closest opportunity we have to replicating the actual surgical environment, no cadaveric specimen can ever truly take the place of operating on an actual patient. Particularly with arthroscopy, there are countless variables to account for, including (but not limited to) blood pressure, body habitus, variable anatomy, and fluid distension, among many others. In such cases where the patient’s anatomy and/or pathology are not “textbook,” it is the surgeon’s personal experience that often allows for safe, efficient, and successful completion of the procedure. The question then becomes—how much experience is needed to perform a safe and efficient surgical procedure? The famous “10,000 hour rule” as popularized by Malcolm Gladwell immediately comes to mind, although interestingly, several studies have debated its validity.
      • Macnamara B.N.
      • Hambrick D.Z.
      • Oswald F.L.
      Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis.
      In a PBP curriculum, in theory, a resident with minimal arthroscopic experience could pass the test and be deemed proficient to perform a specific arthroscopic procedure but have no idea how to react to or manage an intraoperative complication because he/she, quite simply, hasn’t ever seen one. Thinking more broadly, if I am a patient undergoing a rotator cuff repair, do I want a fellow who is deemed proficient at ARCR doing my surgery, or an attending surgeon who has performed hundreds if not thousands of ARCR cases? Admittedly, the answer to this question isn’t clear-cut, as experience is not the only factor that matters, but it is one we must consider in addition to proficiency. It would be interesting to repeat the current study with surgeons who have been in practice for 2, 5, and/or 10 years. It would be even more interesting for the trainees in this study, once deemed proficient (and thus safe), to be graded in a blinded fashion on performing the procedure on a live patient, although certainly, this would be logistically (and potentially ethically) challenging.
      In summary, Angelo et al. are to be commended for their incredible efforts on this study, as this type of research provides novel strategies for more effectively teaching our surgeons-in-training, which will ultimately serve patients for generations to come. The value of PBP training on real-life patient outcomes is still to be determined, but for now, it is safe to say that PBP is an attractive strategy for teaching arthroscopic surgical skills to residents and fellows.

      Supplementary Data

      References

        • Angelo R.L.
        • St. Pierre P.
        • Tauro J.
        • Gallagher A.G.
        A proficiency-based progression simulation training curriculum to acquire the skills needed in performing arthroscopic Bankart and rotator cuff repairs— implementation and impact.
        Arthroscopy. 2021; 37: 1099-1106
        • Ahlberg G.
        • Enochsson L.
        • Gallagher A.G.
        • et al.
        Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies.
        Am J Surg. 2007; 193: 797-804
        • Seymour N.E.
        • Gallagher A.G.
        • Roman S.A.
        • et al.
        Virtual reality training improves operating room performance: Results of a randomized, double-blinded study.
        Ann Surg. 2002; 236 (discussion 463-454): 458-463
        • Van Sickle K.R.
        • Ritter E.M.
        • Baghai M.
        • et al.
        Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying.
        J Am Coll Surg. 2008; 207: 560-568
        • Van Sickle K.R.
        • Ritter E.M.
        • McClusky 3rd, D.A.
        • et al.
        Attempted establishment of proficiency levels for laparoscopic performance on a national scale using simulation: The results from the 2004 SAGES Minimally Invasive Surgical Trainer-Virtual Reality (MIST-VR) learning center study.
        Surg Endosc. 2007; 21: 5-10
        • Dougherty P.J.
        • Marcus R.E.
        ACGME and ABOS changes for the orthopaedic surgery PGY-1 (intern) year.
        Clin Orthop Relat Res. 2013; 471: 3412-3416
        • Sroka G.
        • Feldman L.S.
        • Vassiliou M.C.
        • Kaneva P.A.
        • Fayez R.
        • Fried G.M.
        Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial.
        Am J Surg. 2010; 199: 115-120
        • Frank R.M.
        • Erickson B.
        • Frank J.M.
        • et al.
        Utility of modern arthroscopic simulator training models.
        Arthroscopy. 2014; 30: 121-133
        • Frank R.M.
        • Wang K.C.
        • Davey A.
        • et al.
        Utility of modern arthroscopic simulator training models: A meta-analysis and updated systematic review.
        Arthroscopy. 2018; 34: 1650-1677
        • Angelo R.L.
        • Pedowitz R.A.
        • Ryu R.K.
        • Gallagher A.G.
        The Bankart performance metrics combined with a shoulder model simulator create a precise and accurate training tool for measuring surgeon skill.
        Arthroscopy. 2015; 31: 1639-1654
        • Angelo R.L.
        • Ryu R.K.
        • Pedowitz R.A.
        • et al.
        A proficiency-based progression training curriculum coupled with a model simulator results in the acquisition of a superior arthroscopic Bankart skill set.
        Arthroscopy. 2015; 31: 1854-1871
        • Angelo R.L.
        • Ryu R.K.
        • Pedowitz R.A.
        • Gallagher A.G.
        Metric development for an arthroscopic Bankart procedure: Assessment of face and content validity.
        Arthroscopy. 2015; 31: 1430-1440
        • Angelo R.L.
        • Ryu R.K.
        • Pedowitz R.A.
        • Gallagher A.G.
        The Bankart performance metrics combined with a cadaveric shoulder create a precise and accurate assessment tool for measuring surgeon skill.
        Arthroscopy. 2015; 31: 1655-1670
        • Pedowitz R.A.
        • Nicandri G.T.
        • Angelo R.L.
        • Ryu R.K.
        • Gallagher A.G.
        Objective assessment of knot-tying proficiency with the fundamentals of arthroscopic surgery training program workstation and knot tester.
        Arthroscopy. 2015; 31: 1872-1879
        • Angelo R.L.
        Magellan and Copernicus: Arthroscopy Association of North America seeking excellence in education.
        Arthroscopy. 2015; 31: 1428-1429
        • Macnamara B.N.
        • Hambrick D.Z.
        • Oswald F.L.
        Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis.
        Psychol Sci. 2014; 25: 1608-1618