Intra- and Inter-rater Agreement on Magnetic Resonance Imaging Evaluation of Rotator Cuff Integrity After Repair


      To investigate the intra- and inter-rater agreement of magnetic resonance imaging (MRI) evaluations of rotator cuff integrity at 6 and 24 months after arthroscopic rotator cuff repair (ARCR).


      Three shoulder surgeons reviewed 68 MRI scans from 34 patients who had undergone ARCR and MRI examination at both 6 and 24 months after surgery. Postoperative rotator cuff integrity was investigated by using Owen, Sugaya, and Hayashida classifications to determine whether the rotator cuff was intact or whether there was a partial-thickness retear or full-thickness retear and Burks score to assess tendon appearance. Multirater kappa statistics were used to measure intra- and inter-rater agreement. Kappa values were interpreted according to guidelines adapted from the work of Landis and Koch.


      All classifications had similar intra- and inter-rater agreement (κ = 0.14 to 0.67, 0.23 to 0.60, respectively), but no intra- or inter-rater agreement scored “excellent.” Inter-rater agreement after ARCR was higher at 24 months (κ = 0.31 to 0.60) than at 6 months (κ = 0.23 to 0.44) in all evaluations. Reviewers identified full-thickness retears with a moderate to good degree of inter-rater agreement in all evaluations, at both 6 months (κ = 0.42 to 0.73) and 24 months (κ = 0.61 to 0.80) after ARCR. However, poor inter-rater agreement (κ = 0.13 to 0.19) was found in the identification of partial-thickness retears in all evaluations at 6 months after ARCR.


      Shoulder surgeons showed better intra- and inter-rater agreement in predicting full-thickness tears compared with partial-thickness tears. The inter-rater agreement at 24 months after ARCR was superior to that at 6 months in predicting not only full-thickness retear but also partial-thickness retear. MRI evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used.

      Level of Evidence

      Level III, retrospective comparative study.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Mihata T.
        • Watanabe C.
        • Fukunishi K.
        • et al.
        Functional and structural outcomes of single-row versus double-row versus combined double-row and suture-bridge repair for rotator cuff tears.
        Am J Sports Med. 2011; 39: 2091-2098
        • Sugaya H.
        • Maeda K.
        • Matsuki K.
        • Moriishi J.
        Repair integrity and functional outcome after arthroscopic double-row rotator cuff repair. A prospective outcome study.
        J Bone Joint Surg Am. 2007; 89: 953-960
        • Boileau P.
        • Brassart N.
        • Watkinson D.J.
        • Carles M.
        • Hatzidakis A.M.
        • Krishnan S.G.
        Arthroscopic repair of full-thickness tears of the supraspinatus: does the tendon really heal?.
        J Bone Joint Surg Am. 2005; 87: 1229-1240
        • Harryman 2nd, D.T.
        • Mack L.A.
        • Wang K.Y.
        • Jackins S.E.
        • Richardson M.L.
        • Matsen 3rd, F.A.
        Repairs of the rotator cuff. Correlation of functional results with integrity of the cuff.
        J Bone Joint Surg Am. 1991; 73: 982-989
        • Gazielly D.F.
        • Gleyze P.
        • Montagnon C.
        Functional and anatomical results after rotator cuff repair.
        Clin Orthop Relat Res. 1994; : 43-53
        • Thomazeau H.
        • Boukobza E.
        • Morcet N.
        • Chaperon J.
        • Langlais F.
        Prediction of rotator cuff repair results by magnetic resonance imaging.
        Clin Orthop Relat Res. 1997; : 275-283
        • Gerber C.
        • Fuchs B.
        • Hodler J.
        The results of repair of massive tears of the rotator cuff.
        J Bone Joint Surg Am. 2000; 82: 505-515
        • Galatz L.M.
        • Ball C.M.
        • Teefey S.A.
        • Middleton W.D.
        • Yamaguchi K.
        The outcome and repair integrity of completely arthroscopically repaired large and massive rotator cuff tears.
        J Bone Joint Surg Am. 2004; 86: 219-224
        • Liu S.H.
        • Baker C.L.
        Arthroscopically assisted rotator cuff repair: Correlation of functional results with integrity of the cuff.
        Arthroscopy. 1994; 10: 54-60
        • Jost B.
        • Pfirrmann C.W.
        • Gerber C.
        • Switzerland Z.
        Clinical outcome after structural failure of rotator cuff repairs.
        J Bone Joint Surg Am. 2000; 82: 304-314
        • Park M.C.
        • Elattrache N.S.
        • Ahmad C.S.
        • Tibone J.E.
        “Transosseous-equivalent” rotator cuff repair technique.
        Arthroscopy. 2006; 22: 1360.e1-1360.e5
        • Park M.C.
        • Tibone J.E.
        • ElAttrache N.S.
        • Ahmad C.S.
        • Jun B.J.
        • Lee T.Q.
        Part II: Biomechanical assessment for a footprint-restoring transosseous-equivalent rotator cuff repair technique compared with a double-row repair technique.
        J Shoulder Elbow Surg. 2007; 16: 469-476
        • Mihata T.
        • Fukuhara T.
        • Jun B.J.
        • Watanabe C.
        • Kinoshita M.
        Effect of shoulder abduction angle on biomechanical properties of the repaired rotator cuff tendons with 3 types of double-row technique.
        Am J Sports Med. 2011; 39: 551-556
        • Owen R.S.
        • Iannotti J.P.
        • Kneeland J.B.
        • Dalinka M.K.
        • Deren J.A.
        • Oleaga L.
        Shoulder after surgery: MR imaging with surgical validation.
        Radiology. 1993; 186: 443-447
        • Sugaya H.
        • Maeda K.
        • Matsuki K.
        • Moriishi J.
        Functional and structural outcome after arthroscopic full-thickness rotator cuff repair: Single-row versus dual-row fixation.
        Arthroscopy. 2005; 21: 1307-1316
        • Hayashida K.
        • Tanaka M.
        • Koizumi K.
        • Kakiuchi M.
        Characteristic retear patterns assessed by magnetic resonance imaging after arthroscopic double-row rotator cuff repair.
        Arthroscopy. 2012; 28: 458-464
        • Burks R.T.
        • Crim J.
        • Brown N.
        • Fink B.
        • Greis P.E.
        A prospective randomized clinical trial comparing arthroscopic single- and double-row rotator cuff repair: Magnetic resonance imaging and early clinical evaluation.
        Am J Sports Med. 2009; 37: 674-682
        • Balich S.M.
        • Sheley R.C.
        • Brown T.R.
        • Sauser D.D.
        • Quinn S.F.
        MR imaging of the rotator cuff tendon: Interobserver agreement and analysis of interpretive errors.
        Radiology. 1997; 204: 191-194
        • Robertson P.L.
        • Schweitzer M.E.
        • Mitchell D.G.
        • et al.
        Rotator cuff disorders: Interobserver and intraobserver variation in diagnosis with MR imaging.
        Radiology. 1995; 194: 831-835
        • Singson R.D.
        • Hoang T.
        • Dan S.
        • Friedman M.
        MR evaluation of rotator cuff pathology using T2-weighted fast spin-echo technique with and without fat suppression.
        AJR Am J Roentgenol. 1996; 166: 1061-1065
        • Spencer Jr., E.E.
        • Dunn W.R.
        • Wright R.W.
        • et al.
        Interobserver agreement in the classification of rotator cuff tears using magnetic resonance imaging.
        Am J Sports Med. 2008; 36: 99-103
        • Teefey S.A.
        • Rubin D.A.
        • Middleton W.D.
        • Hildebolt C.F.
        • Leibold R.A.
        • Yamaguchi K.
        Detection and quantification of rotator cuff tears. Comparison of ultrasonographic, magnetic resonance imaging, and arthroscopic findings in seventy-one consecutive cases.
        J Bone Joint Surg Am. 2004; 86: 708-716
        • Grant J.A.
        • Miller B.S.
        • Jacobson J.A.
        • et al.
        Intra- and inter-rater reliability of the detection of tears of the supraspinatus central tendon on MRI by shoulder surgeons.
        J Shoulder Elbow Surg. 2013; 22: 725-731
        • Iannotti J.P.
        • Deutsch A.
        • Green A.
        • et al.
        Time to failure after rotator cuff repair: a prospective imaging study.
        J Bone Joint Surg Am. 2013; 95: 965-971
        • Cho N.S.
        • Yi J.W.
        • Lee B.G.
        • Rhee Y.G.
        Retear patterns after arthroscopic rotator cuff repair: Single-row versus suture bridge technique.
        Am J Sports Med. 2010; 38: 664-671
        • Crim J.
        • Burks R.
        • Manaster B.J.
        • Hanrahan C.
        • Hung M.
        • Greis P.
        Temporal evolution of MRI findings after arthroscopic rotator cuff repair.
        AJR Am J Roentgenol. 2010; 195: 1361-1366
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Khazzam M.
        • Kuhn J.E.
        • Mulligan E.
        • et al.
        Magnetic resonance imaging identification of rotator cuff retears after repair: Interobserver and intraobserver agreement.
        Am J Sports Med. 2012; 40: 1722-1727
        • Gusmer P.B.
        • Potter H.G.
        • Donovan W.D.
        • O'Brien S.J.
        MR imaging of the shoulder after rotator cuff repair.
        AJR Am J Roentgenol. 1997; 168: 559-563
        • Iannotti J.P.
        • Zlatkin M.B.
        • Esterhai J.L.
        • Kressel H.Y.
        • Dalinka M.K.
        • Spindler K.P.
        Magnetic resonance imaging of the shoulder. Sensitivity, specificity, and predictive value.
        J Bone Joint Surg Am. 1991; 73: 17-29
        • Quinn S.F.
        • Sheley R.C.
        • Demlow T.A.
        • Szumowski J.
        Rotator cuff tendon tears: evaluation with fat-suppressed MR imaging with arthroscopic correlation in 100 patients.
        Radiology. 1995; 195: 497-500
        • Reinus W.R.
        • Shady K.L.
        • Mirowitz S.A.
        • Totty W.G.
        MR diagnosis of rotator cuff tears of the shoulder: Value of using T2-weighted fat-saturated images.
        AJR Am J Roentgenol. 1995; 164: 1451-1455
        • Ferrari F.S.
        • Governi S.
        • Burresi F.
        • Vigni F.
        • Stefani P.
        Supraspinatus tendon tears: Comparison of US and MR arthrography with surgical correlation.
        Eur Radiol. 2002; 12: 1211-1217
        • Toyoda H.
        • Ito Y.
        • Tomo H.
        • Nakao Y.
        • Koike T.
        • Takaoka K.
        Evaluation of rotator cuff tears with magnetic resonance arthrography.
        Clin Orthop Relat Res. 2005; 439: 109-115
        • de Jesus J.O.
        • Parker L.
        • Frangos A.J.
        • Nazarian L.N.
        Accuracy of MRI, MR arthrography, and ultrasound in the diagnosis of rotator cuff tears: A meta-analysis.
        AJR Am J Roentgenol. 2009; 192: 1701-1707