- Email: [email protected]
Intra- and Inter-rater Agreement on Magnetic Resonance Imaging Evaluation of Rotator Cuff Integrity After Repair
Intra- and Inter-rater Agreement on Magnetic Resonance Imaging Evaluation of Rotator Cuff Integrity After Repair Akihiko Hasegawa, M.D., Ph.D., Teruhisa Mihata, M.D., Ph.D., Kenji Yasui, M.D., Ph.D., Takeshi Kawakami, M.D., Ph.D., Yasuo Itami, M.D., and Masashi Neo, M.D., Ph.D.
Purpose: To investigate the intra- and inter-rater agreement of magnetic resonance imaging (MRI) evaluations of rotator cuff integrity at 6 and 24 months after arthroscopic rotator cuff repair (ARCR). Methods: Three shoulder surgeons reviewed 68 MRI scans from 34 patients who had undergone ARCR and MRI examination at both 6 and 24 months after surgery. Postoperative rotator cuff integrity was investigated by using Owen, Sugaya, and Hayashida classifications to determine whether the rotator cuff was intact or whether there was a partial-thickness retear or full-thickness retear and Burks score to assess tendon appearance. Multirater kappa statistics were used to measure intra- and inter-rater agreement. Kappa values were interpreted according to guidelines adapted from the work of Landis and Koch. Results: All classifications had similar intra- and inter-rater agreement (k ¼ 0.14 to 0.67, 0.23 to 0.60, respectively), but no intra- or inter-rater agreement scored “excellent.” Inter-rater agreement after ARCR was higher at 24 months (k ¼ 0.31 to 0.60) than at 6 months (k ¼ 0.23 to 0.44) in all evaluations. Reviewers identified full-thickness retears with a moderate to good degree of inter-rater agreement in all evaluations, at both 6 months (k ¼ 0.42 to 0.73) and 24 months (k ¼ 0.61 to 0.80) after ARCR. However, poor inter-rater agreement (k ¼ 0.13 to 0.19) was found in the identification of partial-thickness retears in all evaluations at 6 months after ARCR. Conclusions: Shoulder surgeons showed better intra- and inter-rater agreement in predicting full-thickness tears compared with partial-thickness tears. The inter-rater agreement at 24 months after ARCR was superior to that at 6 months in predicting not only full-thickness retear but also partial-thickness retear. MRI evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used. Level of Evidence: Level III, retrospective comparative study.
R
otator cuff tear is a common injury of the shoulder joint and often requires surgical repair. Arthroscopic rotator cuff repair (ARCR) has become a popular treatment for these tears. The improved integrity of the repair site after rotator cuff repair has been reported to give a better functional outcome,1-7 although some From the Department of Orthopedic Surgery, Osaka Medical College (A.H., T.M., Y.I., M.N.), Takatsuki; Department of Orthopedic Surgery, First Towakai Hospital (A.H., T.M., Y.I.), Takatsuki; Department of Orthopedic Surgery, Nishinomiya Kyoritsu Neurosurgical Hospital (K.Y.), Nishinomiya; and Department of Orthopedic Surgery, Shiroyama Hospital (T.K.), Osaka, Japan. The authors report that they have no conflicts of interest in the authorship and publication of this article. The Institutional Review Board at Osaka Medical College approved the protocol of this study. Received December 10, 2015; accepted April 21, 2016. Address correspondence to Akihiko Hasegawa, M.D., Ph.D., Department of Orthopedic Surgery, Osaka Medical College, 2-7 Daigaku-machi, Takatsuki, Osaka, Japan. E-mail: [email protected] Ó 2016 by the Arthroscopy Association of North America 0749-8063/151135/$36.00 http://dx.doi.org/10.1016/j.arthro.2016.04.027
investigators have shown that structural outcome is not correlated with functional outcome.8-10 The doublerow suture anchor technique and the suture-bridge technique have been reported to improve the biomechanical construct with less tendon retear rate.1,11-13 These techniques, however, have not been sufficient to prevent development of retears. Therefore, retears after repair have been a matter of concern to shoulder surgeons. Recently, magnetic resonance imaging (MRI) has become commonly used to assess rotator cuff integrity before and after arthroscopic repair, and classification and scoring systems have been established.14-17 Several studies have reported on inter-rater agreement in the assessment of rotator cuff integrity before surgery.18-23 It has been reported that radiologists and/or orthopaedic surgeons had good agreement for predicting fullthickness rotator cuff tears but variable (poor to good) agreement in predicting the partial-thickness tear before surgery.18-21,23 However, few studies have assessed the inter- and intraobserver reliability of these
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol
-,
No
-
(Month), 2016: pp 1-8
1
2
A. HASEGAWA ET AL.
classification and scoring systems. It is important to know the degree of reliability of MRI assessment after ARCR to decide the postoperative management of patients who underwent ARCR. Rotator cuff retears occur primarily between 6 and 26 weeks after ARCR, and few additional tears occur thereafter.24 Some studies have used MRI images taken at 6 months after surgery to compare functional outcomes and repair integrity.3,16,25 However, it has been reported that MRI appearance of the rotator cuff after arthroscopic repair showed considerable variability during the first postoperative year and did not correlate with outcome. It has been suggested that it was not prudent to consider the tendon repair as failed according to tendon irregularity, thinning, or increased signal intensity during the first operative year.26 Therefore, the purpose of this study was to investigate the intra- and inter-rater agreement of MRI evaluations of rotator cuff integrity at 6 and 24 months after ARCR. Our hypotheses were that the agreement for assessment of full-thickness retear was higher than that of partial-thickness retear and that intra- and inter-rater agreements after ARCR were higher at 24 months than those at 6 months.
Methods Patient Selection The institutional review board at our university approved the protocol of this study. Potential subjects were identified retrospectively through a review of one surgeon’s database from 2006 to 2010. The inclusion criteria were as follows: complete rotator cuff tear confirmed during arthroscopic surgery; complete repair (complete tuberosity coverage attainable); minimum 2 years of follow-up; and MRI for evaluation of the integrity of the rotator cuff tendons at both 6 and 24 months after surgery. Revision surgery was not included in this study. We retrospectively reviewed our database and initially included 88 patients who, between 2006 and 2010, had received arthroscopic rotator cuff repairs from a single experienced shoulder surgeon (T.M.) using the double-row technique or a combination of double-row and suture-bridge at our University.1,11-13 The titanium suture anchors (diameter, 5 mm; Corkscrew, Arthrex) with two No. 2 permanent sutures (FiberWire, Arthrex) were placed for all surgical procedures. No PEEK (polyether ether ketone)/ bioabsorbable anchor was placed. Of these 88 patients, 34 did not complete the 2-year follow-up and 20 did not undergo postoperative MRI scans at either 6 or 24 months after surgery. Consequently, our final study group consisted of 68 MRI scans of 34 shoulders from 34 patients (20 female and 14 male; mean age, 68.1 years [range, 57 to 81 years]; Table 1).
Table 1. Summary of Patients Patient 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
Age (years) 62 79 81 69 60 63 75 66 73 62 59 66 65 65 59 80 80 63 65 62 71 63 71 71 73 69 64 68 69 68 79 57 70 70
Gender Female Male Male Female Female Male Female Male Female Female Male Female Male Male Female Female Female Male Female Male Male Female Female Male Female Female Female Female Female Male Female Female Male Male
Tear Size Large Small Medium Medium Medium Medium Medium Medium Medium Large Medium Large Medium Large Medium Large Medium Medium Medium Medium Medium Large Large Medium Medium Medium Medium Medium Medium Large Small Small Small Small
Type of Repair DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR DR DR DR
DR, double-row technique; DR and SB, a combination of doublerow and suture-bridge.
Magnetic Resonance Imaging Magnetic resonance imaging was performed with a 1.5-Tesla closed-type scanner (MRT-2000/V2; Toshiba, Tokyo, Japan). Oblique coronal, oblique sagittal, and axial T2-weighted images were acquired for structural and qualitative assessment of the rotator cuff tendon, and repair integrity was determined. The slice thickness was 4 mm, with an interslice gap of 0.5 mm. Postoperative cuff integrity was evaluated according to 4 previously established methods. One of the authors (A.H.) reviewed the clinical data and MRI images. Patients included in the study were considered by this author to meet the criteria of having good-quality MRI images of the shoulder joint and rotator cuff at both 6 and 24 months after ARCR. The author who selected the MRI study sets did not evaluate any of the MRI images. MRI Evaluations Three orthopaedic shoulder surgeons (K.Y., T.K., Y.I.) independently reviewed 68 MRI scans of 34 shoulders
ROTATOR CUFF REPAIR MRI AGREEMENT
3
Fig 1. Representative images of intact rotator cuff after repair. Coronal oblique cut (A) and sagittal oblique cut (B) from magnetic resonance imaging scan of right shoulder with intact rotator cuff after arthroscopic rotator cuff repair. The intact rotator cuff tendon shows sufficient thickness with homogenously low intensity. Among the 3 shoulder surgeons, there was complete agreement for all classifications in this case.
from 34 patients. All 3 orthopaedic shoulder surgeons were specialists of orthopaedic surgery certified by the Japanese Orthopaedic Association. Additionally, they had completed at least 1-year fellowship in shoulder surgery under the mentor. The shoulder surgeons were asked to evaluate each of the 68 scans by using all 4 classification schemes and returned scores for each scan. After an interval of 1 month, they were asked to evaluate each of the same 68 scans in a different random order and returned scores for each scan as second evaluations. They did not have access to clinical data or prior evaluations of the scans, and they did not know when the MRI scans were taken after ARCR. The MRI data were randomized onto an Excel data sheet (Microsoft, Redmond, WA), and all clinical data were dissociated from the MRI data for each patient. MRI images were presented individually to each surgeon on the Digital Imaging and COmmunication in Medicine (DICOM) imaging software, OsiriX (Pixmeo Sarl, Switzerland).
Postoperative rotator cuff integrity was investigated by using 4 different evaluation methods, namely, the Owen, Sugaya, and Hayashida classifications to determine whether the cuff was intact or whether there was a partial- or full-thickness retear (Figs 1-3) and Burks’ score to assess tendon appearance, including footprint coverage, tendon thickness, and signal intensity.14-17 Criteria for the 4 classification schemes were summarized in Table 2. Briefly, Owen et al.14 identified partial-thickness tears by increased signal intensity within the rotator cuff on spin-density and T2-weighted sequences. A fullthickness rotator cuff tear was diagnosed when fluidlike signal intensity on T2-weighted images extended through the entire thickness of the rotator cuff, or when complete nonvisualization of a portion of the rotator cuff was noted on 1 or more sections. Rotator cuffs without such evidence of retear were considered as intact.
Fig 2. Coronal oblique cut (A) and sagittal oblique cut (B) from MRI scan of right shoulder with increased signal intensity within rotator cuff tendon (arrows) after arthroscopic rotator cuff repair. The interpretation of these MRI scans was different among 3 shoulder surgeons (rater 1 diagnosed no tear, Sugaya type 2; rater 2 diagnosed partialthickness retear, Sugaya type 3; rater 3 diagnosed full-thickness retear, Sugaya type 4). (MRI, magnetic resonance imaging.)
4
A. HASEGAWA ET AL.
Fig 3. Representative images of full-thickness retear from right shoulder after repair. Coronal oblique cut (A) and sagittal oblique cut (B) from magnetic resonance imaging scan of right shoulder with torn rotator cuff tendon after arthroscopic rotator cuff repair. The repaired tendon was torn from the footprint completely. The torn tendon was retracted medially at the glenoid surface level. A major discontinuity is observed between the footprint and the torn tendon (arrows). Among the 3 shoulder surgeons, there was complete agreement for all classifications in this case.
Sugaya et al.15 classified cuff integrity into 5 types: type I, repaired cuff appearing to have sufficient thickness with homogeneously low intensity; type II, Table 2. Criteria for Classification Scheme Owen et al.14 1, no tear 2, partial-thickness tear 3, complete rotator cuff tear Sugaya et al.15 Type I, sufficient thickness with homogenously low intensity Type II, sufficient thickness with partial high intensity Type III, insufficient thickness without discontinuity Type IV, presence of a minor discontinuity Type V, presence of a major discontinuity Hayashida et al.16 Type 1, a well-repaired tendon Type 2, partial retearing of the deep-layer tissue Type 3, partial retearing of the superficial layer around the medial anchors Type 4, complete retearing at the middle of the tendon around the medial anchors Type 5, complete retearing of the tendon from the footprint Burks et al.17 Footprint coverage Grade 1, 1%-25% coverage Grade 2, 26%-50% coverage Grade 3, 51%-75% coverage Grade 4, >75% coverage of the width of greater tuberosity Tendon thickness Grade 1, 1%-25% of normal thickness Grade 2, 26%-50% of normal thickness Grade 3, 51%-75% of normal thickness Grade 4, >75% of normal thickness Increased signal intensity of the repaired tendon Grade 1, >2 cm distance or >75% of tendon width Grade 2, 1-2 cm distance or 25%-50% of tendon width Grade 3, <1 cm distance or <25% of tendon width Grade 4, normal signal intensity
sufficient thickness with an area of partial high intensity; type III, insufficient thickness without discontinuity, thus suggesting a partial-thickness delaminated tear; type IV, presence of a minor discontinuity in only 1 or 2 slices on both oblique coronal and sagittal images, suggesting a small full-thickness tear; and type V: presence of a major discontinuity in 3 or more slices on both oblique coronal and sagittal images, suggesting a medium or large full-thickness tear. Hayashida et al.16 classified repair integrity into 5 types according to tear size and location: (1) wellrepaired tendon; (2) partial retearing of the deeplayer tissue, showing interruption of the deep tendon; (3) partial retearing of the superficial layer around the medial anchors, with interruption of the superficial tendon around these anchors; (4) complete retearing at the middle of the tendon around the medial anchors; and (5) complete retearing of the tendon from the footprint. Burks et al.17 evaluated footprint size, tendon thickness, and tendon signal. Each of these parameters was graded numerically on a scale from 1 to 4. Under the Burks’ scoring system, the size of the footprint was compared with that of a normal supraspinatus tendon, which covers the entire greater tuberosity from medial to lateral. In cases where the tendon attachment was medialized, the width of the medialized footprint was compared with the width of the greater tuberosity. Tendon thickness was compared with that of the normal supraspinatus tendon by using quartile divisions, as described for the footprint. Tendon signal intensity was graded based on the length and width of increased signal intensity in the tendon. Footprint size, tendon thickness, and tendon signal intensity scores
5
ROTATOR CUFF REPAIR MRI AGREEMENT Table 3. Number of Patients With a Diagnosis of FullThickness Retear or Partial-Thickness Retear 6 Months
24 Months
Rater 1 Rater 2 Rater 3 Rater 1 Rater 2 Rater 3 Number of full-thickness retears 2 7 6 4 7 3 Owen et al.14 Sugaya et al.15 5 7 6 6 7 6 6 7 6 6 7 3 Hayashida et al.16 Number of partial-thickness retears Owen et al.14 8 4 4 10 7 4 Sugaya et al.15 3 4 4 8 7 1 3 5 4 10 7 3 Hayashida et al.16 NOTE. The numbers of patients with a full- or partial-thickness retear diagnosed by each of the 3 shoulder surgeons at 6 and 24 months after surgery are presented. The total number of patients diagnosed at both 6 and 24 months after surgery was 34.
were added together to create a rotator cuff score. On this scale, a tendon repair with a completely normalized appearance had a score of 12. We measured intra- and inter-rater agreement of footprint size, tendon thickness, and tendon signal intensity by using Burks’ scoring system for each category separately. To perform the subgroup analysis of patients with partial and full-thickness tears on MRI, Sugaya type III, Hayashida type 2 and type 3 were considered partialthickness tear. Sugaya type IV and type V and Hayashida type 4 and type 5 were considered full-thickness tear. The Owen classification was literally interpretable. Statistical Analysis Multirater kappa statistics were used to measure intra- and inter-rater agreement among the experienced orthopaedic surgeons. Inter-rater agreement and the number of retears were calculated by using each reviewer’s initial assessment of the MRI scans. The kappa value is a chance-adjusted measure of agreement. Fleiss kappa coefficients were used for inter-rater assessment among the 3 raters, whereas the Cohen kappa coefficient was used to report the intrarater agreements. A kappa of 0.0 represents agreement owing to random chance alone, whereas a kappa of 1.0 represents perfect agreement. A negative kappa represents agreement worse than what would be expected because of chance alone, and a kappa of 1.0 represents complete discordance between observers.
Statistical analyses were performed using R, version 3.0.0 (http://www.r-project.org/). The kappa values were interpreted according to guidelines adapted from the work of Landis and Koch.27 “Excellent” agreement occurred when the kappa value was between 0.81 and 1.00; “good” was between 0.61 and 0.80; “moderate” between 0.41 and 0.60; “fair” between 0.21 and 0.40; and “poor” 0.20 or less.
Results The number of full- and partial-thickness retears at 6 and 24 months after surgery in Owen, Sugaya, and Hayashida classifications are presented in Table 3. The intra- and inter-rater agreements for the diagnosis of all 34 patients at 6 and 24 months after surgery in the 4 evaluations are presented in Table 4. All 4 classification systems showed that there was considerable variability in interpretation of rotator cuff integrity after ARCR among 3 shoulder surgeons (poor to good for intrarater agreement and fair to moderate for inter-rater agreement). No “excellent” intra- or inter-rater agreement was found in any of the 4 evaluations. Kappa values of the inter-rater agreement at 24 months were higher than those at 6 months after ARCR in all evaluations (Owen criteria: k ¼ 0.37 [6 months] and 0.60 [24 months]; Sugaya classification: k ¼ 0.31 and 0.49; Hayashida classification: k ¼ 0.44 and 0.57; Burks score: k ¼ 0.23 and 0.31). In terms of the identification of full-thickness tears, the reviewers had moderate to excellent levels of intraand inter-rater agreement in all evaluations at both 6 and 24 months after ARCR (Table 5). Kappa values of inter-rater agreement was higher at 24 months after ARCR than at 6 months in the Owen, Sugaya, and Hayashida evaluations (Owen criteria: k ¼ 0.42 [6 months] and 0.61 [24 months]; Sugaya classification: k ¼ 0.63 and 0.72; and Hayashida classification: k ¼ 0.73 and 0.80. In contrast, intra- and inter-rater agreements were less on partial-thickness tear than on full-thickness tear in all 3 of these evaluations (Table 6). Inter-rater agreement after ARCR was higher at 24 months than at 6 months in all 3 evaluations (Owen criteria: k ¼ 0.13 [6 months] and 0.44 [24 months]; Sugaya classification: k ¼ 0.19 [6 months] and 0.33 [24 months]; Hayashida
Table 4. Intra- and Inter-rater Agreements for the Diagnosis of All 34 Patients at 6 and 24 Months After Surgery in 4 Evaluations Intrarater Agreement Classification Owen et al.14 Sugaya et al.15 Hayashida et al.16 Burks et al.17
0.19-0.28 0.24-0.59 0.34-0.65 0.17-0.52
6 Months (poor to fair) (fair to moderate) (fair to good) (poor to moderate)
24 Months 0.19-0.28 (poor to fair) 0.33-0.51 (fair to moderate) 0.40-0.67 (fair to good) 0.14-0.46 (poor to moderate)
NOTE. Kappa values and interpretations of agreement (in parentheses) are presented.
Inter-rater Agreement 6 Months 0.37 (fair) 0.31 (fair) 0.44 (moderate) 0.23 (fair)
24 Months 0.60 (moderate) 0.49 (moderate) 0.57 (moderate) 0.31 (fair)
6
A. HASEGAWA ET AL.
Table 5. Intra- and Inter-rater Agreement for the Diagnosis of Full-Thickness Retear at 6 and 24 Months After Repair in 3 Evaluations Intrarater Agreement on Full-thickness Retear Classification Owen et al.14 Sugaya et al.15 Hayashida et al.16
6 Months 0.68-1.0 (good to excellent) 0.62-1.0 (good to excellent) 0.68-0.91 (good to excellent)
24 Months 0.77-0.91 (good to excellent) 0.68-0.91 (good to excellent) 0.68-0.91 (good to excellent)
Inter-rater Agreement on Full-thickness Retear 6 Months 0.42 (moderate) 0.63 (good) 0.73 (good)
24 Months 0.61 (good) 0.72 (good) 0.80 (good)
NOTE. Kappa values and interpretations of agreement (in parentheses) are presented.
classification: k ¼ 0.15 [6 months] and 0.53 [24 months]). However, inter-rater agreement on partial-thickness tear at 24 months after ARCR was either fair or moderate in the 3 evaluations. With regard to Burks’ footprint size, tendon thickness, and tendon signal intensity, the intrarater agreement demonstrated considerable variability (Table 7). Interrater agreement after ARCR was higher at 24 months than at 6 months with regard to footprint coverage and signal intensity. Inter-rater agreement on tendon thickness did not change between 6 and 24 months after ARCR, remaining at fair.
Discussion As we hypothesized, the agreement for assessment of full-thickness retear was higher than that of partialthickness retear and that intra- and inter-rater agreements after ARCR were higher at 24 months than those at 6 months. These results suggest that partial thickness retears were more difficult to diagnose on postoperative MRI than full thickness retears. The reliability of MRI assessment of retears improved from 6 to 24 months after ARCR. No classification system was found to be superior. Some studies of the reliability of assessments of rotator cuff tear before surgical repair have shown higher agreement in detecting full-thickness than partialthickness tear.19,21,23 Khazzam et al.28 examined the intra- and inter-rater reliability of MRI evaluation after rotator cuff repair. They found moderate inter-rater agreement on predicting full-thickness retears but poor agreement on tear classification, retear location, footprint coverage, tendon thickness, and tendon signal intensity. Balich et al. and Robertson et al. also found moderate to excellent inter-rater agreement on predicting full-thickness retears but poor to fair agreement on predicting partial-thickness tears.18,19 Our results were consistent with these previous studies.
With regard to the time points, kappa values of the inter-rater agreement at 24 months were higher than those at 6 months after ARCR in all evaluations as we hypothesized. Several factors may negatively affect agreement on the identification of rotator cuff retear. They include MRI artifacts from the suture anchor and increased signal intensity within the repaired rotator cuff tendon from scarring. The normal healing response within the rotator cuff tendon can also give a high signal intensity on T2-weighted MRI images.29 Crim et al.26 showed that the MRI appearance of the repaired tendon changes over time, even within 1 year after surgery and suggested that it was not prudent to consider the tendon repair as failed according to tendon irregularity, thinning, or increased signal intensity during the first operative year. With regard to the signal intensity, our results showed that inter-rater agreement after ARCR was moderate at 24 months but fair at 6 months using Burks criteria. These results suggest that shoulder surgeons had difficulties to discriminate partial-thickness retear from normal healing response of the rotator cuff tendon, particularly when they assess the MRI images taken at 6 months after ARCR. Collectively, we wonder whether MRI assessment at 6 months after surgery might be still in the healing phase, and this might be one of the reasons behind low levels of agreement on retear, particularly in the diagnosis of partial-thickness retear. These imply that there might be considerable variability in surgeons’ interpretation for diagnosing retear around the time when the retears generally occur (before 6 months after surgery). Nevertheless, to our knowledge, few studies have assessed intra- and inter-rater agreement on MRI at 2 different time points after repair. This study was designed to investigate intra- and inter-rater agreement on MRI evaluations at 2 different time points, namely, at 6 and 24 months after ARCR. We found higher kappa values of
Table 6. Intra- and Inter-rater Agreement for the Diagnosis of Partial-Thickness Retear at 6 and 24 Months After Repair in 3 Evaluations Intrarater Agreement on Partial-Thickness Retear Classification Owen et al.14 Sugaya et al.15 Hayashida et al.16
6 Months 0.12, 0.68 (poor to good) 0.02, 0.53 (poor to moderate) 0.10, 0.68 (poor to good)
24 Months 0.36, 0.52 (fair to moderate) 0.06, 0.36 (poor to fair) 0.01, 0.53 (poor to moderate)
NOTE. Kappa values and interpretations of agreement (in parentheses) are presented.
Inter-rater Agreement on Partial-Thickness Retear 6 Months 0.13 (poor) 0.19 (poor) 0.15 (poor)
24 Months 0.44 (moderate) 0.33 (fair) 0.53 (moderate)
7
ROTATOR CUFF REPAIR MRI AGREEMENT Table 7. Intra- and Inter-rater Agreement on Footprint Coverage, Tendon Thickness, and Signal Intensity in Burks et al.17 Classification Intrarater Agreement Footprint coverage Tendon thickness Signal intensity
6 Months 0.22-0.53 (fair to moderate) 0.39-0.66 (fair to good) 0.38-0.61 (fair to good)
Inter-rater Agreement
24 Months 0.23-0.74 (fair to good) 0.21-0.64 (fair to good) 0.35-0.64 (fair to good)
6 Months 0.16 (poor) 0.30 (fair) 0.23 (fair)
24 Months 0.31 (fair) 0.35 (fair) 0.43 (moderate)
NOTE. Kappa values and interpretations of agreement in parentheses are presented.
inter-rater agreement at 24 months after ARCR than at 6 months in not only full- but also partial-thickness retear. These results suggest that evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used. Limitations This study had some limitations. First, it did not include arthroscopic assessment of the postoperative rotator cuff to determine whether the MRI findings correctly reflected the rotator cuff integrity after repair. Therefore, we were not able to confirm the accuracy of the MRI findings. Instead, the study was designed to evaluate the inter- and intraobserver reliability of previously established rotator cuff integrity classification and scoring systems: its goal was to determine whether systems for postoperative MRI assessment of cuff integrity were comparatively reliable. MRI assessments before surgery have sensitivities of 90% to 100% in predicting full-thickness tear; in the case of partialthickness tear, the sensitivity ranges from 35% to 82%.22,30-32 Owen et al.14 examined the accuracy of MRI evaluation after surgery in 31 patients before reoperation and found an accuracy of 90% using their evaluation system. Taking the results of these studies and ours into consideration, we believe that MRI has sufficiently high accuracy to be used for structural assessment of the rotator cuff after surgery. The second limitation of our study is that the MRI images were reviewed only by shoulder surgeons but not by musculoskeletal radiologists. However, our shoulder surgeons demonstrated good or excellent intrarater agreement and moderate or good inter-rater agreement in predicting full-thickness tears. Grant et al.23 reported that the reliability of MRI detection of full-thickness tears among shoulder surgeons and between shoulder surgeons and musculoskeletal radiologists was excellent using the criteria similar to the Owen, Sugaya, and Hayashida classifications: the presence of fluid signal identifying discontinuity of the central tendon on 2 imaging planes was identified as full-thickness tears. In addition, Balich et al.18 reported that sensitivity for diagnosis of partial-thickness tear was poor among a group of radiologists, regardless of the radiologists’ levels of experience and knowledge of the arthroscopic findings using the same criteria as the
Owen classification system. We therefore believe that our results do not differ markedly from those of previous studies performed by radiologists. Third, our study did not include fat-suppression images or magnetic resonance arthrography, the use of which might improve sensitivity and levels of agreement. Singson et al.20 evaluated MRI images of 177 patients; when fat-suppression images were used, 21 partialthickness tears were diagnosed that were not detected earlier without the use of fat-suppression images. Magnetic resonance arthrography improves the ability to diagnose both full- and partial-thickness tears, although it is more invasive than MRI.33-35 Further study is needed to investigate the usefulness of fat-suppression images and magnetic resonance arthrography in evaluations of rotator cuff integrity after ARCR. Fourth, this study did not have a statistical comparison among 3 classification systems. We therefore cannot conclude from our results which MRI evaluation system is the best for the assessment of rotator cuff integrity after repair.
Conclusions Shoulder surgeons showed better intra- and inter-rater agreement in predicting full-thickness tears than in predicting partial-thickness tears. The inter-rater agreement at 24 months after ARCR was superior to that at 6 months in predicting not only full-thickness but also partial-thickness retear. MRI evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used.
Acknowledgment The authors thank Yasuichiro Nishimura for his assistance with statistical analysis.
References 1. Mihata T, Watanabe C, Fukunishi K, et al. Functional and structural outcomes of single-row versus double-row versus combined double-row and suture-bridge repair for rotator cuff tears. Am J Sports Med 2011;39:2091-2098. 2. Sugaya H, Maeda K, Matsuki K, Moriishi J. Repair integrity and functional outcome after arthroscopic double-row rotator cuff repair. A prospective outcome study. J Bone Joint Surg Am 2007;89:953-960. 3. Boileau P, Brassart N, Watkinson DJ, Carles M, Hatzidakis AM, Krishnan SG. Arthroscopic repair of
8
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
A. HASEGAWA ET AL. full-thickness tears of the supraspinatus: does the tendon really heal? J Bone Joint Surg Am 2005;87:1229-1240. Harryman DT 2nd, Mack LA, Wang KY, Jackins SE, Richardson ML, Matsen FA 3rd. Repairs of the rotator cuff. Correlation of functional results with integrity of the cuff. J Bone Joint Surg Am 1991;73:982-989. Gazielly DF, Gleyze P, Montagnon C. Functional and anatomical results after rotator cuff repair. Clin Orthop Relat Res 1994:43-53. Thomazeau H, Boukobza E, Morcet N, Chaperon J, Langlais F. Prediction of rotator cuff repair results by magnetic resonance imaging. Clin Orthop Relat Res 1997:275-283. Gerber C, Fuchs B, Hodler J. The results of repair of massive tears of the rotator cuff. J Bone Joint Surg Am 2000;82:505-515. Galatz LM, Ball CM, Teefey SA, Middleton WD, Yamaguchi K. The outcome and repair integrity of completely arthroscopically repaired large and massive rotator cuff tears. J Bone Joint Surg Am 2004;86:219-224. Liu SH, Baker CL. Arthroscopically assisted rotator cuff repair: Correlation of functional results with integrity of the cuff. Arthroscopy 1994;10:54-60. Jost B, Pfirrmann CW, Gerber C, Switzerland Z. Clinical outcome after structural failure of rotator cuff repairs. J Bone Joint Surg Am 2000;82:304-314. Park MC, Elattrache NS, Ahmad CS, Tibone JE. “Transosseous-equivalent” rotator cuff repair technique. Arthroscopy 2006;22:1360.e1-1360.e5. Park MC, Tibone JE, ElAttrache NS, Ahmad CS, Jun BJ, Lee TQ. Part II: Biomechanical assessment for a footprintrestoring transosseous-equivalent rotator cuff repair technique compared with a double-row repair technique. J Shoulder Elbow Surg 2007;16:469-476. Mihata T, Fukuhara T, Jun BJ, Watanabe C, Kinoshita M. Effect of shoulder abduction angle on biomechanical properties of the repaired rotator cuff tendons with 3 types of double-row technique. Am J Sports Med 2011;39:551-556. Owen RS, Iannotti JP, Kneeland JB, Dalinka MK, Deren JA, Oleaga L. Shoulder after surgery: MR imaging with surgical validation. Radiology 1993;186:443-447. Sugaya H, Maeda K, Matsuki K, Moriishi J. Functional and structural outcome after arthroscopic full-thickness rotator cuff repair: Single-row versus dual-row fixation. Arthroscopy 2005;21:1307-1316. Hayashida K, Tanaka M, Koizumi K, Kakiuchi M. Characteristic retear patterns assessed by magnetic resonance imaging after arthroscopic double-row rotator cuff repair. Arthroscopy 2012;28:458-464. Burks RT, Crim J, Brown N, Fink B, Greis PE. A prospective randomized clinical trial comparing arthroscopic single- and double-row rotator cuff repair: Magnetic resonance imaging and early clinical evaluation. Am J Sports Med 2009;37:674-682. Balich SM, Sheley RC, Brown TR, Sauser DD, Quinn SF. MR imaging of the rotator cuff tendon: Interobserver agreement and analysis of interpretive errors. Radiology 1997;204:191-194. Robertson PL, Schweitzer ME, Mitchell DG, et al. Rotator cuff disorders: Interobserver and intraobserver variation in diagnosis with MR imaging. Radiology 1995;194:831-835.
20. Singson RD, Hoang T, Dan S, Friedman M. MR evaluation of rotator cuff pathology using T2-weighted fast spin-echo technique with and without fat suppression. AJR Am J Roentgenol 1996;166:1061-1065. 21. Spencer EE Jr, Dunn WR, Wright RW, et al. Interobserver agreement in the classification of rotator cuff tears using magnetic resonance imaging. Am J Sports Med 2008;36: 99-103. 22. Teefey SA, Rubin DA, Middleton WD, Hildebolt CF, Leibold RA, Yamaguchi K. Detection and quantification of rotator cuff tears. Comparison of ultrasonographic, magnetic resonance imaging, and arthroscopic findings in seventy-one consecutive cases. J Bone Joint Surg Am 2004;86:708-716. 23. Grant JA, Miller BS, Jacobson JA, et al. Intra- and interrater reliability of the detection of tears of the supraspinatus central tendon on MRI by shoulder surgeons. J Shoulder Elbow Surg 2013;22:725-731. 24. Iannotti JP, Deutsch A, Green A, et al. Time to failure after rotator cuff repair: a prospective imaging study. J Bone Joint Surg Am 2013;95:965-971. 25. Cho NS, Yi JW, Lee BG, Rhee YG. Retear patterns after arthroscopic rotator cuff repair: Single-row versus suture bridge technique. Am J Sports Med 2010;38:664-671. 26. Crim J, Burks R, Manaster BJ, Hanrahan C, Hung M, Greis P. Temporal evolution of MRI findings after arthroscopic rotator cuff repair. AJR Am J Roentgenol 2010;195: 1361-1366. 27. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174. 28. Khazzam M, Kuhn JE, Mulligan E, et al. Magnetic resonance imaging identification of rotator cuff retears after repair: Interobserver and intraobserver agreement. Am J Sports Med 2012;40:1722-1727. 29. Gusmer PB, Potter HG, Donovan WD, O’Brien SJ. MR imaging of the shoulder after rotator cuff repair. AJR Am J Roentgenol 1997;168:559-563. 30. Iannotti JP, Zlatkin MB, Esterhai JL, Kressel HY, Dalinka MK, Spindler KP. Magnetic resonance imaging of the shoulder. Sensitivity, specificity, and predictive value. J Bone Joint Surg Am 1991;73:17-29. 31. Quinn SF, Sheley RC, Demlow TA, Szumowski J. Rotator cuff tendon tears: evaluation with fat-suppressed MR imaging with arthroscopic correlation in 100 patients. Radiology 1995;195:497-500. 32. Reinus WR, Shady KL, Mirowitz SA, Totty WG. MR diagnosis of rotator cuff tears of the shoulder: Value of using T2-weighted fat-saturated images. AJR Am J Roentgenol 1995;164:1451-1455. 33. Ferrari FS, Governi S, Burresi F, Vigni F, Stefani P. Supraspinatus tendon tears: Comparison of US and MR arthrography with surgical correlation. Eur Radiol 2002;12:1211-1217. 34. Toyoda H, Ito Y, Tomo H, Nakao Y, Koike T, Takaoka K. Evaluation of rotator cuff tears with magnetic resonance arthrography. Clin Orthop Relat Res 2005;439:109-115. 35. de Jesus JO, Parker L, Frangos AJ, Nazarian LN. Accuracy of MRI, MR arthrography, and ultrasound in the diagnosis of rotator cuff tears: A meta-analysis. AJR Am J Roentgenol 2009;192:1701-1707.
Purpose: To investigate the intra- and inter-rater agreement of magnetic resonance imaging (MRI) evaluations of rotator cuff integrity at 6 and 24 months after arthroscopic rotator cuff repair (ARCR). Methods: Three shoulder surgeons reviewed 68 MRI scans from 34 patients who had undergone ARCR and MRI examination at both 6 and 24 months after surgery. Postoperative rotator cuff integrity was investigated by using Owen, Sugaya, and Hayashida classifications to determine whether the rotator cuff was intact or whether there was a partial-thickness retear or full-thickness retear and Burks score to assess tendon appearance. Multirater kappa statistics were used to measure intra- and inter-rater agreement. Kappa values were interpreted according to guidelines adapted from the work of Landis and Koch. Results: All classifications had similar intra- and inter-rater agreement (k ¼ 0.14 to 0.67, 0.23 to 0.60, respectively), but no intra- or inter-rater agreement scored “excellent.” Inter-rater agreement after ARCR was higher at 24 months (k ¼ 0.31 to 0.60) than at 6 months (k ¼ 0.23 to 0.44) in all evaluations. Reviewers identified full-thickness retears with a moderate to good degree of inter-rater agreement in all evaluations, at both 6 months (k ¼ 0.42 to 0.73) and 24 months (k ¼ 0.61 to 0.80) after ARCR. However, poor inter-rater agreement (k ¼ 0.13 to 0.19) was found in the identification of partial-thickness retears in all evaluations at 6 months after ARCR. Conclusions: Shoulder surgeons showed better intra- and inter-rater agreement in predicting full-thickness tears compared with partial-thickness tears. The inter-rater agreement at 24 months after ARCR was superior to that at 6 months in predicting not only full-thickness retear but also partial-thickness retear. MRI evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used. Level of Evidence: Level III, retrospective comparative study.
R
otator cuff tear is a common injury of the shoulder joint and often requires surgical repair. Arthroscopic rotator cuff repair (ARCR) has become a popular treatment for these tears. The improved integrity of the repair site after rotator cuff repair has been reported to give a better functional outcome,1-7 although some From the Department of Orthopedic Surgery, Osaka Medical College (A.H., T.M., Y.I., M.N.), Takatsuki; Department of Orthopedic Surgery, First Towakai Hospital (A.H., T.M., Y.I.), Takatsuki; Department of Orthopedic Surgery, Nishinomiya Kyoritsu Neurosurgical Hospital (K.Y.), Nishinomiya; and Department of Orthopedic Surgery, Shiroyama Hospital (T.K.), Osaka, Japan. The authors report that they have no conflicts of interest in the authorship and publication of this article. The Institutional Review Board at Osaka Medical College approved the protocol of this study. Received December 10, 2015; accepted April 21, 2016. Address correspondence to Akihiko Hasegawa, M.D., Ph.D., Department of Orthopedic Surgery, Osaka Medical College, 2-7 Daigaku-machi, Takatsuki, Osaka, Japan. E-mail: [email protected] Ó 2016 by the Arthroscopy Association of North America 0749-8063/151135/$36.00 http://dx.doi.org/10.1016/j.arthro.2016.04.027
investigators have shown that structural outcome is not correlated with functional outcome.8-10 The doublerow suture anchor technique and the suture-bridge technique have been reported to improve the biomechanical construct with less tendon retear rate.1,11-13 These techniques, however, have not been sufficient to prevent development of retears. Therefore, retears after repair have been a matter of concern to shoulder surgeons. Recently, magnetic resonance imaging (MRI) has become commonly used to assess rotator cuff integrity before and after arthroscopic repair, and classification and scoring systems have been established.14-17 Several studies have reported on inter-rater agreement in the assessment of rotator cuff integrity before surgery.18-23 It has been reported that radiologists and/or orthopaedic surgeons had good agreement for predicting fullthickness rotator cuff tears but variable (poor to good) agreement in predicting the partial-thickness tear before surgery.18-21,23 However, few studies have assessed the inter- and intraobserver reliability of these
Arthroscopy: The Journal of Arthroscopic and Related Surgery, Vol
-,
No
-
(Month), 2016: pp 1-8
1
2
A. HASEGAWA ET AL.
classification and scoring systems. It is important to know the degree of reliability of MRI assessment after ARCR to decide the postoperative management of patients who underwent ARCR. Rotator cuff retears occur primarily between 6 and 26 weeks after ARCR, and few additional tears occur thereafter.24 Some studies have used MRI images taken at 6 months after surgery to compare functional outcomes and repair integrity.3,16,25 However, it has been reported that MRI appearance of the rotator cuff after arthroscopic repair showed considerable variability during the first postoperative year and did not correlate with outcome. It has been suggested that it was not prudent to consider the tendon repair as failed according to tendon irregularity, thinning, or increased signal intensity during the first operative year.26 Therefore, the purpose of this study was to investigate the intra- and inter-rater agreement of MRI evaluations of rotator cuff integrity at 6 and 24 months after ARCR. Our hypotheses were that the agreement for assessment of full-thickness retear was higher than that of partial-thickness retear and that intra- and inter-rater agreements after ARCR were higher at 24 months than those at 6 months.
Methods Patient Selection The institutional review board at our university approved the protocol of this study. Potential subjects were identified retrospectively through a review of one surgeon’s database from 2006 to 2010. The inclusion criteria were as follows: complete rotator cuff tear confirmed during arthroscopic surgery; complete repair (complete tuberosity coverage attainable); minimum 2 years of follow-up; and MRI for evaluation of the integrity of the rotator cuff tendons at both 6 and 24 months after surgery. Revision surgery was not included in this study. We retrospectively reviewed our database and initially included 88 patients who, between 2006 and 2010, had received arthroscopic rotator cuff repairs from a single experienced shoulder surgeon (T.M.) using the double-row technique or a combination of double-row and suture-bridge at our University.1,11-13 The titanium suture anchors (diameter, 5 mm; Corkscrew, Arthrex) with two No. 2 permanent sutures (FiberWire, Arthrex) were placed for all surgical procedures. No PEEK (polyether ether ketone)/ bioabsorbable anchor was placed. Of these 88 patients, 34 did not complete the 2-year follow-up and 20 did not undergo postoperative MRI scans at either 6 or 24 months after surgery. Consequently, our final study group consisted of 68 MRI scans of 34 shoulders from 34 patients (20 female and 14 male; mean age, 68.1 years [range, 57 to 81 years]; Table 1).
Table 1. Summary of Patients Patient 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
Age (years) 62 79 81 69 60 63 75 66 73 62 59 66 65 65 59 80 80 63 65 62 71 63 71 71 73 69 64 68 69 68 79 57 70 70
Gender Female Male Male Female Female Male Female Male Female Female Male Female Male Male Female Female Female Male Female Male Male Female Female Male Female Female Female Female Female Male Female Female Male Male
Tear Size Large Small Medium Medium Medium Medium Medium Medium Medium Large Medium Large Medium Large Medium Large Medium Medium Medium Medium Medium Large Large Medium Medium Medium Medium Medium Medium Large Small Small Small Small
Type of Repair DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR and SB DR DR DR DR
DR, double-row technique; DR and SB, a combination of doublerow and suture-bridge.
Magnetic Resonance Imaging Magnetic resonance imaging was performed with a 1.5-Tesla closed-type scanner (MRT-2000/V2; Toshiba, Tokyo, Japan). Oblique coronal, oblique sagittal, and axial T2-weighted images were acquired for structural and qualitative assessment of the rotator cuff tendon, and repair integrity was determined. The slice thickness was 4 mm, with an interslice gap of 0.5 mm. Postoperative cuff integrity was evaluated according to 4 previously established methods. One of the authors (A.H.) reviewed the clinical data and MRI images. Patients included in the study were considered by this author to meet the criteria of having good-quality MRI images of the shoulder joint and rotator cuff at both 6 and 24 months after ARCR. The author who selected the MRI study sets did not evaluate any of the MRI images. MRI Evaluations Three orthopaedic shoulder surgeons (K.Y., T.K., Y.I.) independently reviewed 68 MRI scans of 34 shoulders
ROTATOR CUFF REPAIR MRI AGREEMENT
3
Fig 1. Representative images of intact rotator cuff after repair. Coronal oblique cut (A) and sagittal oblique cut (B) from magnetic resonance imaging scan of right shoulder with intact rotator cuff after arthroscopic rotator cuff repair. The intact rotator cuff tendon shows sufficient thickness with homogenously low intensity. Among the 3 shoulder surgeons, there was complete agreement for all classifications in this case.
from 34 patients. All 3 orthopaedic shoulder surgeons were specialists of orthopaedic surgery certified by the Japanese Orthopaedic Association. Additionally, they had completed at least 1-year fellowship in shoulder surgery under the mentor. The shoulder surgeons were asked to evaluate each of the 68 scans by using all 4 classification schemes and returned scores for each scan. After an interval of 1 month, they were asked to evaluate each of the same 68 scans in a different random order and returned scores for each scan as second evaluations. They did not have access to clinical data or prior evaluations of the scans, and they did not know when the MRI scans were taken after ARCR. The MRI data were randomized onto an Excel data sheet (Microsoft, Redmond, WA), and all clinical data were dissociated from the MRI data for each patient. MRI images were presented individually to each surgeon on the Digital Imaging and COmmunication in Medicine (DICOM) imaging software, OsiriX (Pixmeo Sarl, Switzerland).
Postoperative rotator cuff integrity was investigated by using 4 different evaluation methods, namely, the Owen, Sugaya, and Hayashida classifications to determine whether the cuff was intact or whether there was a partial- or full-thickness retear (Figs 1-3) and Burks’ score to assess tendon appearance, including footprint coverage, tendon thickness, and signal intensity.14-17 Criteria for the 4 classification schemes were summarized in Table 2. Briefly, Owen et al.14 identified partial-thickness tears by increased signal intensity within the rotator cuff on spin-density and T2-weighted sequences. A fullthickness rotator cuff tear was diagnosed when fluidlike signal intensity on T2-weighted images extended through the entire thickness of the rotator cuff, or when complete nonvisualization of a portion of the rotator cuff was noted on 1 or more sections. Rotator cuffs without such evidence of retear were considered as intact.
Fig 2. Coronal oblique cut (A) and sagittal oblique cut (B) from MRI scan of right shoulder with increased signal intensity within rotator cuff tendon (arrows) after arthroscopic rotator cuff repair. The interpretation of these MRI scans was different among 3 shoulder surgeons (rater 1 diagnosed no tear, Sugaya type 2; rater 2 diagnosed partialthickness retear, Sugaya type 3; rater 3 diagnosed full-thickness retear, Sugaya type 4). (MRI, magnetic resonance imaging.)
4
A. HASEGAWA ET AL.
Fig 3. Representative images of full-thickness retear from right shoulder after repair. Coronal oblique cut (A) and sagittal oblique cut (B) from magnetic resonance imaging scan of right shoulder with torn rotator cuff tendon after arthroscopic rotator cuff repair. The repaired tendon was torn from the footprint completely. The torn tendon was retracted medially at the glenoid surface level. A major discontinuity is observed between the footprint and the torn tendon (arrows). Among the 3 shoulder surgeons, there was complete agreement for all classifications in this case.
Sugaya et al.15 classified cuff integrity into 5 types: type I, repaired cuff appearing to have sufficient thickness with homogeneously low intensity; type II, Table 2. Criteria for Classification Scheme Owen et al.14 1, no tear 2, partial-thickness tear 3, complete rotator cuff tear Sugaya et al.15 Type I, sufficient thickness with homogenously low intensity Type II, sufficient thickness with partial high intensity Type III, insufficient thickness without discontinuity Type IV, presence of a minor discontinuity Type V, presence of a major discontinuity Hayashida et al.16 Type 1, a well-repaired tendon Type 2, partial retearing of the deep-layer tissue Type 3, partial retearing of the superficial layer around the medial anchors Type 4, complete retearing at the middle of the tendon around the medial anchors Type 5, complete retearing of the tendon from the footprint Burks et al.17 Footprint coverage Grade 1, 1%-25% coverage Grade 2, 26%-50% coverage Grade 3, 51%-75% coverage Grade 4, >75% coverage of the width of greater tuberosity Tendon thickness Grade 1, 1%-25% of normal thickness Grade 2, 26%-50% of normal thickness Grade 3, 51%-75% of normal thickness Grade 4, >75% of normal thickness Increased signal intensity of the repaired tendon Grade 1, >2 cm distance or >75% of tendon width Grade 2, 1-2 cm distance or 25%-50% of tendon width Grade 3, <1 cm distance or <25% of tendon width Grade 4, normal signal intensity
sufficient thickness with an area of partial high intensity; type III, insufficient thickness without discontinuity, thus suggesting a partial-thickness delaminated tear; type IV, presence of a minor discontinuity in only 1 or 2 slices on both oblique coronal and sagittal images, suggesting a small full-thickness tear; and type V: presence of a major discontinuity in 3 or more slices on both oblique coronal and sagittal images, suggesting a medium or large full-thickness tear. Hayashida et al.16 classified repair integrity into 5 types according to tear size and location: (1) wellrepaired tendon; (2) partial retearing of the deeplayer tissue, showing interruption of the deep tendon; (3) partial retearing of the superficial layer around the medial anchors, with interruption of the superficial tendon around these anchors; (4) complete retearing at the middle of the tendon around the medial anchors; and (5) complete retearing of the tendon from the footprint. Burks et al.17 evaluated footprint size, tendon thickness, and tendon signal. Each of these parameters was graded numerically on a scale from 1 to 4. Under the Burks’ scoring system, the size of the footprint was compared with that of a normal supraspinatus tendon, which covers the entire greater tuberosity from medial to lateral. In cases where the tendon attachment was medialized, the width of the medialized footprint was compared with the width of the greater tuberosity. Tendon thickness was compared with that of the normal supraspinatus tendon by using quartile divisions, as described for the footprint. Tendon signal intensity was graded based on the length and width of increased signal intensity in the tendon. Footprint size, tendon thickness, and tendon signal intensity scores
5
ROTATOR CUFF REPAIR MRI AGREEMENT Table 3. Number of Patients With a Diagnosis of FullThickness Retear or Partial-Thickness Retear 6 Months
24 Months
Rater 1 Rater 2 Rater 3 Rater 1 Rater 2 Rater 3 Number of full-thickness retears 2 7 6 4 7 3 Owen et al.14 Sugaya et al.15 5 7 6 6 7 6 6 7 6 6 7 3 Hayashida et al.16 Number of partial-thickness retears Owen et al.14 8 4 4 10 7 4 Sugaya et al.15 3 4 4 8 7 1 3 5 4 10 7 3 Hayashida et al.16 NOTE. The numbers of patients with a full- or partial-thickness retear diagnosed by each of the 3 shoulder surgeons at 6 and 24 months after surgery are presented. The total number of patients diagnosed at both 6 and 24 months after surgery was 34.
were added together to create a rotator cuff score. On this scale, a tendon repair with a completely normalized appearance had a score of 12. We measured intra- and inter-rater agreement of footprint size, tendon thickness, and tendon signal intensity by using Burks’ scoring system for each category separately. To perform the subgroup analysis of patients with partial and full-thickness tears on MRI, Sugaya type III, Hayashida type 2 and type 3 were considered partialthickness tear. Sugaya type IV and type V and Hayashida type 4 and type 5 were considered full-thickness tear. The Owen classification was literally interpretable. Statistical Analysis Multirater kappa statistics were used to measure intra- and inter-rater agreement among the experienced orthopaedic surgeons. Inter-rater agreement and the number of retears were calculated by using each reviewer’s initial assessment of the MRI scans. The kappa value is a chance-adjusted measure of agreement. Fleiss kappa coefficients were used for inter-rater assessment among the 3 raters, whereas the Cohen kappa coefficient was used to report the intrarater agreements. A kappa of 0.0 represents agreement owing to random chance alone, whereas a kappa of 1.0 represents perfect agreement. A negative kappa represents agreement worse than what would be expected because of chance alone, and a kappa of 1.0 represents complete discordance between observers.
Statistical analyses were performed using R, version 3.0.0 (http://www.r-project.org/). The kappa values were interpreted according to guidelines adapted from the work of Landis and Koch.27 “Excellent” agreement occurred when the kappa value was between 0.81 and 1.00; “good” was between 0.61 and 0.80; “moderate” between 0.41 and 0.60; “fair” between 0.21 and 0.40; and “poor” 0.20 or less.
Results The number of full- and partial-thickness retears at 6 and 24 months after surgery in Owen, Sugaya, and Hayashida classifications are presented in Table 3. The intra- and inter-rater agreements for the diagnosis of all 34 patients at 6 and 24 months after surgery in the 4 evaluations are presented in Table 4. All 4 classification systems showed that there was considerable variability in interpretation of rotator cuff integrity after ARCR among 3 shoulder surgeons (poor to good for intrarater agreement and fair to moderate for inter-rater agreement). No “excellent” intra- or inter-rater agreement was found in any of the 4 evaluations. Kappa values of the inter-rater agreement at 24 months were higher than those at 6 months after ARCR in all evaluations (Owen criteria: k ¼ 0.37 [6 months] and 0.60 [24 months]; Sugaya classification: k ¼ 0.31 and 0.49; Hayashida classification: k ¼ 0.44 and 0.57; Burks score: k ¼ 0.23 and 0.31). In terms of the identification of full-thickness tears, the reviewers had moderate to excellent levels of intraand inter-rater agreement in all evaluations at both 6 and 24 months after ARCR (Table 5). Kappa values of inter-rater agreement was higher at 24 months after ARCR than at 6 months in the Owen, Sugaya, and Hayashida evaluations (Owen criteria: k ¼ 0.42 [6 months] and 0.61 [24 months]; Sugaya classification: k ¼ 0.63 and 0.72; and Hayashida classification: k ¼ 0.73 and 0.80. In contrast, intra- and inter-rater agreements were less on partial-thickness tear than on full-thickness tear in all 3 of these evaluations (Table 6). Inter-rater agreement after ARCR was higher at 24 months than at 6 months in all 3 evaluations (Owen criteria: k ¼ 0.13 [6 months] and 0.44 [24 months]; Sugaya classification: k ¼ 0.19 [6 months] and 0.33 [24 months]; Hayashida
Table 4. Intra- and Inter-rater Agreements for the Diagnosis of All 34 Patients at 6 and 24 Months After Surgery in 4 Evaluations Intrarater Agreement Classification Owen et al.14 Sugaya et al.15 Hayashida et al.16 Burks et al.17
0.19-0.28 0.24-0.59 0.34-0.65 0.17-0.52
6 Months (poor to fair) (fair to moderate) (fair to good) (poor to moderate)
24 Months 0.19-0.28 (poor to fair) 0.33-0.51 (fair to moderate) 0.40-0.67 (fair to good) 0.14-0.46 (poor to moderate)
NOTE. Kappa values and interpretations of agreement (in parentheses) are presented.
Inter-rater Agreement 6 Months 0.37 (fair) 0.31 (fair) 0.44 (moderate) 0.23 (fair)
24 Months 0.60 (moderate) 0.49 (moderate) 0.57 (moderate) 0.31 (fair)
6
A. HASEGAWA ET AL.
Table 5. Intra- and Inter-rater Agreement for the Diagnosis of Full-Thickness Retear at 6 and 24 Months After Repair in 3 Evaluations Intrarater Agreement on Full-thickness Retear Classification Owen et al.14 Sugaya et al.15 Hayashida et al.16
6 Months 0.68-1.0 (good to excellent) 0.62-1.0 (good to excellent) 0.68-0.91 (good to excellent)
24 Months 0.77-0.91 (good to excellent) 0.68-0.91 (good to excellent) 0.68-0.91 (good to excellent)
Inter-rater Agreement on Full-thickness Retear 6 Months 0.42 (moderate) 0.63 (good) 0.73 (good)
24 Months 0.61 (good) 0.72 (good) 0.80 (good)
NOTE. Kappa values and interpretations of agreement (in parentheses) are presented.
classification: k ¼ 0.15 [6 months] and 0.53 [24 months]). However, inter-rater agreement on partial-thickness tear at 24 months after ARCR was either fair or moderate in the 3 evaluations. With regard to Burks’ footprint size, tendon thickness, and tendon signal intensity, the intrarater agreement demonstrated considerable variability (Table 7). Interrater agreement after ARCR was higher at 24 months than at 6 months with regard to footprint coverage and signal intensity. Inter-rater agreement on tendon thickness did not change between 6 and 24 months after ARCR, remaining at fair.
Discussion As we hypothesized, the agreement for assessment of full-thickness retear was higher than that of partialthickness retear and that intra- and inter-rater agreements after ARCR were higher at 24 months than those at 6 months. These results suggest that partial thickness retears were more difficult to diagnose on postoperative MRI than full thickness retears. The reliability of MRI assessment of retears improved from 6 to 24 months after ARCR. No classification system was found to be superior. Some studies of the reliability of assessments of rotator cuff tear before surgical repair have shown higher agreement in detecting full-thickness than partialthickness tear.19,21,23 Khazzam et al.28 examined the intra- and inter-rater reliability of MRI evaluation after rotator cuff repair. They found moderate inter-rater agreement on predicting full-thickness retears but poor agreement on tear classification, retear location, footprint coverage, tendon thickness, and tendon signal intensity. Balich et al. and Robertson et al. also found moderate to excellent inter-rater agreement on predicting full-thickness retears but poor to fair agreement on predicting partial-thickness tears.18,19 Our results were consistent with these previous studies.
With regard to the time points, kappa values of the inter-rater agreement at 24 months were higher than those at 6 months after ARCR in all evaluations as we hypothesized. Several factors may negatively affect agreement on the identification of rotator cuff retear. They include MRI artifacts from the suture anchor and increased signal intensity within the repaired rotator cuff tendon from scarring. The normal healing response within the rotator cuff tendon can also give a high signal intensity on T2-weighted MRI images.29 Crim et al.26 showed that the MRI appearance of the repaired tendon changes over time, even within 1 year after surgery and suggested that it was not prudent to consider the tendon repair as failed according to tendon irregularity, thinning, or increased signal intensity during the first operative year. With regard to the signal intensity, our results showed that inter-rater agreement after ARCR was moderate at 24 months but fair at 6 months using Burks criteria. These results suggest that shoulder surgeons had difficulties to discriminate partial-thickness retear from normal healing response of the rotator cuff tendon, particularly when they assess the MRI images taken at 6 months after ARCR. Collectively, we wonder whether MRI assessment at 6 months after surgery might be still in the healing phase, and this might be one of the reasons behind low levels of agreement on retear, particularly in the diagnosis of partial-thickness retear. These imply that there might be considerable variability in surgeons’ interpretation for diagnosing retear around the time when the retears generally occur (before 6 months after surgery). Nevertheless, to our knowledge, few studies have assessed intra- and inter-rater agreement on MRI at 2 different time points after repair. This study was designed to investigate intra- and inter-rater agreement on MRI evaluations at 2 different time points, namely, at 6 and 24 months after ARCR. We found higher kappa values of
Table 6. Intra- and Inter-rater Agreement for the Diagnosis of Partial-Thickness Retear at 6 and 24 Months After Repair in 3 Evaluations Intrarater Agreement on Partial-Thickness Retear Classification Owen et al.14 Sugaya et al.15 Hayashida et al.16
6 Months 0.12, 0.68 (poor to good) 0.02, 0.53 (poor to moderate) 0.10, 0.68 (poor to good)
24 Months 0.36, 0.52 (fair to moderate) 0.06, 0.36 (poor to fair) 0.01, 0.53 (poor to moderate)
NOTE. Kappa values and interpretations of agreement (in parentheses) are presented.
Inter-rater Agreement on Partial-Thickness Retear 6 Months 0.13 (poor) 0.19 (poor) 0.15 (poor)
24 Months 0.44 (moderate) 0.33 (fair) 0.53 (moderate)
7
ROTATOR CUFF REPAIR MRI AGREEMENT Table 7. Intra- and Inter-rater Agreement on Footprint Coverage, Tendon Thickness, and Signal Intensity in Burks et al.17 Classification Intrarater Agreement Footprint coverage Tendon thickness Signal intensity
6 Months 0.22-0.53 (fair to moderate) 0.39-0.66 (fair to good) 0.38-0.61 (fair to good)
Inter-rater Agreement
24 Months 0.23-0.74 (fair to good) 0.21-0.64 (fair to good) 0.35-0.64 (fair to good)
6 Months 0.16 (poor) 0.30 (fair) 0.23 (fair)
24 Months 0.31 (fair) 0.35 (fair) 0.43 (moderate)
NOTE. Kappa values and interpretations of agreement in parentheses are presented.
inter-rater agreement at 24 months after ARCR than at 6 months in not only full- but also partial-thickness retear. These results suggest that evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used. Limitations This study had some limitations. First, it did not include arthroscopic assessment of the postoperative rotator cuff to determine whether the MRI findings correctly reflected the rotator cuff integrity after repair. Therefore, we were not able to confirm the accuracy of the MRI findings. Instead, the study was designed to evaluate the inter- and intraobserver reliability of previously established rotator cuff integrity classification and scoring systems: its goal was to determine whether systems for postoperative MRI assessment of cuff integrity were comparatively reliable. MRI assessments before surgery have sensitivities of 90% to 100% in predicting full-thickness tear; in the case of partialthickness tear, the sensitivity ranges from 35% to 82%.22,30-32 Owen et al.14 examined the accuracy of MRI evaluation after surgery in 31 patients before reoperation and found an accuracy of 90% using their evaluation system. Taking the results of these studies and ours into consideration, we believe that MRI has sufficiently high accuracy to be used for structural assessment of the rotator cuff after surgery. The second limitation of our study is that the MRI images were reviewed only by shoulder surgeons but not by musculoskeletal radiologists. However, our shoulder surgeons demonstrated good or excellent intrarater agreement and moderate or good inter-rater agreement in predicting full-thickness tears. Grant et al.23 reported that the reliability of MRI detection of full-thickness tears among shoulder surgeons and between shoulder surgeons and musculoskeletal radiologists was excellent using the criteria similar to the Owen, Sugaya, and Hayashida classifications: the presence of fluid signal identifying discontinuity of the central tendon on 2 imaging planes was identified as full-thickness tears. In addition, Balich et al.18 reported that sensitivity for diagnosis of partial-thickness tear was poor among a group of radiologists, regardless of the radiologists’ levels of experience and knowledge of the arthroscopic findings using the same criteria as the
Owen classification system. We therefore believe that our results do not differ markedly from those of previous studies performed by radiologists. Third, our study did not include fat-suppression images or magnetic resonance arthrography, the use of which might improve sensitivity and levels of agreement. Singson et al.20 evaluated MRI images of 177 patients; when fat-suppression images were used, 21 partialthickness tears were diagnosed that were not detected earlier without the use of fat-suppression images. Magnetic resonance arthrography improves the ability to diagnose both full- and partial-thickness tears, although it is more invasive than MRI.33-35 Further study is needed to investigate the usefulness of fat-suppression images and magnetic resonance arthrography in evaluations of rotator cuff integrity after ARCR. Fourth, this study did not have a statistical comparison among 3 classification systems. We therefore cannot conclude from our results which MRI evaluation system is the best for the assessment of rotator cuff integrity after repair.
Conclusions Shoulder surgeons showed better intra- and inter-rater agreement in predicting full-thickness tears than in predicting partial-thickness tears. The inter-rater agreement at 24 months after ARCR was superior to that at 6 months in predicting not only full-thickness but also partial-thickness retear. MRI evaluation of rotator cuff integrity at 6 months after ARCR may be less reliable, regardless of which classification system is used.
Acknowledgment The authors thank Yasuichiro Nishimura for his assistance with statistical analysis.
References 1. Mihata T, Watanabe C, Fukunishi K, et al. Functional and structural outcomes of single-row versus double-row versus combined double-row and suture-bridge repair for rotator cuff tears. Am J Sports Med 2011;39:2091-2098. 2. Sugaya H, Maeda K, Matsuki K, Moriishi J. Repair integrity and functional outcome after arthroscopic double-row rotator cuff repair. A prospective outcome study. J Bone Joint Surg Am 2007;89:953-960. 3. Boileau P, Brassart N, Watkinson DJ, Carles M, Hatzidakis AM, Krishnan SG. Arthroscopic repair of
8
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
A. HASEGAWA ET AL. full-thickness tears of the supraspinatus: does the tendon really heal? J Bone Joint Surg Am 2005;87:1229-1240. Harryman DT 2nd, Mack LA, Wang KY, Jackins SE, Richardson ML, Matsen FA 3rd. Repairs of the rotator cuff. Correlation of functional results with integrity of the cuff. J Bone Joint Surg Am 1991;73:982-989. Gazielly DF, Gleyze P, Montagnon C. Functional and anatomical results after rotator cuff repair. Clin Orthop Relat Res 1994:43-53. Thomazeau H, Boukobza E, Morcet N, Chaperon J, Langlais F. Prediction of rotator cuff repair results by magnetic resonance imaging. Clin Orthop Relat Res 1997:275-283. Gerber C, Fuchs B, Hodler J. The results of repair of massive tears of the rotator cuff. J Bone Joint Surg Am 2000;82:505-515. Galatz LM, Ball CM, Teefey SA, Middleton WD, Yamaguchi K. The outcome and repair integrity of completely arthroscopically repaired large and massive rotator cuff tears. J Bone Joint Surg Am 2004;86:219-224. Liu SH, Baker CL. Arthroscopically assisted rotator cuff repair: Correlation of functional results with integrity of the cuff. Arthroscopy 1994;10:54-60. Jost B, Pfirrmann CW, Gerber C, Switzerland Z. Clinical outcome after structural failure of rotator cuff repairs. J Bone Joint Surg Am 2000;82:304-314. Park MC, Elattrache NS, Ahmad CS, Tibone JE. “Transosseous-equivalent” rotator cuff repair technique. Arthroscopy 2006;22:1360.e1-1360.e5. Park MC, Tibone JE, ElAttrache NS, Ahmad CS, Jun BJ, Lee TQ. Part II: Biomechanical assessment for a footprintrestoring transosseous-equivalent rotator cuff repair technique compared with a double-row repair technique. J Shoulder Elbow Surg 2007;16:469-476. Mihata T, Fukuhara T, Jun BJ, Watanabe C, Kinoshita M. Effect of shoulder abduction angle on biomechanical properties of the repaired rotator cuff tendons with 3 types of double-row technique. Am J Sports Med 2011;39:551-556. Owen RS, Iannotti JP, Kneeland JB, Dalinka MK, Deren JA, Oleaga L. Shoulder after surgery: MR imaging with surgical validation. Radiology 1993;186:443-447. Sugaya H, Maeda K, Matsuki K, Moriishi J. Functional and structural outcome after arthroscopic full-thickness rotator cuff repair: Single-row versus dual-row fixation. Arthroscopy 2005;21:1307-1316. Hayashida K, Tanaka M, Koizumi K, Kakiuchi M. Characteristic retear patterns assessed by magnetic resonance imaging after arthroscopic double-row rotator cuff repair. Arthroscopy 2012;28:458-464. Burks RT, Crim J, Brown N, Fink B, Greis PE. A prospective randomized clinical trial comparing arthroscopic single- and double-row rotator cuff repair: Magnetic resonance imaging and early clinical evaluation. Am J Sports Med 2009;37:674-682. Balich SM, Sheley RC, Brown TR, Sauser DD, Quinn SF. MR imaging of the rotator cuff tendon: Interobserver agreement and analysis of interpretive errors. Radiology 1997;204:191-194. Robertson PL, Schweitzer ME, Mitchell DG, et al. Rotator cuff disorders: Interobserver and intraobserver variation in diagnosis with MR imaging. Radiology 1995;194:831-835.
20. Singson RD, Hoang T, Dan S, Friedman M. MR evaluation of rotator cuff pathology using T2-weighted fast spin-echo technique with and without fat suppression. AJR Am J Roentgenol 1996;166:1061-1065. 21. Spencer EE Jr, Dunn WR, Wright RW, et al. Interobserver agreement in the classification of rotator cuff tears using magnetic resonance imaging. Am J Sports Med 2008;36: 99-103. 22. Teefey SA, Rubin DA, Middleton WD, Hildebolt CF, Leibold RA, Yamaguchi K. Detection and quantification of rotator cuff tears. Comparison of ultrasonographic, magnetic resonance imaging, and arthroscopic findings in seventy-one consecutive cases. J Bone Joint Surg Am 2004;86:708-716. 23. Grant JA, Miller BS, Jacobson JA, et al. Intra- and interrater reliability of the detection of tears of the supraspinatus central tendon on MRI by shoulder surgeons. J Shoulder Elbow Surg 2013;22:725-731. 24. Iannotti JP, Deutsch A, Green A, et al. Time to failure after rotator cuff repair: a prospective imaging study. J Bone Joint Surg Am 2013;95:965-971. 25. Cho NS, Yi JW, Lee BG, Rhee YG. Retear patterns after arthroscopic rotator cuff repair: Single-row versus suture bridge technique. Am J Sports Med 2010;38:664-671. 26. Crim J, Burks R, Manaster BJ, Hanrahan C, Hung M, Greis P. Temporal evolution of MRI findings after arthroscopic rotator cuff repair. AJR Am J Roentgenol 2010;195: 1361-1366. 27. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174. 28. Khazzam M, Kuhn JE, Mulligan E, et al. Magnetic resonance imaging identification of rotator cuff retears after repair: Interobserver and intraobserver agreement. Am J Sports Med 2012;40:1722-1727. 29. Gusmer PB, Potter HG, Donovan WD, O’Brien SJ. MR imaging of the shoulder after rotator cuff repair. AJR Am J Roentgenol 1997;168:559-563. 30. Iannotti JP, Zlatkin MB, Esterhai JL, Kressel HY, Dalinka MK, Spindler KP. Magnetic resonance imaging of the shoulder. Sensitivity, specificity, and predictive value. J Bone Joint Surg Am 1991;73:17-29. 31. Quinn SF, Sheley RC, Demlow TA, Szumowski J. Rotator cuff tendon tears: evaluation with fat-suppressed MR imaging with arthroscopic correlation in 100 patients. Radiology 1995;195:497-500. 32. Reinus WR, Shady KL, Mirowitz SA, Totty WG. MR diagnosis of rotator cuff tears of the shoulder: Value of using T2-weighted fat-saturated images. AJR Am J Roentgenol 1995;164:1451-1455. 33. Ferrari FS, Governi S, Burresi F, Vigni F, Stefani P. Supraspinatus tendon tears: Comparison of US and MR arthrography with surgical correlation. Eur Radiol 2002;12:1211-1217. 34. Toyoda H, Ito Y, Tomo H, Nakao Y, Koike T, Takaoka K. Evaluation of rotator cuff tears with magnetic resonance arthrography. Clin Orthop Relat Res 2005;439:109-115. 35. de Jesus JO, Parker L, Frangos AJ, Nazarian LN. Accuracy of MRI, MR arthrography, and ultrasound in the diagnosis of rotator cuff tears: A meta-analysis. AJR Am J Roentgenol 2009;192:1701-1707.