- Email: [email protected]
The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants
THEKNE-02358; No of Pages 7 The Knee xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
The Knee
The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants A. Hirschmann a,1, F.F. Schiapparelli b,1, L. Schenk b, L. Keller b, F. Amsler c, M.T. Hirschmann b,⁎ a b c
Radiology and Nuclear Medicine, University Hospital Basel, Switzerland Department of Orthopaedic Surgery and Traumatology, Kantonsspital Baselland (Bruderholz, Liestal, Laufen), CH-4101 Bruderholz, Switzerland Amsler Consulting, Basel, Switzerland
a r t i c l e
i n f o
Article history: Received 24 June 2016 Received in revised form 15 October 2016 Accepted 19 October 2016 Available online xxxx Keywords: Collagen meniscus MRI Genovese grading Knee Meniscal substitution
a b s t r a c t Background: The purpose of the study was to evaluate the intra- and inter-observer reliabilities of the Genovese grading on MRI in patients after collagen meniscus substitution. Methods: 84 MRI images of 74 consecutive patients who underwent partial meniscus substitution using collagen meniscus implant (CMI) were assessed. MRIs were evaluated using the Genovese grading system. Furthermore, meniscal extrusion was assessed. Two observers performed the grading twice, blinded to each other and to the previous results, with a six weeks interval. The inter- and intra-observer reliabilities were assessed using kappa and weighted kappa values. Results: The criterion “morphology/size” showed a weighted kappa value inter-observer reliability of 0.069 (rater 1)/0.352 (rater 2) and intra-observer reliability of 0.170 (1st rating)/ 0.582 (2nd rating). The criterion “signal intensity” showed a weighted kappa value inter-observer reliability of 0.175/0.284 and intra-observer reliability of 0.294/0.458. The criterion “cartilage lesions” showed a kappa value inter-observer reliability of 0.091/0.525 and intra-observer reliability of 0.409/0.413. The criterion “bone marrow edema” showed a kappa value inter-observer reliability of 0.667/0.808 and intra-observer reliability of 0.702/0.715. The criterion “cartilage lesions” showed a kappa value inter-observer reliability of 0.091/0.525 and intra-observer reliability of 0.409/0.413. Regarding meniscal extrusion kappa values for the inter-observer reliability were 0.625/0.940 and 0.625/0.889 for intra-observer reliability. Conclusions: Three of the four Genovese grading items showed only slight to moderate interand intra-observer reliabilities in evaluating CMI on MRI. Hence, such grading results need to be considered with all due care. Only the criteria “bone marrow edema” and “meniscal extrusion” showed a good agreement for both inter- and intra-observer reliabilities. © 2016 Elsevier B.V. All rights reserved.
1. Introduction Meniscal substitution using either a collagen or polyurethane meniscus is an important part of the surgeon's armamentarium to treat medial compartment overloading or early osteoarthritis (OA) after partial or subtotal meniscectomy [1–6]. A number of ⁎ Corresponding author at: Kantonsspital Baselland (Bruderholz, Liestal, Laufen), Department of Orthopaedic Surgery and Traumatology, CH-4101 Bruderholz, Switzerland. E-mail addresses: [email protected], [email protected] (M.T. Hirschmann). 1 The first and second authors contributed equally to this study.
http://dx.doi.org/10.1016/j.knee.2016.10.011 0968-0160/© 2016 Elsevier B.V. All rights reserved.
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
2
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
studies have reported clinical and radiological results including magnetic resonance imaging (MRI) findings after meniscal substitution [1,2,4,5,7–13]. Although there is not always a clear relationship between clinical symptoms and MRI findings, MRI is currently considered as gold standard for radiological follow-up of patients after partial meniscal substitution [12–14]. The most commonly used MRI scoring system after partial meniscal substitution is the one described in 2007 by Genovese et al. [9]. It consists of direct and indirect criteria. Direct criteria assess the size, morphology and the signal intensity of the meniscal implant. Indirect criteria assess possible associated cartilage lesions and bone marrow edema. Although these criteria are widely used in clinical studies the intra-observer as well as the inter-observer reliabilities have not been evaluated so far. Hence, it was the primary purpose of the present study to evaluate the intra- and inter-observer reliabilities of the Genovese grading in patients after collagen meniscus implant (CMI). The secondary purpose was to evaluate the criterion “meniscal extrusion” as a complement to the MRI evaluation of CMIs. 2. Materials and methods 2.1. Participants MRIs of 84 knees and 74 patients (mean age ± standard deviation 37 ± 10, male:female = 50:24) after collagen meniscus implantation (CMI, Ivy Sports Medicine, Switzerland) were prospectively collected. Only patients with at least one-year followup after collagen meniscus implantation were included in this study. Prior to surgery, all patients had a loss of meniscal tissue more than 25% and also complained about medial sided knee pain. Ethical approval was obtained from the local Ethics Committee. Informed consent was obtained from all individual participants included in the study. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. 2.2. Radiological evaluation MRI was performed preoperatively and at minimum one-year follow-up using 0.35 T to three Tesla MRI systems including a dedicated knee coil. The MRI protocol included T1-weighted (repetition time ms/echo time ms, 511/16) and proton densityweighted fat-saturated coronal images (3000/15), proton density-weighted sagittal images (3400/17) and gradient echo fatsaturated axial images (24.4/7) with a slice thickness of three to four millimeters. MRI images were retrospectively evaluated using the Genovese grading system [9]. Direct and indirect criteria were assessed on proton density-weighted fat-saturated coronal and proton density-weighted sagittal images (Table 1; Figures 1 and 2).
Table 1 Quantitative MRI assessment of the collagen meniscal implant and the adjacent knee compartment using the Genovese criteria and additionally meniscal extrusion. Observer 1
Genovese criteria
Direct criteria
Indirect criteria
Morphology and size Totally resorbed CMI Small CMI with regular and/or irregular morphology CMI with identical shape and size to the normal meniscus Total observations Signal intensity Markedly hyperintense Slightly hyperintense Isointense relatively to the normal meniscus Total observations Cartilage lesions in relation to present cartilage N50% b50% Total observations Bone marrow edema Present Not present Total observations Meniscal extrusion Present (b3 mm) Not present (N3 mm) Total observations
Observer 2
Total
1st observation
2nd observation
1st observation
2nd observation
N
%
N
%
N
%
N
%
%
6 69 9 84
7.1% 82.1% 10.7% 100.0%
5 75 1 81
6.2% 92.6% 1.2% 100.0%
10 49 25 84
11.9% 58.3% 29.8% 100.0%
11 32 39 82
13.4% 39.0% 47.6% 100.0%
9.7% 68.0% 22.4% 100.0%
13 70 1 84
15.5% 83.3% 1.2% 100.0%
15 63 3 81
18.5% 77.8% 3.7% 100.0%
22 59 3 84
26.2% 70.2% 3.6% 100.0%
15 58 9 82
18.3% 70.7% 11.0% 100.0%
19.6% 75.5% 4.8% 100.0%
47 37 84
56.0% 44.0% 100.0%
47 34 81
58.0% 42.0% 100.0%
56 28 84
66.7% 33.3% 100.0%
60 22 82
73.2% 26.8% 100.0%
63.4% 36.6% 100.0%
44 40 84
52.4% 47.6% 100.0%
45 36 81
55.6% 44.4% 100.0%
48 36 84
57.1% 42.9% 100.0%
55 27 82
67.1% 32.9% 100.0%
58.0% 42.0% 100.0%
26 58 84
31.0% 69.0% 100.0%
28 53 81
34.6% 65.4% 100.0%
24 60 84
28.6% 71.4% 100.0%
39 43 82
47.6% 52.4% 100.0%
35.3% 64.7% 100.0%
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
3
Figure 1. Morphological MRI characteristics of the CMI according to the Genovese grading system. One year postoperatively the CMI is totally resorbed (b; arrowheads) as compared to an MRI 6 months postoperatively (a; arrowheads), which was not included in the evaluation of the study. (c; arrowheads) CMI with identical size, shape and signal intensity compared to the regular meniscus (Type 3).
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
4
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
Figure 2. Indirect MRI criteria for evaluation of the CMI are shown with (a; arrowheads) a cartilage lesion greater than 50% (Type 1) and (b; arrowheads) a cartilage lesion less than 50% on sagittal proton density-weighted images (Type 2).
Meniscal extrusion was also assessed. The meniscus was judged as “extruded” when overlapping the medial or lateral joint edge more than three millimeters. 2.3. Statistical methods Two experienced musculoskeletal radiologists analyzed the MR images independently with a six weeks interval twice. These two observers were blinded to their own and previous results. In the process of image analysis, the observers were allowed to modify individually the intensity, contrast and zoom of MR images. Data of the ordinally scaled direct criteria were analyzed for intra- and inter-observer reliabilities using the weighted kappa. Data of the nominally scaled indirect criteria and meniscal extrusion for inter- and intra-observer reliabilities were analyzed using the kappa-coefficient. Intra-observer reliability coefficients were presented for observer 1 and observer 2 separately, inter-observer reliability for first and second rating separately. Results have been evaluated as described from Landis and Koch [15]. Kappa values of 0 to 0.2 were indicative of slight agreement, 0.21 to 0.4 fair agreement, 0.41 to 0.6 moderate agreement, 0.61 to 0.8 substantial agreement and 0.81 to one almost perfect agreement. For all analyses, p b 0.05 was considered statistically significant. Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
5
Table 2 Inter- and intra-observer reliabilities (weighted Cohen's kappa value) of direct MRI criteria in the assessment of the collagen meniscus implant.
Morphology and size Signal intensity
Inter-observer reliability (observer 1/observer 2)
Intra-observer reliability (1st rating/2nd rating)
0.069/0.352 0.175/0.284
0.170/0.582 0.294/0.458
3. Results The results from the two observers are presented in Table 1. For the direct criteria the following was found: The analysis of morphology and size of the meniscal implant showed a slight to fair agreement for inter-observer reliability and a slight to moderate agreement for intra-observer reliability (Table 2). The signal intensity grading showed a slight to fair agreement for inter-observer reliability and a fair to moderate agreement for intra-observer reliability (Table 2). For the indirect criteria the following was found: The MRI evaluation of cartilage lesions grading showed a slight to fair agreement for inter-observer reliability and a moderate agreement for intra-observer-reliability (Table 3). The bone marrow edema showed a substantial to almost perfect agreement for inter-observer and intra-observer reliabilities (Table 3). The evaluation of meniscus extrusion with the kappa value presented a substantial to almost perfect agreement or for both inter and intra-observer reliabilities (Table 3). 4. Discussion The most important findings of the present study were the following: firstly, the direct criteria of the Genovese grading (morphology, size and signal intensity of the meniscal implant) showed only slight to moderate inter- and intra-observerreliabilities. Considering the experience of the observers, the detailed instruction of the observers and the fact that all parameters were clearly presented, no misunderstandings were possible. Based on the findings of the present study the capability to differentiate a totally reabsorbed CMI (Type 1) from a small CMI with regular and/or irregular morphology (Type 2) and from a CMI with identical shape and size to the normal meniscus (Type 3) is slight to fair. In the same way it appears difficult to differentiate between a slightly hyperintense or a markedly hyperintense signal intensity of the collagen meniscus/residual meniscus complex. A higher reliability could be achieved giving a more detailed definition of the specific grading parameter. With regard to signal intensity, creating a guideline table, with sample images in which the signal intensity of the three types is shown, could serve as guidance and improve the MR rating (Figures 1a–c, 2a,b). It could also be speculated if an introduction of more grading levels into the criteria of “morphology and size” would lead to an improved specificity and more clinical significance of the measurements. Secondly, the indirect criteria of the Genovese grading system (bone marrow edema and cartilage lesion) showed a varying but in general better inter- and intra-observer reliabilities. For assessment of bone marrow edema the intra- and inter-observer agreements were good. The explanation for these improved findings, when compared to the direct criteria, could be the clearer cut-off between presence and absence of bone marrow edema. However, assessment reliability of the criteria “cartilage lesion” was slight to moderate for inter- and intra-observer reliabilities. This finding could be due to the use of proton density-weighted fat-saturated coronal and non fat-saturated sagittal sequences with an echo time of 17 ms. In sequences with a short echo time, joint fluid and hyaline cartilage have almost the same signal intensities and differentiation is limited. Increasing the echo time to 30 to 40 ms (intermediate-weighted sequence) can resolve this restriction. Yulish et al. [16] proposed to classify cartilage lesions into five different grades. A grade I cartilage lesion is defined as a cartilage with normal contour and signal abnormality. A grade II cartilage lesion is defined as a cartilage with superficial fraying and erosion or ulceration of less than 50% of thickness. A grade III cartilage lesion is defined as a partial-thickness defect of more than 50% but less than 100% and a grade IV as a cartilage with a full-thickness defect. Normal cartilage was classified as grade 0. The
Table 3 Inter- and intra-observer reliabilities (kappa value) of indirect MRI criteria and meniscal extrusion in the assessment of the collagen meniscus implant.
Cartilage lesion Bone marrow edema Meniscal extrusion
Inter-observer reliability (observer 1/observer 2)
Intra-observer reliability (1st rating/2nd rating)
0.091/0.525 0.667/0.808 0.625/0.940
0.409/0.413 0.702/0.715 0.625/0.889
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
6
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
Yulish score was modified by Verdonk et al. [13]. They introduced intermediate grades like grade 3.5 for cartilage lesions between grade III and grade IV. Another important aspect should be considered. Despite an established role of MRI in the diagnosis of knee pathologies such as cartilage, meniscus and ligament lesions [7,12], the role of MRI in the postoperative evaluation of meniscus implantation is less clear. There is only a moderate correlation of clinical symptoms and MRI findings [12–14]. With regard to the Genovese grading, current studies showed that clinical results after a collagen meniscus implantation are better than the reported radiological outcomes [10,16,17]. Thirdly the evaluation of meniscus extrusion presented good to excellent agreement for both inter- and intra-observer reliabilities. The clear and measurable cutoff of three millimeters was probably the reason for this high agreement. In summary, the Genovese grading system, especially with regard to direct criteria, is not reliable enough to draw meaningful clinical conclusions on the state and healing of a collagen meniscus substitution. Clearly, a revised and more specific grading system is needed for clinical research as well as clinical follow-up purposes. In particular, the direct criteria need to be improved. Only the indirect criteria “bone marrow edema” and “meniscal extrusion” might be recommended for follow-up of collagen meniscus substitution. However, to date there is no better alternative to the Genovese grading system available. Some limitations have to be acknowledged. Firstly, the findings of the study might have been influenced by the heterogeneity of the included patients. Hence, these should be interpreted with all due care. However, the patients included here represent a normal study sample of a knee clinic. Secondly, the follow-up of the patients included was limited to one year after collagen meniscus implantation. A fact, which might influence the reliability assessment, as it is known that the integration process of collagen meniscus implants lasts for several years. An analysis of MR images at a later follow-up could give different results. Thirdly, in this study MRI was performed using 0.35 T to three Tesla MRI systems, the grading published from Genovese was based on images coming from a 1.5 T MRI [9]. This could have influenced the quality of our results, however, our study included also MRI with a magnetic field up to three Tesla and this could balance the results obtained from MRI with a lower magnetic field. It could be speculated if the use of 3D reconstructed MRI is superior to 2D MRI. In 3D MRI size and morphology of the CMI might be better identified and measured. MRI would then give the surgeon more precise and secure information about the actual size and morphology of the implant. Based on the findings of the present study a more reliable MRI grading system for patients after collagen meniscus substitution is needed. However, as there is so far no better alternative to the Genovese grading system it should be improved in particular in terms of the direct criteria. 5. Conclusions Three of four Genovese grading items showed only slight to moderate inter- and intra-observer reliabilities for the evaluation of patients after partial meniscus substitution using collagen meniscus implants. Hence, such grading results need to be considered with all due care. Only the criteria “bone marrow edema” and “meniscal extrusion” showed a good agreement for both inter and intra-observer reliabilities. Conflict of interest The authors declare that they have no conflicts of interest. References [1] Stone KR, Rodkey WG, Webber R, McKinney L, Steadman JR. Meniscal regeneration with copolymeric collagen scaffolds. In vitro and in vivo studies evaluated clinically, histologically, and biochemically. Am J Sports Med 1992;20:101–11. [2] Steadman JR, Rodkey WG. Tissue-engineered collagen meniscus implants: 5- to 6-year feasibility study results. Arthroscopy 2005;21:515–25. [3] Zaffagnini S, Giordano G, Vascellari A, Bruni D, Neri MP, Iacono F, et al. Arthroscopic collagen meniscus implant results at 6 to 8 years follow up. Knee Surg Sports Traumatol Arthrosc 2006;15:175–83. [4] Rodkey WG, DeHaven KE, Montgomery WH, Baker CLJ, Beck CLJ, Hormel SE, et al. Comparison of the collagen meniscus implant with partial meniscectomy. A prospective randomized trial. J Bone Joint Surg Am 2008;90:1414–26. [5] Bulgheroni P, Murena L, Ratti C, Bulgheroni E, Ronga M, Cherubino P. Follow-up of collagen meniscus implant patients: clinical, radiological, and magnetic resonance imaging results at 5 years. Knee 2009;17:224–9. [6] Verdonk R, Verdonk P, Heinrichs EL. Polyurethane meniscus implant: technique. In: Beaufils P, Verdonk R, editors. The Meniscus. Berlin Heidelberg: Springer; 2010. [7] Lefevre N, Naouri JF, Herman S, Gerometta A, Klouche S, Bohu Y. A current review of the meniscus imaging: proposition of a useful tool for its radiologic analysis. Radiol Res Pract 2016;2016:25. [8] Hirschmann MT, Keller L, Hirschmann A, Schenk L, Berbig R, Lüthi U, et al. One-year clinical and MR imaging outcome after partial meniscal replacement in stabilized knees using a collagen meniscus implant. Knee Surg Sports Traumatol Arthrosc 2013;21:740–7. [9] Genovese E, Angeretti MG, Ronga M, Leonardi A, Novario R, Callegari L, et al. Follow-up of collagen meniscus implants by MRI. Radiol Med 2007;112:1036–48. [10] Monllau JC, Gelber PE, Abat F, Pelfort X, Abad R, Hinarejos P, et al. Outcome after partial medial meniscus substitution with the collagen meniscal implant at a minimum of 10 years' follow-up. Arthroscopy 2011;27:933–43. [11] Ronga M, Bulgheroni P, Manelli A, Genovese E, Grassi F, Cherubino P. Short-term evaluation of collagen meniscus implants by MRI and morphological analysis. J Orthop Traumatol 2002;4:5–10. [12] van Arkel ER, Goei R, De Ploeg I, De Boer HH. Meniscal allografts: evaluation with magnetic resonance imaging and correlation with arthroscopy. Arthroscopy 2000;16:517–21. [13] Verdonk PC, Verstraete KL, Almqvist KF, De Cuyper K, Veys EM, Verbruggen G, et al. Meniscal allograft transplantation: long-term clinical results with radiological and magnetic resonance imaging correlations. Knee Surg Sports Traumatol Arthrosc 2006;14:694–706. [14] Verstraete KL, Verdonk R, Lootens T, Verstraete P, De Rooy J, Kunnen M. Current status and imaging of allograft meniscal transplantation. Eur J Radiol 1997;26: 16–22.
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
7
[15] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. [16] Yulish BS, Montanez J, Goodfellow DB, Bryan PJ, Mulopulos GP, Modic MT. Chondromalacia patellae: assessment with MR imaging. Radiology 1987;164:763–6. [17] Zaffagnini S, Marcheggiani MGM, Bulgheroni P, Bulgheroni E, Grassi A, Bonanzinga T, et al. Arthroscopic collagen meniscus implantation for partial lateral meniscal defects: a 2-year minimum follow-up study. Am J Sports Med 2012;40:2281–8.
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
Contents lists available at ScienceDirect
The Knee
The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants A. Hirschmann a,1, F.F. Schiapparelli b,1, L. Schenk b, L. Keller b, F. Amsler c, M.T. Hirschmann b,⁎ a b c
Radiology and Nuclear Medicine, University Hospital Basel, Switzerland Department of Orthopaedic Surgery and Traumatology, Kantonsspital Baselland (Bruderholz, Liestal, Laufen), CH-4101 Bruderholz, Switzerland Amsler Consulting, Basel, Switzerland
a r t i c l e
i n f o
Article history: Received 24 June 2016 Received in revised form 15 October 2016 Accepted 19 October 2016 Available online xxxx Keywords: Collagen meniscus MRI Genovese grading Knee Meniscal substitution
a b s t r a c t Background: The purpose of the study was to evaluate the intra- and inter-observer reliabilities of the Genovese grading on MRI in patients after collagen meniscus substitution. Methods: 84 MRI images of 74 consecutive patients who underwent partial meniscus substitution using collagen meniscus implant (CMI) were assessed. MRIs were evaluated using the Genovese grading system. Furthermore, meniscal extrusion was assessed. Two observers performed the grading twice, blinded to each other and to the previous results, with a six weeks interval. The inter- and intra-observer reliabilities were assessed using kappa and weighted kappa values. Results: The criterion “morphology/size” showed a weighted kappa value inter-observer reliability of 0.069 (rater 1)/0.352 (rater 2) and intra-observer reliability of 0.170 (1st rating)/ 0.582 (2nd rating). The criterion “signal intensity” showed a weighted kappa value inter-observer reliability of 0.175/0.284 and intra-observer reliability of 0.294/0.458. The criterion “cartilage lesions” showed a kappa value inter-observer reliability of 0.091/0.525 and intra-observer reliability of 0.409/0.413. The criterion “bone marrow edema” showed a kappa value inter-observer reliability of 0.667/0.808 and intra-observer reliability of 0.702/0.715. The criterion “cartilage lesions” showed a kappa value inter-observer reliability of 0.091/0.525 and intra-observer reliability of 0.409/0.413. Regarding meniscal extrusion kappa values for the inter-observer reliability were 0.625/0.940 and 0.625/0.889 for intra-observer reliability. Conclusions: Three of the four Genovese grading items showed only slight to moderate interand intra-observer reliabilities in evaluating CMI on MRI. Hence, such grading results need to be considered with all due care. Only the criteria “bone marrow edema” and “meniscal extrusion” showed a good agreement for both inter- and intra-observer reliabilities. © 2016 Elsevier B.V. All rights reserved.
1. Introduction Meniscal substitution using either a collagen or polyurethane meniscus is an important part of the surgeon's armamentarium to treat medial compartment overloading or early osteoarthritis (OA) after partial or subtotal meniscectomy [1–6]. A number of ⁎ Corresponding author at: Kantonsspital Baselland (Bruderholz, Liestal, Laufen), Department of Orthopaedic Surgery and Traumatology, CH-4101 Bruderholz, Switzerland. E-mail addresses: [email protected], [email protected] (M.T. Hirschmann). 1 The first and second authors contributed equally to this study.
http://dx.doi.org/10.1016/j.knee.2016.10.011 0968-0160/© 2016 Elsevier B.V. All rights reserved.
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
2
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
studies have reported clinical and radiological results including magnetic resonance imaging (MRI) findings after meniscal substitution [1,2,4,5,7–13]. Although there is not always a clear relationship between clinical symptoms and MRI findings, MRI is currently considered as gold standard for radiological follow-up of patients after partial meniscal substitution [12–14]. The most commonly used MRI scoring system after partial meniscal substitution is the one described in 2007 by Genovese et al. [9]. It consists of direct and indirect criteria. Direct criteria assess the size, morphology and the signal intensity of the meniscal implant. Indirect criteria assess possible associated cartilage lesions and bone marrow edema. Although these criteria are widely used in clinical studies the intra-observer as well as the inter-observer reliabilities have not been evaluated so far. Hence, it was the primary purpose of the present study to evaluate the intra- and inter-observer reliabilities of the Genovese grading in patients after collagen meniscus implant (CMI). The secondary purpose was to evaluate the criterion “meniscal extrusion” as a complement to the MRI evaluation of CMIs. 2. Materials and methods 2.1. Participants MRIs of 84 knees and 74 patients (mean age ± standard deviation 37 ± 10, male:female = 50:24) after collagen meniscus implantation (CMI, Ivy Sports Medicine, Switzerland) were prospectively collected. Only patients with at least one-year followup after collagen meniscus implantation were included in this study. Prior to surgery, all patients had a loss of meniscal tissue more than 25% and also complained about medial sided knee pain. Ethical approval was obtained from the local Ethics Committee. Informed consent was obtained from all individual participants included in the study. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. 2.2. Radiological evaluation MRI was performed preoperatively and at minimum one-year follow-up using 0.35 T to three Tesla MRI systems including a dedicated knee coil. The MRI protocol included T1-weighted (repetition time ms/echo time ms, 511/16) and proton densityweighted fat-saturated coronal images (3000/15), proton density-weighted sagittal images (3400/17) and gradient echo fatsaturated axial images (24.4/7) with a slice thickness of three to four millimeters. MRI images were retrospectively evaluated using the Genovese grading system [9]. Direct and indirect criteria were assessed on proton density-weighted fat-saturated coronal and proton density-weighted sagittal images (Table 1; Figures 1 and 2).
Table 1 Quantitative MRI assessment of the collagen meniscal implant and the adjacent knee compartment using the Genovese criteria and additionally meniscal extrusion. Observer 1
Genovese criteria
Direct criteria
Indirect criteria
Morphology and size Totally resorbed CMI Small CMI with regular and/or irregular morphology CMI with identical shape and size to the normal meniscus Total observations Signal intensity Markedly hyperintense Slightly hyperintense Isointense relatively to the normal meniscus Total observations Cartilage lesions in relation to present cartilage N50% b50% Total observations Bone marrow edema Present Not present Total observations Meniscal extrusion Present (b3 mm) Not present (N3 mm) Total observations
Observer 2
Total
1st observation
2nd observation
1st observation
2nd observation
N
%
N
%
N
%
N
%
%
6 69 9 84
7.1% 82.1% 10.7% 100.0%
5 75 1 81
6.2% 92.6% 1.2% 100.0%
10 49 25 84
11.9% 58.3% 29.8% 100.0%
11 32 39 82
13.4% 39.0% 47.6% 100.0%
9.7% 68.0% 22.4% 100.0%
13 70 1 84
15.5% 83.3% 1.2% 100.0%
15 63 3 81
18.5% 77.8% 3.7% 100.0%
22 59 3 84
26.2% 70.2% 3.6% 100.0%
15 58 9 82
18.3% 70.7% 11.0% 100.0%
19.6% 75.5% 4.8% 100.0%
47 37 84
56.0% 44.0% 100.0%
47 34 81
58.0% 42.0% 100.0%
56 28 84
66.7% 33.3% 100.0%
60 22 82
73.2% 26.8% 100.0%
63.4% 36.6% 100.0%
44 40 84
52.4% 47.6% 100.0%
45 36 81
55.6% 44.4% 100.0%
48 36 84
57.1% 42.9% 100.0%
55 27 82
67.1% 32.9% 100.0%
58.0% 42.0% 100.0%
26 58 84
31.0% 69.0% 100.0%
28 53 81
34.6% 65.4% 100.0%
24 60 84
28.6% 71.4% 100.0%
39 43 82
47.6% 52.4% 100.0%
35.3% 64.7% 100.0%
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
3
Figure 1. Morphological MRI characteristics of the CMI according to the Genovese grading system. One year postoperatively the CMI is totally resorbed (b; arrowheads) as compared to an MRI 6 months postoperatively (a; arrowheads), which was not included in the evaluation of the study. (c; arrowheads) CMI with identical size, shape and signal intensity compared to the regular meniscus (Type 3).
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
4
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
Figure 2. Indirect MRI criteria for evaluation of the CMI are shown with (a; arrowheads) a cartilage lesion greater than 50% (Type 1) and (b; arrowheads) a cartilage lesion less than 50% on sagittal proton density-weighted images (Type 2).
Meniscal extrusion was also assessed. The meniscus was judged as “extruded” when overlapping the medial or lateral joint edge more than three millimeters. 2.3. Statistical methods Two experienced musculoskeletal radiologists analyzed the MR images independently with a six weeks interval twice. These two observers were blinded to their own and previous results. In the process of image analysis, the observers were allowed to modify individually the intensity, contrast and zoom of MR images. Data of the ordinally scaled direct criteria were analyzed for intra- and inter-observer reliabilities using the weighted kappa. Data of the nominally scaled indirect criteria and meniscal extrusion for inter- and intra-observer reliabilities were analyzed using the kappa-coefficient. Intra-observer reliability coefficients were presented for observer 1 and observer 2 separately, inter-observer reliability for first and second rating separately. Results have been evaluated as described from Landis and Koch [15]. Kappa values of 0 to 0.2 were indicative of slight agreement, 0.21 to 0.4 fair agreement, 0.41 to 0.6 moderate agreement, 0.61 to 0.8 substantial agreement and 0.81 to one almost perfect agreement. For all analyses, p b 0.05 was considered statistically significant. Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
5
Table 2 Inter- and intra-observer reliabilities (weighted Cohen's kappa value) of direct MRI criteria in the assessment of the collagen meniscus implant.
Morphology and size Signal intensity
Inter-observer reliability (observer 1/observer 2)
Intra-observer reliability (1st rating/2nd rating)
0.069/0.352 0.175/0.284
0.170/0.582 0.294/0.458
3. Results The results from the two observers are presented in Table 1. For the direct criteria the following was found: The analysis of morphology and size of the meniscal implant showed a slight to fair agreement for inter-observer reliability and a slight to moderate agreement for intra-observer reliability (Table 2). The signal intensity grading showed a slight to fair agreement for inter-observer reliability and a fair to moderate agreement for intra-observer reliability (Table 2). For the indirect criteria the following was found: The MRI evaluation of cartilage lesions grading showed a slight to fair agreement for inter-observer reliability and a moderate agreement for intra-observer-reliability (Table 3). The bone marrow edema showed a substantial to almost perfect agreement for inter-observer and intra-observer reliabilities (Table 3). The evaluation of meniscus extrusion with the kappa value presented a substantial to almost perfect agreement or for both inter and intra-observer reliabilities (Table 3). 4. Discussion The most important findings of the present study were the following: firstly, the direct criteria of the Genovese grading (morphology, size and signal intensity of the meniscal implant) showed only slight to moderate inter- and intra-observerreliabilities. Considering the experience of the observers, the detailed instruction of the observers and the fact that all parameters were clearly presented, no misunderstandings were possible. Based on the findings of the present study the capability to differentiate a totally reabsorbed CMI (Type 1) from a small CMI with regular and/or irregular morphology (Type 2) and from a CMI with identical shape and size to the normal meniscus (Type 3) is slight to fair. In the same way it appears difficult to differentiate between a slightly hyperintense or a markedly hyperintense signal intensity of the collagen meniscus/residual meniscus complex. A higher reliability could be achieved giving a more detailed definition of the specific grading parameter. With regard to signal intensity, creating a guideline table, with sample images in which the signal intensity of the three types is shown, could serve as guidance and improve the MR rating (Figures 1a–c, 2a,b). It could also be speculated if an introduction of more grading levels into the criteria of “morphology and size” would lead to an improved specificity and more clinical significance of the measurements. Secondly, the indirect criteria of the Genovese grading system (bone marrow edema and cartilage lesion) showed a varying but in general better inter- and intra-observer reliabilities. For assessment of bone marrow edema the intra- and inter-observer agreements were good. The explanation for these improved findings, when compared to the direct criteria, could be the clearer cut-off between presence and absence of bone marrow edema. However, assessment reliability of the criteria “cartilage lesion” was slight to moderate for inter- and intra-observer reliabilities. This finding could be due to the use of proton density-weighted fat-saturated coronal and non fat-saturated sagittal sequences with an echo time of 17 ms. In sequences with a short echo time, joint fluid and hyaline cartilage have almost the same signal intensities and differentiation is limited. Increasing the echo time to 30 to 40 ms (intermediate-weighted sequence) can resolve this restriction. Yulish et al. [16] proposed to classify cartilage lesions into five different grades. A grade I cartilage lesion is defined as a cartilage with normal contour and signal abnormality. A grade II cartilage lesion is defined as a cartilage with superficial fraying and erosion or ulceration of less than 50% of thickness. A grade III cartilage lesion is defined as a partial-thickness defect of more than 50% but less than 100% and a grade IV as a cartilage with a full-thickness defect. Normal cartilage was classified as grade 0. The
Table 3 Inter- and intra-observer reliabilities (kappa value) of indirect MRI criteria and meniscal extrusion in the assessment of the collagen meniscus implant.
Cartilage lesion Bone marrow edema Meniscal extrusion
Inter-observer reliability (observer 1/observer 2)
Intra-observer reliability (1st rating/2nd rating)
0.091/0.525 0.667/0.808 0.625/0.940
0.409/0.413 0.702/0.715 0.625/0.889
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
6
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
Yulish score was modified by Verdonk et al. [13]. They introduced intermediate grades like grade 3.5 for cartilage lesions between grade III and grade IV. Another important aspect should be considered. Despite an established role of MRI in the diagnosis of knee pathologies such as cartilage, meniscus and ligament lesions [7,12], the role of MRI in the postoperative evaluation of meniscus implantation is less clear. There is only a moderate correlation of clinical symptoms and MRI findings [12–14]. With regard to the Genovese grading, current studies showed that clinical results after a collagen meniscus implantation are better than the reported radiological outcomes [10,16,17]. Thirdly the evaluation of meniscus extrusion presented good to excellent agreement for both inter- and intra-observer reliabilities. The clear and measurable cutoff of three millimeters was probably the reason for this high agreement. In summary, the Genovese grading system, especially with regard to direct criteria, is not reliable enough to draw meaningful clinical conclusions on the state and healing of a collagen meniscus substitution. Clearly, a revised and more specific grading system is needed for clinical research as well as clinical follow-up purposes. In particular, the direct criteria need to be improved. Only the indirect criteria “bone marrow edema” and “meniscal extrusion” might be recommended for follow-up of collagen meniscus substitution. However, to date there is no better alternative to the Genovese grading system available. Some limitations have to be acknowledged. Firstly, the findings of the study might have been influenced by the heterogeneity of the included patients. Hence, these should be interpreted with all due care. However, the patients included here represent a normal study sample of a knee clinic. Secondly, the follow-up of the patients included was limited to one year after collagen meniscus implantation. A fact, which might influence the reliability assessment, as it is known that the integration process of collagen meniscus implants lasts for several years. An analysis of MR images at a later follow-up could give different results. Thirdly, in this study MRI was performed using 0.35 T to three Tesla MRI systems, the grading published from Genovese was based on images coming from a 1.5 T MRI [9]. This could have influenced the quality of our results, however, our study included also MRI with a magnetic field up to three Tesla and this could balance the results obtained from MRI with a lower magnetic field. It could be speculated if the use of 3D reconstructed MRI is superior to 2D MRI. In 3D MRI size and morphology of the CMI might be better identified and measured. MRI would then give the surgeon more precise and secure information about the actual size and morphology of the implant. Based on the findings of the present study a more reliable MRI grading system for patients after collagen meniscus substitution is needed. However, as there is so far no better alternative to the Genovese grading system it should be improved in particular in terms of the direct criteria. 5. Conclusions Three of four Genovese grading items showed only slight to moderate inter- and intra-observer reliabilities for the evaluation of patients after partial meniscus substitution using collagen meniscus implants. Hence, such grading results need to be considered with all due care. Only the criteria “bone marrow edema” and “meniscal extrusion” showed a good agreement for both inter and intra-observer reliabilities. Conflict of interest The authors declare that they have no conflicts of interest. References [1] Stone KR, Rodkey WG, Webber R, McKinney L, Steadman JR. Meniscal regeneration with copolymeric collagen scaffolds. In vitro and in vivo studies evaluated clinically, histologically, and biochemically. Am J Sports Med 1992;20:101–11. [2] Steadman JR, Rodkey WG. Tissue-engineered collagen meniscus implants: 5- to 6-year feasibility study results. Arthroscopy 2005;21:515–25. [3] Zaffagnini S, Giordano G, Vascellari A, Bruni D, Neri MP, Iacono F, et al. Arthroscopic collagen meniscus implant results at 6 to 8 years follow up. Knee Surg Sports Traumatol Arthrosc 2006;15:175–83. [4] Rodkey WG, DeHaven KE, Montgomery WH, Baker CLJ, Beck CLJ, Hormel SE, et al. Comparison of the collagen meniscus implant with partial meniscectomy. A prospective randomized trial. J Bone Joint Surg Am 2008;90:1414–26. [5] Bulgheroni P, Murena L, Ratti C, Bulgheroni E, Ronga M, Cherubino P. Follow-up of collagen meniscus implant patients: clinical, radiological, and magnetic resonance imaging results at 5 years. Knee 2009;17:224–9. [6] Verdonk R, Verdonk P, Heinrichs EL. Polyurethane meniscus implant: technique. In: Beaufils P, Verdonk R, editors. The Meniscus. Berlin Heidelberg: Springer; 2010. [7] Lefevre N, Naouri JF, Herman S, Gerometta A, Klouche S, Bohu Y. A current review of the meniscus imaging: proposition of a useful tool for its radiologic analysis. Radiol Res Pract 2016;2016:25. [8] Hirschmann MT, Keller L, Hirschmann A, Schenk L, Berbig R, Lüthi U, et al. One-year clinical and MR imaging outcome after partial meniscal replacement in stabilized knees using a collagen meniscus implant. Knee Surg Sports Traumatol Arthrosc 2013;21:740–7. [9] Genovese E, Angeretti MG, Ronga M, Leonardi A, Novario R, Callegari L, et al. Follow-up of collagen meniscus implants by MRI. Radiol Med 2007;112:1036–48. [10] Monllau JC, Gelber PE, Abat F, Pelfort X, Abad R, Hinarejos P, et al. Outcome after partial medial meniscus substitution with the collagen meniscal implant at a minimum of 10 years' follow-up. Arthroscopy 2011;27:933–43. [11] Ronga M, Bulgheroni P, Manelli A, Genovese E, Grassi F, Cherubino P. Short-term evaluation of collagen meniscus implants by MRI and morphological analysis. J Orthop Traumatol 2002;4:5–10. [12] van Arkel ER, Goei R, De Ploeg I, De Boer HH. Meniscal allografts: evaluation with magnetic resonance imaging and correlation with arthroscopy. Arthroscopy 2000;16:517–21. [13] Verdonk PC, Verstraete KL, Almqvist KF, De Cuyper K, Veys EM, Verbruggen G, et al. Meniscal allograft transplantation: long-term clinical results with radiological and magnetic resonance imaging correlations. Knee Surg Sports Traumatol Arthrosc 2006;14:694–706. [14] Verstraete KL, Verdonk R, Lootens T, Verstraete P, De Rooy J, Kunnen M. Current status and imaging of allograft meniscal transplantation. Eur J Radiol 1997;26: 16–22.
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011
A. Hirschmann et al. / The Knee xxx (2016) xxx–xxx
7
[15] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. [16] Yulish BS, Montanez J, Goodfellow DB, Bryan PJ, Mulopulos GP, Modic MT. Chondromalacia patellae: assessment with MR imaging. Radiology 1987;164:763–6. [17] Zaffagnini S, Marcheggiani MGM, Bulgheroni P, Bulgheroni E, Grassi A, Bonanzinga T, et al. Arthroscopic collagen meniscus implantation for partial lateral meniscal defects: a 2-year minimum follow-up study. Am J Sports Med 2012;40:2281–8.
Please cite this article as: Hirschmann A, et al, The Genovese grading scale is not reliable for MR assessment of collagen meniscus implants, Knee (2016), http://dx.doi.org/10.1016/j.knee.2016.10.011