- Email: [email protected]
Assessment of Inter- and Intraobserver Reliability and Accuracy to Evaluate Apical Vertebral Rotation Using Four Methods: An Experimental Study Using a Saw Bone Model
Spine Deformity 7 (2019) 11e17 www.spine-deformity.org
Basic Science
Assessment of Inter- and Intraobserver Reliability and Accuracy to Evaluate Apical Vertebral Rotation Using Four Methods: An Experimental Study Using a Saw Bone Model Satyajit V. Marawar, MDa, Nathaniel R. Ordway, MS/PEb, Darryl A. Auston, MD, PhDc, Swamy Kurra, MBBSb, Dongliang Wang, PhDd, Venita M. Simpson, MDe, Richard A. Tallarico, MDb, Danielle A. Katz, MDb, Kathryn Palomino, MDb, Mark Palumbo, MDf, William F. Lavelle, MDb,* a
Department of Orthopedics, Syracuse Veterans Affairs Medical Center, 800 Irving Ave., Syracuse, NY 13210, USA Department of Orthopedic Surgery, SUNY Upstate Medical University, 750 E. Adams St., Syracuse, NY 13210, USA c Orthopedic Trauma, Hughston Clinic, Orange Park Medical Center, 2001 Kingsley Ave, Orange Park, FL 32073, USA d Department of Public Health and Preventive Medicine, SUNY Upstate Medical University, 766 Irving Ave, Syracuse, NY 13210, USA e Department of Neurosurgery, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, USA f Warren Alpert School of Medicine at Brown University, 222 Richmond St, Providence, RI 02903, USA Received 21 November 2017; revised 5 June 2018; accepted 9 June 2018 b
Abstract Study Design: After placing a thoracic three-vertebra segment saw bones model on a standardized turntable, a series of anteroposterior (AP) radiographs were obtained and then set in increments to 90 rotation. Then the specimen was instrumented with 35-mm pedicle screws bilaterally and the rotation process and image acquisition were repeated. Objective: Assess reliability and accuracy of spine surgeons evaluating apical vertebral rotation (AVR) through surgeon’s visual x-ray estimation, Nash-Moe system, Upasani trigonometric method, and Upasani grading system. Background Context: Accurate assessment of AVR is one measure surgeons can evaluate the success of intervention and potential loss of correction in scoliotic deformities. Methods: Eighty-four representative images of uninstrumented and instrumented vertebral segments were blinded. AVR was estimated by five experienced spinal deformity surgeons using the four techniques. The surgeons’ grading, estimates, and errors compared to actual rotation were calculated. Inter- and intraobserver reliability were calculated using interclass correlation (ICC). Results: Each surgeon’s error for simple visual estimation for uninstrumented segments was 8.7 to 17.4 (average error 5 12.4 ), and for instrumented segments it was 7.7 to 11.3 (average error 5 9.5 ). Error for the Upasani trigonometric method was 6.7 to 11.6 (average error 5 0.9 ). There was relatively poor accuracy for Nash-Moe system (38.2%e53.9%) compared with the Upasani grading system (76.74%e80.23%). Interobserver reliability using the Nash-Moe method was good (0.844), with intraobserver reliability from fair to excellent (0.684e0.949). Interobserver reliability for the Upasani grading method was good (0.829), with intraobserver reliability from fair to good (0.751e0.869). We found excellent interobserver reliability for Upasani trigonometric classification (0.935) with fair to excellent intraobserver reliability (0.775e0.991). The interobserver reliability of surgeons’ visual estimates was good (0.898) and the intraobserver reliability from good to excellent (0.866e0.99) without pedicle screws, and interobserver reliability was excellent (0.948) and intraobserver reliability also excellent (0.959e0.986) with pedicle screws. Conclusions: We confirm that both techniques described by Upasani have good reliability and accuracy, appearing more accurate than surgeon’s visual estimates or Nash-Moe system. Level of Evidence: Level III. Ó 2018 Scoliosis Research Society. All rights reserved. Keywords: Apical vertebral rotation; Observer reliability; Visual x-ray estimation; Nash-Moe system; Upasani methods
Author disclosures: SVM (none), NROrdway (grants from Premier Orthopedic Solutions, outside the submitted work), DAA (none), SK (none), DW (none), VMS (none), RAT (personal fees from Stryker, grants from
Vertiflex, outside the submitted work), DAK (other from AAOS, American College of Surgeons, Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Johnson & Johnson, Merck, Novartis, Procter & Gamble, Roche, and
2212-134X/$ - see front matter Ó 2018 Scoliosis Research Society. All rights reserved. https://doi.org/10.1016/j.jspd.2018.06.009
12
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
Sanofi-Aventis, outside the submitted work), KP (none), MP (personal fees from Stryker, Globus, Orthofix, Spineart, and Various Law Firms, outside the submitted work), WFL (other from SAS, Prosydian, Innovasis, 4Web, and Cardan Robotics; grants from DePuy Spine, Signus, Spinal Kinetics, K2M, Vertebral Technologies, Synthes Spine, Medtronic, IntegraLife, Providence Technologies, Stryker, Vertiflex, and
Amedica, outside the submitted work). This study required no IRB approval. *Corresponding author. Department of Orthopedic Surgery, SUNY Upstate Medical University, 750 E. Adams St., Syracuse, NY 13210, USA. Tel.: (315) 464-8602; fax: (315) 464-5223. E-mail address: [email protected] (W.F. Lavelle).
Introduction
vertebrae with a Nash-Moe Grade 0 had up to 11 of rotation when measured using the CT method [7]. More importantly, the utility of the Nash-Moe system is further limited in postoperative patients as pedicle screws obscure the pedicles. To address this limitation, Upasani et al. developed a radiographic technique to measure vertebral rotation in postoperative scoliotic patients treated with bilateral segmental pedicle screws. This method uses a trigonometric equation to estimate vertebral rotation based on screw tip displacement on standard anteroposterior radiographs that correlates closely to CT measures of rotation. Based on this technique, they also developed a simple clinically applicable grading system to radiographically approximate apical vertebral rotation (AVR) [9]. Though well described by the authors, the Upasani method has not been independently validated, nor has it been compared to other classification systems, particularly the NashMoe system. The purpose of this study was to assess the inter- and intraobserver reliability as well as the accuracy of spinal deformity surgeons in their evaluation of AVR through (1) simple radiographic estimation, (2) the Nash-Moe system, (3) the Upasani grading system, and (4) the Upasani trigonometric method to estimate vertebral rotation by measuring screw tip displacement.
The Cobb angle and axial vertebral rotation are two major factors that can be used to measure the severity of scoliosis, estimate the risk of progression, and evaluate the treatment outcome [1]. Currently, the Cobb angle is the gold standard measurement recommended by the Scoliosis Research Society. However, the importance of vertebral rotation as a component of the three-dimensional deformity of the scoliotic spine cannot be stressed enough. Axial rotation of the apical vertebrae (AVR) has been found to correlate with the lateral curve, the location and length of the curve, and the cosmetic deformity of the rib hump and lumbar prominence [2]. An accurate measurement of vertebral rotation is of great importance for surgical planning, analysis of surgical results, and understanding the mechanics of the deformity [3]. Multiple plain radiographic techniques and scoring systems have been published in the literature to assess apical vertebral rotation. In 1948, Cobb described a technique for the measurement of rotation based on the position of the tip of the spinous process in relation to the underlying vertebral body [4]. However, because of the asymmetric development of the spinous process in the scoliotic spine, Nash and Moe found Cobb’s method to be unreliable and developed the Nash-Moe grading system based on the displacement of the convex side pedicle toward the midline. In comparison to the spinous process, the pedicles are located closer to the vertebral body and consequently are not subject to as much distortion in severe scoliotic curves [5]. Perdriolle further refined this method by using a torsion meter to measure the offset of the pedicle from the vertebral body edge to provide an estimate of rotation [6]. CT-based assessments have also been described as a method for the assessment of apical vertebral rotations [2,7]. These CT measurements of rotation correlate well with actual vertebral rotation in cadaveric studies [2] and have been found to be superior to plain radiographic techniques in detecting smaller degrees of vertebral rotation [7,8]. Despite their accuracy, these methods require repeated radiation exposure to the patient, which is a concern especially in the pediatric population. The Nash-Moe system is the most commonly used radiographic system for assessing vertebral rotation. A limitation of the Nash-Moe system is the overestimation of rotation at higher grades, whereas significant rotation may exist at ‘‘Grade 0’’ vertebrae [5]. Ho et al. found that
Materials and Methods An individual thoracic three-vertebra segment (T8e10) of a saw bones model (Pacific Research Laboratories, Vashon Island, WA) was placed on a graduated turntable (Parker Hannifin Corporation, Rohnert Park, CA). The vertebral segment was radiographed at 50 kVp, 5 mAs, and 40-inch SID (source-image distance). A standard anteroposterior image was collected and then the vertebral segment was rotated and reimaged from 0 to 20 of rotation in 2 increments. Images were then collected in 5 increments up to 50 and 10 increments up to 90 . The amount of actual (true) vertebral rotation was recorded on an answer key. The specimen was then instrumented using a free-hand technique by one of the fellowship-trained spine deformity surgeons with 35-mm pedicle monoaxial screws (DePuy Synthes Spine, Raynham, MA) bilaterally, and the above rotation process was repeated and recorded on the answer key. A total of 84 images were obtained: 42 images for the uninstrumented specimen and 42 images for the instrumented specimen.
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
13
Fig. 2. Upasani categorical grading: Upasani grading system that uses bilateral pedicle screw tip displacement with respect to the rods and grades vertebral rotation.
Fig. 1. Nash-Moe: The displacement of the pedicle is used to categorically rate the rotation of the vertebra.
The two sets of 42 representative images obtained were blinded, randomized, and evaluated by five experienced fellowship-trained spine deformity surgeons (three adult spine deformity surgeons and two pediatric spine deformity surgeons). They were asked to estimate vertebral rotation using the four different methods described above. Three repeat images were included to assess intraobserver reliability assessments (n 5 87 images). First, the surgeons were asked to measure the degree of the vertebral rotation for instrumented and uninstrumented specimens by simple visual estimation from the radiographs. Then, they used the Nash-Moe system to
record vertebral rotation for the uninstrumented specimen. The Nash-Moe method describes the percentage displacement of the convex pedicle with respect to the vertebral body width, which is used to approximate the angle of vertebral rotation (Fig. 1). It is graded from 0 to 4 with respect to convex pedicle displacement across the vertebral body width [4,5]. For the instrumented specimen, the surgeons graded the vertebral rotation using the Upasani grading system that uses bilateral pedicle screw tip displacement with respect to the rods and grades vertebral rotation from 0 to 2, which is correlated with the angle of vertebral rotation [9] (Fig. 2). For the fourth method, the surgeons measured the length of the pedicle screw and the displacement of the pedicle screw tip from its axis of rotation in PA radiographs (Fig. 3A) and calculated the rotation using the following Upasani trigonometric equation: Vertebral rotation (s) 5 (q1þq2)/2 [4] (Fig. 3B). The accuracy of the surgeon estimate and Upasani trigonometric method in assessing vertebral rotation was
Fig. 3. (A) Upasani trigonometric method: length of the screw (L1 and L2), screw tip displacement (D1 and D2). (B) Upasani trigonometric rotation equation.
14
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
measured by its difference with the recorded actual rotation; the average error (standard deviation) and median, minimum, and maximum errors were calculated. The accuracy of the Nash-Moe system and Upasani grading system were measured by the frequency (proportion) of correct estimates, which were calculated for each possible level and overall. The interclass correlation (ICC) (and the associated 95% CI) was used to assess both inter- and intrarater reliability for each method using an SAS macro [10] that allows accurate confidence interval calculation accounting for more than one replicates by surgeons and images. We noted the equivalence of the weighted Kappa and the ICC [11] and used the ICC to assess the reliability of the Upasani and NashMoe methods as well. All analyses were done in SAS 9.3. Results Calculations for the error for each of the surgeons for simple visual estimation of vertebral rotation in vertebrae without pedicle screw implantation ranged from 8.7 to 17.4 with an average error of 12.4 ; and in vertebrae with pedicle screw implantation ranged from 7.7 to 11.2 with an average error of 9.5 . Calculations of the error for the Upasani trigonometric classification system ranged from 6.7 to 11.6 with an average error of 0.9 . Surgeon accuracy for the Nash-Moe system was low. The surgeons frequently selected a Nash-Moe score that was one grade away from the accurate score based on recorded actual rotation answer key. Overall, surgeon accuracy ranged from 38.2% to 53.9%. Surgeon accuracy was lower for Grades 2 and 3. Accuracy for the Upasani categorical classification system was much better and ranged from 76.74% to 80.23%. For qualitative discussion purposes, the inter- and intraobserver correlation of 0.0e0.25 reflects absent to poor; 0.25e0.49, low; 0.50e0.69, fair/moderate; 0.70e0.89, good; and 0.90e1.0, excellent correlations [12]. The inter- and intraobserver reliabilities of the four methods are summarized in Table 1. There was a good interobserver reliability for surgeon visual estimates for vertebral rotation in vertebrae before pedicle screw placement (0.898), with intraobserver reliability ranging from fair to excellent (0.866e0.99). Visual apical vertebral rotation estimates after pedicle screw implantation had excellent interobserver reliability (0.948) along with excellent intraobserver reliability (0.959e0.986). There was good interobserver reliability utilizing the Nash-Moe system (0.844), with intraobserver reliability ranging from fair to excellent (0.684e0.949). Good interobserver reliability for the Upasani grading method (0.829), with intraobserver reliability ranging from fair to good
Table 1 Reliability of four methods to determine AVR. Surgeon
ICC
Nash-Moe classification 1 0.838 2 0.949 3 0.923 4 0.921 5 0.684 Interrater reliability 0.844 Upasani screw tip classification 1 0.801 2 0.815 3 0.869 4 0.859 5 0.751 Interrater reliability 0.829 Upasani trigonometric classification 1 0.991 2 0.976 3 0.974 4 0.991 5 0.775 Interrater reliability 0.935 Surgeon visual estimate without pedicle screws 1 0.958 2 0.989 3 0.99 4 0.989 5 0.866 Interrater reliability 0.898 Surgeons visual estimate with pedicle screws 1 0.975 2 0.986 3 0.986 4 0.979 5 0.959 Interrater reliability 0.948
95% CI 0.659e0.928 0.892e0.976 0.840e0.964 0.836e0.963 0.447e0.832 0.720e0.915 0.623e0.900 0.648e0.907 0.744e0.935 0.726e0.930 0.553e0.869 0.718e0.899 0.985e0.995 0.951e0.988 0.947e0.987 0.981e0.996 0.502e0.908 0.882e0.964 0.933e0.974 0.982e0.993 0.984e0.994 0.982e0.993 0.791e0.915 0.816e0.945 0.960e0.985 0.977e0.991 0.977e0.991 0.966e0.987 0.934e0.975 0.906e0.972
AVR, apical vertebral rotation; CI, confidence interval; ICC, intraclass correlation.
(0.751e0.869), and excellent interobserver reliability for the Upasani trigonometric classification (0.935), with fair to excellent intraobserver reliability (0.775e0.991). We performed a subanalysis examining the calculated estimates for the Upasani trigonometric method and the surgeons’ simple visual estimates (vertebrae with pedicle screws) as these provided numeric values for our subanalysis. We based our groupings on the true values (answer key) for spinal rotation to examine accuracy and reliability differences. These are presented in Tables 2 and 3. We found the surgeons’ simple visual estimate accuracy for vertebrae with pedicle screws was poorer in the 20 e50 range. The calculated Upasani trigonometric accuracy similarly fell as the rotation became higher, but did not worsen as profoundly as the surgeon’s visual estimated accuracy.
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
15
Table 2 Subanalysis by rotation degree: surgeon estimate and Upasani trigonometric. Upasani trigonometric accuracy
Surgeon estimate with pedicle screws accuracy Rater 0 e20 1 2 3 4 5 Overall 20 e50 1 2 3 4 5 Overall 50 e90 1 2 3 4 5 Overall
n
Mean (SD)
Median
Min, max
n
Mean (SD)
Median
Min, max
42 41 42 41 41 207
5.0 6.8 2.2 6.5 4.9 5.0
(6.90) (7.41) (3.25) (7.87) (5.10) (6.48)
3 6 1 4 4 4
4.0, 3.0, 3.0, 4.0, 4.0, 4.0,
23.00 36.00 10.00 24.00 20.00 36.00
41 28 28 28 27 152
2.3 1.2 0.9 2.3 11.2 2.2
(2.98) (3.78) (5.16) (3.35) (6.90) (6.36)
3.1 1.8 1.1 1.5 11.3 1.4
7.9, 13.2, 20.0, 3.3, 10.0, 20.0,
4.09 5.90 10.17 9.27 21.22 21.22
26 26 26 26 26 130
14.8 19.4 19.0 16.3 21.1 18.1
(7.28) (13.88) (10.43) (6.41) (11.41) (10.35)
15 20 20 15 22.5 19
0.0, 10.0, 10.0, 3.0, 0.0, 10.0,
30.00 45.00 35.00 30.00 40.00 45.00
26 18 18 18 3 83
7.5 7.1 4.5 6.4 14.8 2.1
(2.68) (4.50) (4.15) (4.27) (8.49) (7.85)
6.6 7.2 3.9 6.4 18.4 3.4
13.5, 2.7, 2.5, 0.0, 5.1, 13.5,
4.28 15.39 12.81 14.04 20.86 20.86
19 19 19 19 19 95
4.2 9.4 7.3 7.6 9.1 7.5
(5.84) (8.19) (7.61) (6.14) (7.22) (7.14)
5 10 7 5 10 7
10.0, 5.0, 5.0, 0.0, 2.0, 10.0,
15.00 25.00 20.00 20.00 20.00 25.00
18
15.3 (8.52)
15.1
39.9, 6.54
SD, standard deviation.
Table 3 Surgeon inter- and intrarater reliability. Surgeon estimate with pedicle screws Rater 0 e20 1 2 3 4 5 Interrater 20 e50 1 2 3 4 5 Interrater 50 e90 1 2 3 4 5 Interrater
Upasani trigonometric
ICC
95% confidence interval
ICC
95% confidence interval
0.924 0.889 0.932 0.835 0.675 0.711
0.850e0.962 0.784e0.944 0.866e0.966 0.691e0.915 0.429e0.828 0.459e0.857
0.902 0.775 0.626 0.907 0.696 0.433
0.808e0.951 0.501e0.908 0.247e0.839 0.773e0.964 0.346e0.876 0.179e0.633
0.609 0.889 0.849 0.823 0.716 0.573
0.257e0.818 0.750e0.953 0.669e0.935 0.618e0.923 0.426e0.872 0.178e0.808
0.973 0.957 0.959 0.974
0.936e0.989 0.869e0.986 0.875e0.987 0.920e0.992
0.643
0.250e0.854
0.85 0.935 0.736 0.874 0.663 0.85
0.630e0.944 0.828e0.976 0.403e0.897 0.683e0.953 0.276e0.865 0.630e0.944
0.805
0.506e0.931
ICC, intraclass correlation.
Discussion In patients with scoliosis, radiographic measures serve to determine when surgical intervention is indicated, help determine fusion levels, and assess correction following
surgery [13]. Accurate assessment of vertebral rotation has proven utility in predicting curve progression, surgical correction, and decompensation [14]. Behensky et al. retrospectively reviewed surgically treated double major curves (Lenke type 3C) in adolescent idiopathic scoliosis and concluded that lumbar apical vertebral rotation of less than 40% provided the radiographic prediction of postoperative coronal spinal imbalance [15]. Inaccurate knowledge of vertebral rotation may lead to unnecessary surgical operations and, in the case of pedicle screws, misplacements that may incur risks of spinal cord injury [16]. Routine radiographs remain the most common modality to estimate vertebral rotation, and the Nash-Moe system is the most commonly used modality to assess vertebral body rotation on radiographs. There are, however, difficulties with using the Nash-Moe system in postoperative patients with bilateral segmental pedicle screws as these obscure the pedicle projections. To address this limitation, Upasani et al. developed a radiographic technique using a trigonometric equation to estimate vertebral rotation postoperatively based on screw tip displacement on standard standing posteroanterior (PA) radiographs. In their study of 17 patients who had surgery for adolescent idiopathic scoliosis, the average absolute difference between vertebral rotation calculated using the Upasani technique and CTbased measurement was only 1.9 2.0 . Based on this technique, they developed a simple radiographic grading system based on the position of pedicle screw tips in relation to the rods [9].
16
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
Knowing the reliability and reproducibility of the radiographic measures allows one to determine if the effect of the intervention is greater than the variability of the measurement [14]. Although the Upasani technique appears to provide a more direct estimation of the vertebral body rotation as compared to the Nash-Moe system, it has not been independently validated. There is limited clinical data available using the Upasani method to evaluate vertebral rotation. Upasani et al. validated their technique by comparing vertebral rotation calculated on anteroposterior radiographs and compared this with postoperative CT scans [9]. This, however, does raise the issue of validity of comparing measurements taken on standing radiographs with measurements taken on CT scans which are done in the supine position. One would have to assume that because these are instrumented vertebrae, the rotation would not change significantly from a supine to a standing position. Additionally, there are no studies assessing the reliability of both these techniques or comparing their accuracy. The purpose of this study was to assess the inter- and intraobserver reliability as well as accuracy of spinal deformity surgeons in their evaluation of AVR through (1) simple radiograph estimation, (2) use of the Nash-Moe system, (3) the Upasani grading system, and (4) the Upasani trigonometric method to estimate vertebral rotation by measuring screw tip displacement. We found that the average error of the surgeons for simple visual estimation of vertebral rotation in vertebrae without pedicle screw implantation was 12.4 , and in vertebrae with pedicle screw implantation was 9.5 . The trigonometric calculation of AVR using the Upasani technique in the same images resulted in an average error of 0.9 . Overall, the error is much lower using the Upasani trigonometric method than the simple visual estimate. It was noted that the accuracy for the Upasani trigonometric method decreased as the degree of rotation approached greater than 45 . The Nash-Moe system accuracy was most accurate for Grade 4 and least accurate for Grades 2 and 3. This would potentially be attributable to a floor and ceiling effect for the classification system. Accuracy for the Upasani grading system was much better than Nash-Moe and ranged from 76.74% to 80.23%. We evaluated the inter- and intraobserver reliability of surgeon visual estimate (before and after pedicle screw placement), Nash-Moe classification, the Upasani grading system, and Upasani trigonometric calculation of vertebral rotation. For the Upasani trigonometric method, we observed excellent interobserver reliability with good to excellent intraobserver reliability. Coupled with these results showing high accuracy while determining vertebral rotation, we believe that this study independently validates the Upasani trigonometric method. For the Nash-Moe system, we found good interobserver reliability and fair to excellent intraobserver reliability. Our findings for the Nash-Moe system is in contrast to previous studies where only fair to good intraobserver reliability and fair
interobserver reliability using manual measurements and moderate inter- and intraobserver reliability with digital measurements [13]. For the Upasani grading system, we found good interobserver as well as intraobserver reliability. Coupled with better accuracy of this grading system as compared to the Nash-Moe system, we believe that this should be the preferred modality to grade vertebral rotation in instrumented patients postoperatively. There are several limitations to our study. The study was done using an individual segment of a saw bones model, which certainly provides ease of use and consistent repeatability, but this may not accurately represent challenges faced in doing these measurements on imaging in scoliosis patients, especially in cases with severe scoliosis. Changes in the shape of the apical vertebra in such cases may affect the accuracy and repeatability of any of these techniques. Another limitation is both the Upasani trigonometric and grading system can only be used postoperatively. Even though the results of this study validate this technique and the grading system and show improved accuracy as compared to surgeons’ visual estimate and the Nash-Moe system, these measurements cannot replace the current utility of the Nash-Moe system in assessing apical vertebral rotation preoperatively. Conclusion The most accurate technique for assessing vertebral rotation is limited by cost, feasibility, and postural changes. The most frequently used technique, the Nash-Moe system, has inherent variability and is frequently difficult to assess postoperatively following posterior instrumentation, particularly with thoracic pedicle screws, where the construct has the greatest potential for direct rotational correction [14]. We confirm that both techniques described by Upasani et al. have good reliability and accuracy, which appear better than a surgeon’s simple visual estimates or the Nash-Moe system. The study results cannot be directly extrapolated to clinical situations in AIS patients. Although clinical data are needed to validate the conclusions in this study, we do believe this study provides a sound basis for a clinical study in the future. References [1] Vo QN, Lou EH, Le LH. Measurement of axial vertebral rotation using three-dimensional ultrasound images. Scoliosis 2015;10(Suppl 2):S7. [2] Aaro S, Dahlborn M. The longitudinal axis rotation of the apical vertebra, the vertebral, spinal, and rib cage deformity in idiopathic scoliosis studied by computer tomography. Spine (Phila Pa 1976) 1981;6:567e72. [3] Birchall D, Hughes DG, Hindle J, et al. Measurement of vertebral rotation in adolescent idiopathic scoliosis using three-dimensional magnetic resonance imaging. Spine (Phila Pa 1976) 1997;22: 2403e7. [4] Cobb J. Outline for the study of scoliosis. Am Acad Orthop Surg Instr Course Lect 1948;5:261e75.
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17 [5] Nash CL, Moe JH. A study of vertebral rotation. J Bone Joint Surg Am 1969;51:223e9. [6] Perdriolle R, Vidal J. Thoracic idiopathic scoliosis curve evolution and prognosis. Spine (Phila Pa 1976) 1985;10:785e91. [7] Ho EK, Upadhyay SS, Ferris L, et al. A comparative study of computed tomographic and plain radiographic methods to measure vertebral rotation in adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 1992;17:771e4. [8] Ho EK, Upadhyay SS, Chan FL, et al. New methods of measuring vertebral rotation from computed tomographic scans. An intraobserver and interobserver study on girls with scoliosis. Spine (Phila Pa 1976) 1993;18:1173e7. [9] Upasani VV, Chambers RC, Dalal AH, et al. Grading apical vertebral rotation without a computed tomography scan: a clinically relevant system based on the radiographic appearance of bilateral pedicle screws. Spine (Phila Pa 1976) 2009;34:1855e62. [10] Carrasco JL, Phllips BR, Puig-Martinez J, et al. Estimation of the concordance correlation coefficient for repeated measures using SAS and R. Comput Methods Programs Biomed 2013;109:293e304.
17
[11] Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973;33:613e9. [12] Fleiss JL. Reliability of measurement. In: Design and Analysis of Clinical Experiments. New York, NY: John Wiley & Sons, Inc.; 2011. p. 1e32. [13] Kuklo TR, Potter BK, Schroeder TM, O’Brien MF. Comparison of manual and digital measurements in adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 2006;31:1240e6. [14] Kuklo TR, Potter BK, Lenke LG. Vertebral rotation and thoracic torsion in adolescent idiopathic scoliosis: what is the best radiographic correlate? J Spinal Disord Tech 2005;18:139e47. [15] Behensky HH, Cole AA, Freeman BJ, et al. Fixed lumbar apical vertebral rotation predicts spinal decompensation in Lenke type 3C adolescent idiopathic scoliosis after selective posterior thoracic correction and fusion. Eur Spine J 2007;16:1570e8. [16] Sugimoto Y, Tanaka M, Nakanishi K, et al. Predicting intraoperative vertebral rotation in patients with scoliosis using posterior elements as anatomical landmarks. Spine (Phila Pa 1976) 2007;32:E761e3.
Basic Science
Assessment of Inter- and Intraobserver Reliability and Accuracy to Evaluate Apical Vertebral Rotation Using Four Methods: An Experimental Study Using a Saw Bone Model Satyajit V. Marawar, MDa, Nathaniel R. Ordway, MS/PEb, Darryl A. Auston, MD, PhDc, Swamy Kurra, MBBSb, Dongliang Wang, PhDd, Venita M. Simpson, MDe, Richard A. Tallarico, MDb, Danielle A. Katz, MDb, Kathryn Palomino, MDb, Mark Palumbo, MDf, William F. Lavelle, MDb,* a
Department of Orthopedics, Syracuse Veterans Affairs Medical Center, 800 Irving Ave., Syracuse, NY 13210, USA Department of Orthopedic Surgery, SUNY Upstate Medical University, 750 E. Adams St., Syracuse, NY 13210, USA c Orthopedic Trauma, Hughston Clinic, Orange Park Medical Center, 2001 Kingsley Ave, Orange Park, FL 32073, USA d Department of Public Health and Preventive Medicine, SUNY Upstate Medical University, 766 Irving Ave, Syracuse, NY 13210, USA e Department of Neurosurgery, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, USA f Warren Alpert School of Medicine at Brown University, 222 Richmond St, Providence, RI 02903, USA Received 21 November 2017; revised 5 June 2018; accepted 9 June 2018 b
Abstract Study Design: After placing a thoracic three-vertebra segment saw bones model on a standardized turntable, a series of anteroposterior (AP) radiographs were obtained and then set in increments to 90 rotation. Then the specimen was instrumented with 35-mm pedicle screws bilaterally and the rotation process and image acquisition were repeated. Objective: Assess reliability and accuracy of spine surgeons evaluating apical vertebral rotation (AVR) through surgeon’s visual x-ray estimation, Nash-Moe system, Upasani trigonometric method, and Upasani grading system. Background Context: Accurate assessment of AVR is one measure surgeons can evaluate the success of intervention and potential loss of correction in scoliotic deformities. Methods: Eighty-four representative images of uninstrumented and instrumented vertebral segments were blinded. AVR was estimated by five experienced spinal deformity surgeons using the four techniques. The surgeons’ grading, estimates, and errors compared to actual rotation were calculated. Inter- and intraobserver reliability were calculated using interclass correlation (ICC). Results: Each surgeon’s error for simple visual estimation for uninstrumented segments was 8.7 to 17.4 (average error 5 12.4 ), and for instrumented segments it was 7.7 to 11.3 (average error 5 9.5 ). Error for the Upasani trigonometric method was 6.7 to 11.6 (average error 5 0.9 ). There was relatively poor accuracy for Nash-Moe system (38.2%e53.9%) compared with the Upasani grading system (76.74%e80.23%). Interobserver reliability using the Nash-Moe method was good (0.844), with intraobserver reliability from fair to excellent (0.684e0.949). Interobserver reliability for the Upasani grading method was good (0.829), with intraobserver reliability from fair to good (0.751e0.869). We found excellent interobserver reliability for Upasani trigonometric classification (0.935) with fair to excellent intraobserver reliability (0.775e0.991). The interobserver reliability of surgeons’ visual estimates was good (0.898) and the intraobserver reliability from good to excellent (0.866e0.99) without pedicle screws, and interobserver reliability was excellent (0.948) and intraobserver reliability also excellent (0.959e0.986) with pedicle screws. Conclusions: We confirm that both techniques described by Upasani have good reliability and accuracy, appearing more accurate than surgeon’s visual estimates or Nash-Moe system. Level of Evidence: Level III. Ó 2018 Scoliosis Research Society. All rights reserved. Keywords: Apical vertebral rotation; Observer reliability; Visual x-ray estimation; Nash-Moe system; Upasani methods
Author disclosures: SVM (none), NROrdway (grants from Premier Orthopedic Solutions, outside the submitted work), DAA (none), SK (none), DW (none), VMS (none), RAT (personal fees from Stryker, grants from
Vertiflex, outside the submitted work), DAK (other from AAOS, American College of Surgeons, Bristol-Myers Squibb, Eli Lilly, GlaxoSmithKline, Johnson & Johnson, Merck, Novartis, Procter & Gamble, Roche, and
2212-134X/$ - see front matter Ó 2018 Scoliosis Research Society. All rights reserved. https://doi.org/10.1016/j.jspd.2018.06.009
12
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
Sanofi-Aventis, outside the submitted work), KP (none), MP (personal fees from Stryker, Globus, Orthofix, Spineart, and Various Law Firms, outside the submitted work), WFL (other from SAS, Prosydian, Innovasis, 4Web, and Cardan Robotics; grants from DePuy Spine, Signus, Spinal Kinetics, K2M, Vertebral Technologies, Synthes Spine, Medtronic, IntegraLife, Providence Technologies, Stryker, Vertiflex, and
Amedica, outside the submitted work). This study required no IRB approval. *Corresponding author. Department of Orthopedic Surgery, SUNY Upstate Medical University, 750 E. Adams St., Syracuse, NY 13210, USA. Tel.: (315) 464-8602; fax: (315) 464-5223. E-mail address: [email protected] (W.F. Lavelle).
Introduction
vertebrae with a Nash-Moe Grade 0 had up to 11 of rotation when measured using the CT method [7]. More importantly, the utility of the Nash-Moe system is further limited in postoperative patients as pedicle screws obscure the pedicles. To address this limitation, Upasani et al. developed a radiographic technique to measure vertebral rotation in postoperative scoliotic patients treated with bilateral segmental pedicle screws. This method uses a trigonometric equation to estimate vertebral rotation based on screw tip displacement on standard anteroposterior radiographs that correlates closely to CT measures of rotation. Based on this technique, they also developed a simple clinically applicable grading system to radiographically approximate apical vertebral rotation (AVR) [9]. Though well described by the authors, the Upasani method has not been independently validated, nor has it been compared to other classification systems, particularly the NashMoe system. The purpose of this study was to assess the inter- and intraobserver reliability as well as the accuracy of spinal deformity surgeons in their evaluation of AVR through (1) simple radiographic estimation, (2) the Nash-Moe system, (3) the Upasani grading system, and (4) the Upasani trigonometric method to estimate vertebral rotation by measuring screw tip displacement.
The Cobb angle and axial vertebral rotation are two major factors that can be used to measure the severity of scoliosis, estimate the risk of progression, and evaluate the treatment outcome [1]. Currently, the Cobb angle is the gold standard measurement recommended by the Scoliosis Research Society. However, the importance of vertebral rotation as a component of the three-dimensional deformity of the scoliotic spine cannot be stressed enough. Axial rotation of the apical vertebrae (AVR) has been found to correlate with the lateral curve, the location and length of the curve, and the cosmetic deformity of the rib hump and lumbar prominence [2]. An accurate measurement of vertebral rotation is of great importance for surgical planning, analysis of surgical results, and understanding the mechanics of the deformity [3]. Multiple plain radiographic techniques and scoring systems have been published in the literature to assess apical vertebral rotation. In 1948, Cobb described a technique for the measurement of rotation based on the position of the tip of the spinous process in relation to the underlying vertebral body [4]. However, because of the asymmetric development of the spinous process in the scoliotic spine, Nash and Moe found Cobb’s method to be unreliable and developed the Nash-Moe grading system based on the displacement of the convex side pedicle toward the midline. In comparison to the spinous process, the pedicles are located closer to the vertebral body and consequently are not subject to as much distortion in severe scoliotic curves [5]. Perdriolle further refined this method by using a torsion meter to measure the offset of the pedicle from the vertebral body edge to provide an estimate of rotation [6]. CT-based assessments have also been described as a method for the assessment of apical vertebral rotations [2,7]. These CT measurements of rotation correlate well with actual vertebral rotation in cadaveric studies [2] and have been found to be superior to plain radiographic techniques in detecting smaller degrees of vertebral rotation [7,8]. Despite their accuracy, these methods require repeated radiation exposure to the patient, which is a concern especially in the pediatric population. The Nash-Moe system is the most commonly used radiographic system for assessing vertebral rotation. A limitation of the Nash-Moe system is the overestimation of rotation at higher grades, whereas significant rotation may exist at ‘‘Grade 0’’ vertebrae [5]. Ho et al. found that
Materials and Methods An individual thoracic three-vertebra segment (T8e10) of a saw bones model (Pacific Research Laboratories, Vashon Island, WA) was placed on a graduated turntable (Parker Hannifin Corporation, Rohnert Park, CA). The vertebral segment was radiographed at 50 kVp, 5 mAs, and 40-inch SID (source-image distance). A standard anteroposterior image was collected and then the vertebral segment was rotated and reimaged from 0 to 20 of rotation in 2 increments. Images were then collected in 5 increments up to 50 and 10 increments up to 90 . The amount of actual (true) vertebral rotation was recorded on an answer key. The specimen was then instrumented using a free-hand technique by one of the fellowship-trained spine deformity surgeons with 35-mm pedicle monoaxial screws (DePuy Synthes Spine, Raynham, MA) bilaterally, and the above rotation process was repeated and recorded on the answer key. A total of 84 images were obtained: 42 images for the uninstrumented specimen and 42 images for the instrumented specimen.
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
13
Fig. 2. Upasani categorical grading: Upasani grading system that uses bilateral pedicle screw tip displacement with respect to the rods and grades vertebral rotation.
Fig. 1. Nash-Moe: The displacement of the pedicle is used to categorically rate the rotation of the vertebra.
The two sets of 42 representative images obtained were blinded, randomized, and evaluated by five experienced fellowship-trained spine deformity surgeons (three adult spine deformity surgeons and two pediatric spine deformity surgeons). They were asked to estimate vertebral rotation using the four different methods described above. Three repeat images were included to assess intraobserver reliability assessments (n 5 87 images). First, the surgeons were asked to measure the degree of the vertebral rotation for instrumented and uninstrumented specimens by simple visual estimation from the radiographs. Then, they used the Nash-Moe system to
record vertebral rotation for the uninstrumented specimen. The Nash-Moe method describes the percentage displacement of the convex pedicle with respect to the vertebral body width, which is used to approximate the angle of vertebral rotation (Fig. 1). It is graded from 0 to 4 with respect to convex pedicle displacement across the vertebral body width [4,5]. For the instrumented specimen, the surgeons graded the vertebral rotation using the Upasani grading system that uses bilateral pedicle screw tip displacement with respect to the rods and grades vertebral rotation from 0 to 2, which is correlated with the angle of vertebral rotation [9] (Fig. 2). For the fourth method, the surgeons measured the length of the pedicle screw and the displacement of the pedicle screw tip from its axis of rotation in PA radiographs (Fig. 3A) and calculated the rotation using the following Upasani trigonometric equation: Vertebral rotation (s) 5 (q1þq2)/2 [4] (Fig. 3B). The accuracy of the surgeon estimate and Upasani trigonometric method in assessing vertebral rotation was
Fig. 3. (A) Upasani trigonometric method: length of the screw (L1 and L2), screw tip displacement (D1 and D2). (B) Upasani trigonometric rotation equation.
14
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
measured by its difference with the recorded actual rotation; the average error (standard deviation) and median, minimum, and maximum errors were calculated. The accuracy of the Nash-Moe system and Upasani grading system were measured by the frequency (proportion) of correct estimates, which were calculated for each possible level and overall. The interclass correlation (ICC) (and the associated 95% CI) was used to assess both inter- and intrarater reliability for each method using an SAS macro [10] that allows accurate confidence interval calculation accounting for more than one replicates by surgeons and images. We noted the equivalence of the weighted Kappa and the ICC [11] and used the ICC to assess the reliability of the Upasani and NashMoe methods as well. All analyses were done in SAS 9.3. Results Calculations for the error for each of the surgeons for simple visual estimation of vertebral rotation in vertebrae without pedicle screw implantation ranged from 8.7 to 17.4 with an average error of 12.4 ; and in vertebrae with pedicle screw implantation ranged from 7.7 to 11.2 with an average error of 9.5 . Calculations of the error for the Upasani trigonometric classification system ranged from 6.7 to 11.6 with an average error of 0.9 . Surgeon accuracy for the Nash-Moe system was low. The surgeons frequently selected a Nash-Moe score that was one grade away from the accurate score based on recorded actual rotation answer key. Overall, surgeon accuracy ranged from 38.2% to 53.9%. Surgeon accuracy was lower for Grades 2 and 3. Accuracy for the Upasani categorical classification system was much better and ranged from 76.74% to 80.23%. For qualitative discussion purposes, the inter- and intraobserver correlation of 0.0e0.25 reflects absent to poor; 0.25e0.49, low; 0.50e0.69, fair/moderate; 0.70e0.89, good; and 0.90e1.0, excellent correlations [12]. The inter- and intraobserver reliabilities of the four methods are summarized in Table 1. There was a good interobserver reliability for surgeon visual estimates for vertebral rotation in vertebrae before pedicle screw placement (0.898), with intraobserver reliability ranging from fair to excellent (0.866e0.99). Visual apical vertebral rotation estimates after pedicle screw implantation had excellent interobserver reliability (0.948) along with excellent intraobserver reliability (0.959e0.986). There was good interobserver reliability utilizing the Nash-Moe system (0.844), with intraobserver reliability ranging from fair to excellent (0.684e0.949). Good interobserver reliability for the Upasani grading method (0.829), with intraobserver reliability ranging from fair to good
Table 1 Reliability of four methods to determine AVR. Surgeon
ICC
Nash-Moe classification 1 0.838 2 0.949 3 0.923 4 0.921 5 0.684 Interrater reliability 0.844 Upasani screw tip classification 1 0.801 2 0.815 3 0.869 4 0.859 5 0.751 Interrater reliability 0.829 Upasani trigonometric classification 1 0.991 2 0.976 3 0.974 4 0.991 5 0.775 Interrater reliability 0.935 Surgeon visual estimate without pedicle screws 1 0.958 2 0.989 3 0.99 4 0.989 5 0.866 Interrater reliability 0.898 Surgeons visual estimate with pedicle screws 1 0.975 2 0.986 3 0.986 4 0.979 5 0.959 Interrater reliability 0.948
95% CI 0.659e0.928 0.892e0.976 0.840e0.964 0.836e0.963 0.447e0.832 0.720e0.915 0.623e0.900 0.648e0.907 0.744e0.935 0.726e0.930 0.553e0.869 0.718e0.899 0.985e0.995 0.951e0.988 0.947e0.987 0.981e0.996 0.502e0.908 0.882e0.964 0.933e0.974 0.982e0.993 0.984e0.994 0.982e0.993 0.791e0.915 0.816e0.945 0.960e0.985 0.977e0.991 0.977e0.991 0.966e0.987 0.934e0.975 0.906e0.972
AVR, apical vertebral rotation; CI, confidence interval; ICC, intraclass correlation.
(0.751e0.869), and excellent interobserver reliability for the Upasani trigonometric classification (0.935), with fair to excellent intraobserver reliability (0.775e0.991). We performed a subanalysis examining the calculated estimates for the Upasani trigonometric method and the surgeons’ simple visual estimates (vertebrae with pedicle screws) as these provided numeric values for our subanalysis. We based our groupings on the true values (answer key) for spinal rotation to examine accuracy and reliability differences. These are presented in Tables 2 and 3. We found the surgeons’ simple visual estimate accuracy for vertebrae with pedicle screws was poorer in the 20 e50 range. The calculated Upasani trigonometric accuracy similarly fell as the rotation became higher, but did not worsen as profoundly as the surgeon’s visual estimated accuracy.
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
15
Table 2 Subanalysis by rotation degree: surgeon estimate and Upasani trigonometric. Upasani trigonometric accuracy
Surgeon estimate with pedicle screws accuracy Rater 0 e20 1 2 3 4 5 Overall 20 e50 1 2 3 4 5 Overall 50 e90 1 2 3 4 5 Overall
n
Mean (SD)
Median
Min, max
n
Mean (SD)
Median
Min, max
42 41 42 41 41 207
5.0 6.8 2.2 6.5 4.9 5.0
(6.90) (7.41) (3.25) (7.87) (5.10) (6.48)
3 6 1 4 4 4
4.0, 3.0, 3.0, 4.0, 4.0, 4.0,
23.00 36.00 10.00 24.00 20.00 36.00
41 28 28 28 27 152
2.3 1.2 0.9 2.3 11.2 2.2
(2.98) (3.78) (5.16) (3.35) (6.90) (6.36)
3.1 1.8 1.1 1.5 11.3 1.4
7.9, 13.2, 20.0, 3.3, 10.0, 20.0,
4.09 5.90 10.17 9.27 21.22 21.22
26 26 26 26 26 130
14.8 19.4 19.0 16.3 21.1 18.1
(7.28) (13.88) (10.43) (6.41) (11.41) (10.35)
15 20 20 15 22.5 19
0.0, 10.0, 10.0, 3.0, 0.0, 10.0,
30.00 45.00 35.00 30.00 40.00 45.00
26 18 18 18 3 83
7.5 7.1 4.5 6.4 14.8 2.1
(2.68) (4.50) (4.15) (4.27) (8.49) (7.85)
6.6 7.2 3.9 6.4 18.4 3.4
13.5, 2.7, 2.5, 0.0, 5.1, 13.5,
4.28 15.39 12.81 14.04 20.86 20.86
19 19 19 19 19 95
4.2 9.4 7.3 7.6 9.1 7.5
(5.84) (8.19) (7.61) (6.14) (7.22) (7.14)
5 10 7 5 10 7
10.0, 5.0, 5.0, 0.0, 2.0, 10.0,
15.00 25.00 20.00 20.00 20.00 25.00
18
15.3 (8.52)
15.1
39.9, 6.54
SD, standard deviation.
Table 3 Surgeon inter- and intrarater reliability. Surgeon estimate with pedicle screws Rater 0 e20 1 2 3 4 5 Interrater 20 e50 1 2 3 4 5 Interrater 50 e90 1 2 3 4 5 Interrater
Upasani trigonometric
ICC
95% confidence interval
ICC
95% confidence interval
0.924 0.889 0.932 0.835 0.675 0.711
0.850e0.962 0.784e0.944 0.866e0.966 0.691e0.915 0.429e0.828 0.459e0.857
0.902 0.775 0.626 0.907 0.696 0.433
0.808e0.951 0.501e0.908 0.247e0.839 0.773e0.964 0.346e0.876 0.179e0.633
0.609 0.889 0.849 0.823 0.716 0.573
0.257e0.818 0.750e0.953 0.669e0.935 0.618e0.923 0.426e0.872 0.178e0.808
0.973 0.957 0.959 0.974
0.936e0.989 0.869e0.986 0.875e0.987 0.920e0.992
0.643
0.250e0.854
0.85 0.935 0.736 0.874 0.663 0.85
0.630e0.944 0.828e0.976 0.403e0.897 0.683e0.953 0.276e0.865 0.630e0.944
0.805
0.506e0.931
ICC, intraclass correlation.
Discussion In patients with scoliosis, radiographic measures serve to determine when surgical intervention is indicated, help determine fusion levels, and assess correction following
surgery [13]. Accurate assessment of vertebral rotation has proven utility in predicting curve progression, surgical correction, and decompensation [14]. Behensky et al. retrospectively reviewed surgically treated double major curves (Lenke type 3C) in adolescent idiopathic scoliosis and concluded that lumbar apical vertebral rotation of less than 40% provided the radiographic prediction of postoperative coronal spinal imbalance [15]. Inaccurate knowledge of vertebral rotation may lead to unnecessary surgical operations and, in the case of pedicle screws, misplacements that may incur risks of spinal cord injury [16]. Routine radiographs remain the most common modality to estimate vertebral rotation, and the Nash-Moe system is the most commonly used modality to assess vertebral body rotation on radiographs. There are, however, difficulties with using the Nash-Moe system in postoperative patients with bilateral segmental pedicle screws as these obscure the pedicle projections. To address this limitation, Upasani et al. developed a radiographic technique using a trigonometric equation to estimate vertebral rotation postoperatively based on screw tip displacement on standard standing posteroanterior (PA) radiographs. In their study of 17 patients who had surgery for adolescent idiopathic scoliosis, the average absolute difference between vertebral rotation calculated using the Upasani technique and CTbased measurement was only 1.9 2.0 . Based on this technique, they developed a simple radiographic grading system based on the position of pedicle screw tips in relation to the rods [9].
16
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17
Knowing the reliability and reproducibility of the radiographic measures allows one to determine if the effect of the intervention is greater than the variability of the measurement [14]. Although the Upasani technique appears to provide a more direct estimation of the vertebral body rotation as compared to the Nash-Moe system, it has not been independently validated. There is limited clinical data available using the Upasani method to evaluate vertebral rotation. Upasani et al. validated their technique by comparing vertebral rotation calculated on anteroposterior radiographs and compared this with postoperative CT scans [9]. This, however, does raise the issue of validity of comparing measurements taken on standing radiographs with measurements taken on CT scans which are done in the supine position. One would have to assume that because these are instrumented vertebrae, the rotation would not change significantly from a supine to a standing position. Additionally, there are no studies assessing the reliability of both these techniques or comparing their accuracy. The purpose of this study was to assess the inter- and intraobserver reliability as well as accuracy of spinal deformity surgeons in their evaluation of AVR through (1) simple radiograph estimation, (2) use of the Nash-Moe system, (3) the Upasani grading system, and (4) the Upasani trigonometric method to estimate vertebral rotation by measuring screw tip displacement. We found that the average error of the surgeons for simple visual estimation of vertebral rotation in vertebrae without pedicle screw implantation was 12.4 , and in vertebrae with pedicle screw implantation was 9.5 . The trigonometric calculation of AVR using the Upasani technique in the same images resulted in an average error of 0.9 . Overall, the error is much lower using the Upasani trigonometric method than the simple visual estimate. It was noted that the accuracy for the Upasani trigonometric method decreased as the degree of rotation approached greater than 45 . The Nash-Moe system accuracy was most accurate for Grade 4 and least accurate for Grades 2 and 3. This would potentially be attributable to a floor and ceiling effect for the classification system. Accuracy for the Upasani grading system was much better than Nash-Moe and ranged from 76.74% to 80.23%. We evaluated the inter- and intraobserver reliability of surgeon visual estimate (before and after pedicle screw placement), Nash-Moe classification, the Upasani grading system, and Upasani trigonometric calculation of vertebral rotation. For the Upasani trigonometric method, we observed excellent interobserver reliability with good to excellent intraobserver reliability. Coupled with these results showing high accuracy while determining vertebral rotation, we believe that this study independently validates the Upasani trigonometric method. For the Nash-Moe system, we found good interobserver reliability and fair to excellent intraobserver reliability. Our findings for the Nash-Moe system is in contrast to previous studies where only fair to good intraobserver reliability and fair
interobserver reliability using manual measurements and moderate inter- and intraobserver reliability with digital measurements [13]. For the Upasani grading system, we found good interobserver as well as intraobserver reliability. Coupled with better accuracy of this grading system as compared to the Nash-Moe system, we believe that this should be the preferred modality to grade vertebral rotation in instrumented patients postoperatively. There are several limitations to our study. The study was done using an individual segment of a saw bones model, which certainly provides ease of use and consistent repeatability, but this may not accurately represent challenges faced in doing these measurements on imaging in scoliosis patients, especially in cases with severe scoliosis. Changes in the shape of the apical vertebra in such cases may affect the accuracy and repeatability of any of these techniques. Another limitation is both the Upasani trigonometric and grading system can only be used postoperatively. Even though the results of this study validate this technique and the grading system and show improved accuracy as compared to surgeons’ visual estimate and the Nash-Moe system, these measurements cannot replace the current utility of the Nash-Moe system in assessing apical vertebral rotation preoperatively. Conclusion The most accurate technique for assessing vertebral rotation is limited by cost, feasibility, and postural changes. The most frequently used technique, the Nash-Moe system, has inherent variability and is frequently difficult to assess postoperatively following posterior instrumentation, particularly with thoracic pedicle screws, where the construct has the greatest potential for direct rotational correction [14]. We confirm that both techniques described by Upasani et al. have good reliability and accuracy, which appear better than a surgeon’s simple visual estimates or the Nash-Moe system. The study results cannot be directly extrapolated to clinical situations in AIS patients. Although clinical data are needed to validate the conclusions in this study, we do believe this study provides a sound basis for a clinical study in the future. References [1] Vo QN, Lou EH, Le LH. Measurement of axial vertebral rotation using three-dimensional ultrasound images. Scoliosis 2015;10(Suppl 2):S7. [2] Aaro S, Dahlborn M. The longitudinal axis rotation of the apical vertebra, the vertebral, spinal, and rib cage deformity in idiopathic scoliosis studied by computer tomography. Spine (Phila Pa 1976) 1981;6:567e72. [3] Birchall D, Hughes DG, Hindle J, et al. Measurement of vertebral rotation in adolescent idiopathic scoliosis using three-dimensional magnetic resonance imaging. Spine (Phila Pa 1976) 1997;22: 2403e7. [4] Cobb J. Outline for the study of scoliosis. Am Acad Orthop Surg Instr Course Lect 1948;5:261e75.
S.V. Marawar et al. / Spine Deformity 7 (2019) 11e17 [5] Nash CL, Moe JH. A study of vertebral rotation. J Bone Joint Surg Am 1969;51:223e9. [6] Perdriolle R, Vidal J. Thoracic idiopathic scoliosis curve evolution and prognosis. Spine (Phila Pa 1976) 1985;10:785e91. [7] Ho EK, Upadhyay SS, Ferris L, et al. A comparative study of computed tomographic and plain radiographic methods to measure vertebral rotation in adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 1992;17:771e4. [8] Ho EK, Upadhyay SS, Chan FL, et al. New methods of measuring vertebral rotation from computed tomographic scans. An intraobserver and interobserver study on girls with scoliosis. Spine (Phila Pa 1976) 1993;18:1173e7. [9] Upasani VV, Chambers RC, Dalal AH, et al. Grading apical vertebral rotation without a computed tomography scan: a clinically relevant system based on the radiographic appearance of bilateral pedicle screws. Spine (Phila Pa 1976) 2009;34:1855e62. [10] Carrasco JL, Phllips BR, Puig-Martinez J, et al. Estimation of the concordance correlation coefficient for repeated measures using SAS and R. Comput Methods Programs Biomed 2013;109:293e304.
17
[11] Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973;33:613e9. [12] Fleiss JL. Reliability of measurement. In: Design and Analysis of Clinical Experiments. New York, NY: John Wiley & Sons, Inc.; 2011. p. 1e32. [13] Kuklo TR, Potter BK, Schroeder TM, O’Brien MF. Comparison of manual and digital measurements in adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 2006;31:1240e6. [14] Kuklo TR, Potter BK, Lenke LG. Vertebral rotation and thoracic torsion in adolescent idiopathic scoliosis: what is the best radiographic correlate? J Spinal Disord Tech 2005;18:139e47. [15] Behensky HH, Cole AA, Freeman BJ, et al. Fixed lumbar apical vertebral rotation predicts spinal decompensation in Lenke type 3C adolescent idiopathic scoliosis after selective posterior thoracic correction and fusion. Eur Spine J 2007;16:1570e8. [16] Sugimoto Y, Tanaka M, Nakanishi K, et al. Predicting intraoperative vertebral rotation in patients with scoliosis using posterior elements as anatomical landmarks. Spine (Phila Pa 1976) 2007;32:E761e3.