Interrater Reliability of the International Standards for Neurological Classification of Spinal Cord Injury in Youths With Chronic Spinal Cord Injury

1264

ORIGINAL ARTICLE

Interrater Reliability of the International Standards for Neurological Classification of Spinal Cord Injury in Youths With Chronic Spinal Cord Injury Mary Jane Mulcahey, PhD, OTR\L, John P. Gaughan, PhD, Ross S. Chafetz, DPT, Larry C. Vogel, MD, Amer F. Samdani, MD, Randal R. Betz, MD ABSTRACT. Mulcahey MJ, Gaughan JP, Chafetz RS, Vogel LC, Samdani AF, Betz RR. Interrater reliability of the International Standards for Neurological Classification of Spinal Cord Injury in youths with chronic spinal cord injury. Arch Phys Med Rehabil 2011;92:1264-9. Objectives: To evaluate the interrater reliability of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) in children with chronic spinal cord injury (SCI), and to define the lower age limit at which the examinations have clinical utility. Design: Repeated measures, multicenter reliability study. Setting: Two U.S. pediatric specialty hospitals with recognized SCI programs. Participants: Children (N⫽236) with chronic SCI. Interventions: Subjects underwent 4 examinations by 2 raters: sensory tests (pin prick [PP] and light touch [LT]), a motor test, and a test of anal sensation (AS) and anal contraction (AC). Main Outcome Measures: A 2-way general linear model analysis of variance was used for analysis. Intraclass correlation coefficients (ICCs) and 95% confidence intervals were calculated for PP, LT, motor, AS, and AC. Results: No child younger than 6 years completed the examination. When examined as a function of age, interrater reliability for motor, PP, LT, AS, and AC was moderate (ICC⫽.89) to high (ICC⫽.99). There was poor reliability for AS (ICC⫽.49) in subjects with complete injuries but moderate reliability for all other variables. There was moderate to high reliability for classification of type (tetraplegia/paraplegia) and severity (complete/incomplete) of injury across age groups. Conclusions: The ISNCSCI does not have utility for children younger than 6 years. For children older than 6 years, interrater reliability of PP, LT, and motor examinations is high. Key Words: Paraplegia; Quadriplegia; Rehabilitation; Neurological impairment; Tetraplegia. © 2011 by the American Congress of Rehabilitation Medicine

HE INTERNATIONAL STANDARDS for Neurological T Classification of Spinal Cord Injury (ISNCSCI) provide a method for the neurologic evaluation of persons after spinal 1,2

cord injury (SCI) and for the classification of the neurologic consequence of the injury. The neurologic assessments, which include the motor, sensory, and anorectal examinations, provide the basis for classifying the neurologic level, motor scores and motor level, sensory scores and sensory level, the zone of partial preservation, and the degree of impairment or severity of the SCI according to the American Spinal Injury Association (ASIA) Impairment Scale (AIS).1 The sensory, motor, and anorectal examination techniques and the classification methodology of the ISNCSCI have been well described1,2 and are summarized here. The sensory examination involves testing of 28 dermatomes on the right and left side of the body for sensitivity to pin prick (PP) and light touch (LT). The motor examination is completed through the testing of 10 muscles bilaterally. The strength of each muscle is graded on an ordinal scale from 0 (complete paralysis) to 5 (normal active movement, full range of motion against full resistance). These scores are summed across myotomes and sides of the body to generate a single total motor (TM) score. There has been a recommendation to separate the upper and lower extremity motor (UEM and LEM, respectively) scores because they can better predict functional activities on the motor FIM.3 The anorectal examination involves the evaluation of sensation and contraction of the external anal sphincter. For this, the examiner applies pressure with the index finger to the rectal wall to test for anal sensation (AS) and, to test for anal contraction (AC), makes the request of the person being examined to squeeze as if holding a bowel movement. Classification of the motor, sensory, and neurologic levels are determined based on the motor and sensory examination findings. The sensory level is defined as the most caudal dermatome with bilateral normal sensation to both PP and LT. Likewise, the

List of Abbreviations

From the Shriners Hospitals for Children, Philadelphia, PA (Mulcahey, Betz, Samdani, Chafetz); Shriners Hospitals for Children, Chicago, IL (Vogel); and Biostatistical Consulting Center, Temple University School of Medicine, Philadelphia, PA (Gaughan). Presented in part to the American Academy of Physical Medicine and Rehabilitation, November 6, 2010, Seattle, WA. Supported by Shriners Hospitals for Children (grant no. 8956). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated. Correspondence to Mary Jane Mulcahey, PhD, OTR/L, Shriners Hospitals for Children, 3551 N Broad St, Philadelphia, PA 19140, e-mail: mmulcahey@ shrinenet.org. Reprints are not available from the authors. 0003-9993/11/9208-01057$36.00/0 doi:10.1016/j.apmr.2011.03.003

Arch Phys Med Rehabil Vol 92, August 2011

AC AIS AS ASIA CI ICC ISNCSCI LEM LT PP SCI TM UEM

anal contraction ASIA Impairment Scale anal sensation American Spinal Injury Association confidence interval intraclass correlation coefficient International Standards for Neurological Classification of Spinal Cord Injury lower extremity motor light touch pin prick spinal cord injury total motor upper extremity motor

EVALUATING NEUROLOGIC OUTCOME OF SCI, Mulcahey

single motor level is the most caudal level where there is bilateral strength of at least 3/5, assuming the remaining rostral key muscles are all 5/5. The neurologic level is the lowest spinal segment where sensory and motor functions are normal on both sides. The AIS is based on sacral segments and the key muscle scores. The designation of complete or incomplete is based on sensory and motor findings at S4-5 and\or presence of voluntary contraction and\or deep pressure during anorectal examination.2 If there is sacral sparing as evidenced by preservation of PP and LT sensation in the S4-5 dermatome, preservation of anorectal sensation (deep pressure), or volitional anal contraction, then the SCI is an incomplete injury. For incomplete injuries, the designation of AIS “B” refers to sensory incomplete, whereas designations of AIS “C” and “D” refer to motor incomplete with either gravity eliminated or against gravity motor function, respectively. The ISNCSCI has undergone several revisions of both examination and classification techniques in adults with SCI.4-9 In their current form,10 they are the recommended method for measuring neurologic outcomes in persons with SCI.11 Despite widespread use of the ISNCSCI in clinical practice and clinical trials, with few exceptions,12,13 reliability studies on the motor and sensory examinations have been conducted only in adults, using relatively small samples. Until recently, there has been a serious void in studies that evaluate the use of the anorectal examination as an indicator of injury severity.13,14 Despite the limited empirical work on the ISNCSCI examinations and classification techniques, as in adult practice, they are used routinely in children and adolescents as a way to diagnose their injury, prognosticate recovery, and define potential outcomes of rehabilitation. This multicenter study had 3 main goals. The first was to evaluate the interrater reliability of the ISNCSCI examinations as functions of age and type of injury (tetraplegia/paraplegia) in children with chronic SCI. The second goal was to evaluate the reliability of the ISNCSCI classification in children with chronic SCI. The third goal was to use the data generated from the study to establish guidelines for use of the ISNCSCI with children. Results of intrarater reliability, which showed good to strong reliability of total motor and sensory scores on repeated examinations by the same rater, are published elsewhere.12,13,15,16 To our knowledge, this study is the first to report the results of interrater (among raters) reliability for the ISNCSCI examinations and classification in children and adolescents. METHODS The design was a repeated measures, multicenter reliability study. The study protocol was approved by the institutional review boards of record at each participating center. Written informed consent was obtained from the legal guardian of each subject younger than 18 years. Children between 7 and 18 years of age also provided written informed assent. Subjects 18 years and older provided their own consent. The institutional review board-approved Health Insurance Privacy and Portability Act forms were also reviewed with subjects for their consent. Sample The sample was one of convenience consisting of youths between 3 months and 21 years of age. To minimize variation in repeated test scores resulting from actual neurologic changes, by study design, we enrolled youths with chronic SCI who were not changing neurologically. Children were recruited

1265

from 2 U.S. pediatric orthopedic SCI specialty hospitals. Children were excluded if they had any neurologic changes within the last 3 months, if they were receiving mechanical ventilation without an effective method to communicate responses during test sessions, or if neurologic comorbidities existed that would influence the neurologic examination (eg, brachial plexus injury, traumatic brain injury). The scores from the first of 4 examinations were used to classify subjects’ level and severity of injury unless the first examination differed from the 3 subsequent examinations and all 3 subsequent examinations had the same scores; under this condition, subjects were classified according to the 3 subsequent examinations. Although this report and the articles by Vogel,13 Samdani,14 and Chafetz15 and colleagues report on distinct types of reliability outcomes and used different stratifications and statistical methods for analysis, the subjects were drawn from the same sample. Data Collection Seven raters who were formally trained in the evaluation16 and classification17 methods of the ISNCSCI performed all evaluations. Formal training included a series of lectures on testing techniques, viewing of the testing technique video published by the ISNCSCI training packet, and hands-on practice with immediate feedback from the instructor. The training session was provided by an expert physical therapist external to the institutions participating in the study and who has conducted formal competency programs on the ISNCSCI examination and classification techniques for international clinical trials. Three raters carried out the examinations at 1 institution, and the 4 other raters conducted them at the second institution. All the subjects participated in 4 repeated ISNCSCI examinations conducted by 2 different raters who performed 2 examinations each, on 4 separate days. The time period between the 2 examinations ranged from 24 hours to 4 days (average time, 2.4 days). The examinations were conducted using the standardized techniques published by the ASIA.1,2 As an effort to improve the standardization of the anorectal examinations, for this study, AS was tested by having the examiner gently apply pressure to the rectal wall a minimum of 3 times. If, without prompting, subjects accurately identified the examiner applying pressure, they were scored as having rectal sensation. If their response was inconsistent, a minimum of 8 of 10 accurate responses was required for designating the injury as incomplete. After sensation testing, subjects were asked to squeeze “as if to hold in a bowel movement.” Standardization of examination16 and scoring17 among the raters was implemented before data collection. Scores were documented immediately throughout the examinations using the ISNCSCI form that was modified, with permission, for this study (fig 1). Data underwent double entry into a secure database by research assistants blinded to the study. Classification of neurologic level, motor level, sensory level, injury severity, and AIS were determined using the examination scores and standard techniques published by ASIA.1 Classification was confirmed using a computer program.18 Data Analysis Data were deidentified for analyses by the biostatistician who was blinded to the subjects’ injury characteristics. As a result of failing the Wilk-Shapiro test of normality, data were transformed to normalized ranks to accommodate the nonnormality of ordinal scale measurements and allow use of parametric methods.19,20 The use of rank transformation before analysis of variance is well established and has been described elsewhere as a way to bridge between parametric and nonparaArch Phys Med Rehabil Vol 92, August 2011

1266

EVALUATING NEUROLOGIC OUTCOME OF SCI, Mulcahey

Fig 1. ISNCSCI form. Modified with permission from the American Spinal Injury Association: International Standards for Neurological Classification of Spinal Cord Injury, revised 2006; Chicago, IL. Abbreviations: Add, adduction; Df, dorsiflexion; Ext, extension; Flx, flexion; Pf, plantarflexion.

metric statistics.19,21,22 A 2-way (subjects, raters) general linear model analysis of variance was used for analysis. Raters were nested within subjects. For interrater reliability, the intraclass correlation coefficient (ICC) and 95% confidence intervals (CIs) were calculated according to the model ICC(2,k) as described by Shrout and Fleiss.23 The application of ICC to rank data has been previously reported.24 Data were analyzed as a function of age and type of injury (tetrapelgia/paraplegia) for total PP, LT, TM, UEM, and LEM scores and anorectal responses. ICC values greater than .90 reflect high agreement, and values below .75 reflect poor agreement; values between .75 and .90 reflect moderate agreement. Data were analyzed using SAS V9.1.3 software.a RESULTS A total of 236 children were consented; 28 subjects withdrew from the study before the completion of 4 examinations. While their participation contributed to one of the major purposes of this study (defining the lower age limit for use of the ISNCSCI), examinations from another 27 subjects, all of whom Arch Phys Med Rehabil Vol 92, August 2011

were 8 years or younger, were not used in the analyses because the subjects were unable to comprehend and follow the test instructions because of their young age. As summarized in table 1, the final sample consisted of 181 subjects between 6 and 20 years of age, with comparable inclusion of subjects with tetraplegia and paraplegia, and slightly more subjects having complete injuries. No child younger than 6 years was able to complete the ISNCSCI in its entirety. Only 3 (17%) of 18 subjects in the 0to 5-year age group (all 5 years of age) completed the PP (n⫽2) and motor examinations (n⫽3). Nine (21%) of 42 subjects in the 6- to 11-year age group (all between 6 and 8 years of age) were unable to complete the entire examination. These data suggest that most children 6 years or older can comprehend the directions for, and participate in the ISNCSCI examinations, with a small number of children up to 8 years of age having difficulty. Table 2 provides ICC and 95% CI values for LT, PP, TM, UEM, LEM, AS, and AC as a function of age. Interrater reliability for each variable in all 3 age groups was high (ICC

1267

EVALUATING NEUROLOGIC OUTCOME OF SCI, Mulcahey Table 1: Summary of Study Sample Sample Characteristics

Table 3: ICCs, Lower and Upper 95% CIs for LT and PP at S4-5 Dermatome

Values S4-5 Dermatome

Sex Boy Girl Race White Hispanic Black Asian Other Age at examination (y) Age at injury (y) Time since injury (y) Type of injury Tetraplegia Paraplegia Severity of injury Complete Incomplete Cause of injury MVC Med/Surg Recreation/sports Violence Diving Fall

Age (y)

109 (60.22) 72 (39.78)

6–11

138 (76.24) 21 (11.60) 13 (7.18) 3 (1.66) 6 (3.31) 14.5⫾4.2 10⫾6.1 5⫾4.4

12–15

16–21

89 (49.17) 92 (50.83)

81 (44.75) 36 (19.89) 22 (12.15) 17 (9.39) 14 (7.73) 11 (6.07)

0.93– 0.99) except for AC in the 12- to 15-year age group, which showed moderate reliability (ICC⫽.88). Because they contribute to the classification of injury severity, interrater reliability for S4-5 PP and LT was also analyzed as a function

Table 2: ICCs for ISNCSCI Variables as a Function of Age

6–11

12–15

16–21

n

ICC

LCI

UCI

LTR LTL PPR PPL LTR LTL PPR PPL LTR LTL PPR PPL

42

.91 .87 .85 .77 .84 .90 .85 .80 .83 .78 .85 .83

.84 .77 .75 .61 .73 .82 .73 .66 .75 .69 .77 .75

.95 .92 .92 .87 .91 .94 .91 .88 .88 .85 .90 .88

47

89

NOTE. Shaded values denote LCI values with poor interrater reliability. Abbreviations: LCI, lower 95% CI; LTL, left LT; LTR, right LT; PPL, left PP; PPR, right PP; UCI, upper 95% CI.

97 (53.60) 84 (46.40)

NOTE. Values are n (%) or mean ⫾ SD. Abbreviations: Med/Surg, medical/surgical; MVC, motor vehicle collision.

Age (y)

ISNCSCI Variables

ISNCSCI Variables

ICC

95% CI

n

LT PP TM UEM LEM AS AC LT PP UEM LEM TM AS AC LT PP UEM LEM TM AS AC

.98 .97 .99 .99 .98 .99 .98 .99 .99 .99 .98 .99 .94 .88 .99 .98 .99 .96 .99 .95 .93

.98–.99 .95–.98 .98–.99 .98–.99 .97–.99 .97–.99 .93–.99 .98–.99 .98–.99 .99–.99 .97–.99 .99–.99 .90–.97 .79–.93 .98–.99 .98–.99 .99–.99 .95–.98 .99–.99 .93–.97 .90–.95

43

49

89

NOTE. ICCs and 95% CIs reflecting moderate (AC, 12- to 15-y age group) and strong interrater reliability for LT, PP, UEM, LEM, TM, AS, and AC as a function of age.

of age. As shown in table 3, ICC values were all higher than .75, indicating moderate interrater reliability for S4-5 PP and LT across each age group. Reliability for each variable at each age group was moderate to high when examined as a function of type of injury (tetraplegia/paraplegia), with ICC values ranging from .89-.99. Interrater reliability for classification of severity (complete/incomplete; ICCⱖ.92) and type (tetraplegia/paraplegia; ICCⱖ.92) of injury was high. DISCUSSION To our knowledge, this is the first study designed to evaluate interrater reliability of the ISNCSCI sensory, motor, and anorectal examinations and classification of SCI in children and adolescents. The study also represents one of the few efforts, adult studies included, to evaluate the interrater agreement of the anorectal examinations. For children able to complete the examination, the results support the use of the ISNCSCI sensory (LT and PP), motor, and anorectal examinations with children as young as 6 years, regardless of type of injury (tetraplegia/paraplegia) or severity of injury (complete/incomplete), with moderate to high reliability among trained raters. These results are closely aligned with the results of our previous studies on intrarater reliability where agreement for total motor and sensory scores was moderate to strong.12,16 However, unlike our pilot work,12 which defined 4 years of age as the lower limit for use of the ISNCSCI with children, the present study provides strong evidence that the examinations are too complex for the cognitive abilities of most children younger than 6 years. Only 3 subjects in the 0- to 5-year age group were able to understand the directions for motor and PP testing, and none understood the directions for LT, AS, and AC testing. Thus, no child younger than 6 years demonstrated the ability to engage in the examination to the extent that the necessary scores for classification of the injury were generated. Among the 27 children who were enrolled in this study and unable to complete any of the examinations, 17 (63%) were younger than 6 years, and 10 (37%) were between 6 and 8 years of age. Our results do support previous reports that some children as old as 8 years may have difficulty with the examinations. Based on these findings, we suggest that children 6 years and older should undergo formal testing of LT, PP, and motor Arch Phys Med Rehabil Vol 92, August 2011

1268

EVALUATING NEUROLOGIC OUTCOME OF SCI, Mulcahey

function based on the ISNCSCI, with the understanding that some children as old as 8 years may not be able to comprehend the test instructions. We suggest that for those children too young for the ISNCSCI, neurologic, sensory, and motor levels should be estimated by way of a clinical examination, and the results should be documented as estimates as opposed to documentation of definitive levels based on the ISNCSCI. Work is underway to develop and validate alternative methods to evaluate the neurologic consequence of SCI in the youngest of children, who developmentally cannot engage in the ISNCSCI.25-27 Clearly, the results of this study provide evidence that once children with chronic SCI reach 6 years of age, and assuming they are able to understand and follow test directions, the reliability of the ISNCSCI motor, LT, and PP examinations is strong. Based on this finding, we encourage centers working with children with SCI to apply the ISNCSCI motor and sensory examinations as a way to standardize evaluation for purposes of comparison of outcomes and to foster common language across centers. Because S4-5 PP and LT contribute to the classification of injury severity, we analyzed reliability for this single dermatome and found that agreement for PP and LT was moderate for all age groups. This and our past work demonstrate inconsistency in the reliability of anorectal testing. Previously, we have examined the reliability of repeated motor and sensory testing at individual spinal segments28; further analysis of agreement of sensory scores at individual dermatomes, including S4-5, is planned. Also, our work described by Vogel13 evaluated intrarater reliability of the anorectal examination as a function of type (tetraplegia/paraplegia) of injury and further stratified by 3 age groups, and found poor reliability, likely because of the low numbers in each group as a result of the breakdown of type of injury for 3 different age groups. Based on the limitations described by Vogel13 by study design, we did not stratify type of injury into 3 age groups because the numbers were insufficient for adequate analyses. van Middendorp et al29 evaluated the usefulness of the sacral sparing criteria for predicting walking after SCI and found that only in the chronic injury phase did AC, PP, and LT have predictive value; the results of this study, showing strong reliability of the scores in a population with chronic SCI, provide further relevance for the use of the ISNCSCI in clinical trials enrolling persons with chronic SCI. The ISNCSCI involves both examination and classification. Consistent with the findings for examination, when evaluated as a function of age, agreement on classification of type (tetraplegia/paraplegia) and severity (complete/incomplete) was high for all age groups; this finding provides further support for the use of the ISNCSCI in pediatrics. The current study examined children with chronic SCI, and consideration should be given to the applicability of these findings to children with acute injuries. Ideally, to evaluate reliability of a measure, the study design involves inclusion of people who are not intended to change and a testing interval that allows for sufficient time between tests to minimize recall, but a short enough period to ensure no “real” change occurs. Intentionally, we excluded youths with newly acquired injuries because of the nature of neurologic changes that often occur during this phase. The reliability results from the sample with chronic injuries provide some evidence in support of the ISNCSCI in youths with acute injuries. In fact, because of the results of this study, clinicians can apply the ISNCSCI with youths with acute injuries and have confidence that repeated scores provide a reliable indicator of neurologic status. Arch Phys Med Rehabil Vol 92, August 2011

Study Limitations There are limitations to this study. For example, even though we standardized the testing techniques, youths participating in the study had varying degrees of experience with the examination. Some children, despite being injured for years, were never formally exposed to the ISNCSCI, while other children had the examination on an annual basis. Thus, youths differed in their knowledge about the examination, and we do not know the effect of this difference on the examination results. Secondly, the intervals for the repeated tests were relatively short, risking subject and tester recall. However, enrollment into the study occurred during routine rehabilitation periods, which typically ranged between 1 and 2 weeks. Because many of the families traveled a distance, we designed the testing period to fit the period during which families would be available, rather than asking them to travel long distances on multiple occasions for repeated testing. Another limitation of the study involves the testers who conducted the examinations and their strict adherence to the standardization of testing methods. Formal training was conducted for the testers that included both lecture and hands-on practice, and they complied with the rigor of standardization of testing techniques. We feel that one of the strengths of the study is the trained testers and their adherence to the testing procedures. Routine use of the standards in clinical practice may not reflect the same rigor in training and testing technique as seen in this study, which may contribute to different findings in reliability of the results. Intentionally, this study does not report on reliability of repeated scores at individual spinal segments, nor do we report reliability as a function of time since injury. These considerations are being addressed in future work, in addition to work underway on the reliability and validity of the anorectal examinations. Finally, unlike the adult population with SCI, where there are a relatively high number of adults with acute injuries in known centers (eg, the U.S. Model System Centers), children with acute SCI are typically cared for in regional children’s hospitals; children are not triaged to designated acute care centers that specialize in SCI. Hence, a real challenge to conducting a study with children with acute injuries is the relatively low number of children with SCI available at one or a few centers. Despite this challenge, additional studies to further strengthen the psychometric properties of the ISNCSCI in youths with acute injuries should focus not only on reliability in the acute phase but also on concurrent validity in which the ISNCSCI is compared against another measure that evaluates muscle and sensory function. CONCLUSIONS This study demonstrated moderate to high interrater reliability of repeated ISNCSCI motor and sensory scores in children with chronic SCI and provides evidence in support of the use of the ISNCSCI with children. Based on the study findings, children with SCI who are 6 years and older should undergo formal neurologic testing based on the ISNCSCI. For younger children, the neurologic level should be estimated and documented as such; physicians should refrain from designating an injury in a young child as complete or incomplete unless they communicate clearly to families and document that they are making this determination based on findings other than the ISNCSCI. More psychometric research is recommended on the anorectal examination. References 1. American Spinal Injury Association. Reference Manual for international standards for neurological classification of spinal cord injury. Chicago: American Spinal Injury Association; 2003.

EVALUATING NEUROLOGIC OUTCOME OF SCI, Mulcahey

2. Marino RJ, Barros T, Biering-Sorensen F, et al. International standards for neurological classification of spinal cord injury. J Spinal Cord Med 2003;26(Suppl 1):S50-6. 3. Marino RJ, Graves DE. Metric properties of the ASIA motor score: subscales improve correlation with functional activities. Arch Phys Med Rehabil 2004;85:1804-10. 4. Donovan WH, Wilkerson MA, Rossi D, Mechoulam F, Frankowsh RF. A test of the ASIA guidelines for classification of spinal cord injury. J Neurol Rehabil 1990;4:39-53. 5. Cohen ME, Sheehan TP, Herbison GJ. Content validity and reliability of the international standards for neurological classification of spinal cord injury. Top Spinal Cord Inj Rehabil 1996;1:15-31. 6. Cohen ME, Ditunno JF, Donovan WH, Maynard FM. A test of the 1992 international standards for neurological and functional classification of spinal cord injury. Spinal Cord 1998;36:554-60. 7. Jonsson M, Tollback A, Gonzales H, Borg J. Inter-rater reliability of the 1992 international standards for neurological and functional classification of incomplete spinal cord injury. Spinal Cord 2000; 38:675-9. 8. Marino RJ, Jones L, Kirshblum S, Tal J. Reliability of the ASIA motor and sensory examination [abstract]. J Spinal Cord Med 2004;27:194. 9. Savic G, Bergstrom EMK, Frankel HL, Jamous MA, Jones PW. Inter-rater reliability of motor and sensory examinations performed according to American Spinal Injury Association standards. Spinal Cord 2007;45:444-51. 10. Waring WP III, Biering-Sorensen F, Burns S, et al. 2009 review and revisions of the international standards for the neurological classification of spinal cord injury. J Spinal Cord Med 2010;33; 346-52. 11. Steeves JD, Lammertse D, Curt A, et al. Guidelines for the conduct of clinical trials for spinal cord injury (SCI) as developed by the ICCP panel: clinical trials outcome measures. Spinal Cord 2007;45:206-21. 12. Mulcahey MJ, Gaughan J, Betz RR, Johansen KJ. The international standards for neurological classification of spinal cord injury: reliability of data when applied to children and youths. Spinal Cord 2007;45:452-9. 13. Vogel L, Samdani A, Chafetz R, Gaughan J, Betz R, Mulcahey MJ. Intra-rater agreement of the anorectal exam and classification of injury severity in children with spinal cord injury. Spinal Cord 2009;47:687-91. 14. Samdani A, Chafetz R, Vogel L, Betz R, Gaughan J, Mulcahey MJ. The international standards for neurological classification: relationship between S4-5 dermatome testing and anorectal testing. Spinal Cord 2010;49:352-56. 15. Chafetz R, Gaughan J, Vogel L, Betz R, Mulcahey MJ. The international standards for neurological classification of spinal cord injury: intra-rater agreement of total motor and sensory

16.

17.

18.

19. 20. 21.

22.

23. 24. 25.

26.

27.

28.

29.

1269

scores in the pediatric population. J Spinal Cord Med 2009;32:157-61. Mulcahey MJ, Gaughan J, Betz R, Vogel L. Rater agreement on the ISCSCI motor and sensory scores obtained before and after formal training in testing technique. J Spinal Cord Med 2007;30: S146-9. Chafetz R, Vogel L, Betz R, Mulcahey MJ. International standards for neurological classification of spinal cord injury: training effect on accurate classification. J Spinal Cord Med 2008;31:538-42. Chafetz R, Prak S, Mulcahey MJ. Computerized classification of neurological injury based on the international standards for classification of spinal cord injury. J Spinal Cord Med 2009;32:59194. Conover WJ, Iman RL. Rank transformations as a bridge between parametric and nonparametric statistics. Am Stat 1981;35:124-9. Harter HL. Expected values of normal order statistics. Biometrika 1961;48:151-65. Conover WJ, Iman RL. The rank transformation as a method of discrimination with some examples. Commun Stat 1980;A9: 465-87. Iman RL. A power study of a rank transform for the two-way classification model when interactions may be present. Can J Stat 1974;2:227-39. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. Muller R, Buttner P. A critical discussion of intraclass correlation coefficient. Stat Med 1994;13:2465-76. Calhoun C, Chafetz R, Mulcahey MJ. A pilot study of observational motor assessment in infants and toddlers with spinal cord injury. Pediatr Phys Ther 2009;21:62-70. Samdani A, Fayssoux R, Ashgar J, et al. Chronic spinal cord injury in the pediatric population: does magnetic resonance imaging correlate with the international standards for neurological classification of spinal cord injury examination? Spine 2009;34: 74-81. Mohamed F, Hunter L, Barakat N, et al. Diffusion tensor imaging of the pediatric spinal cord at 1.5 T: preliminary results. AJNR Am J Neuroradiol 2011;32:339-45. Mulcahey MJ, Gaughan J, Betz RR. Agreement of ISCSCI scores at individual myotomes and dermatomes in youth with complete injuries. Spinal Cord 2009;47:56-61. van Middendorp JJ, Hosman AJF, Pouw MH, Van de Meent H, EM-SCI Study/Group. Is determination between complete and incomplete traumatic spinal cord injury clinically relevant? Validation of the ASIA sacral sparing criteria in a prospective cohort of 432 patients. Spinal Cord 2009;47:555-60.

Supplier a. SAS Institute Inc, 100 SAS Campus Dr, Cary, NC 27513.

Arch Phys Med Rehabil Vol 92, August 2011