- Email: [email protected]
A clinical prediction rule for ambulation outcomes after traumatic spinal cord injury: a longitudinal cohort study
Articles
A clinical prediction rule for ambulation outcomes after traumatic spinal cord injury: a longitudinal cohort study Joost J van Middendorp, Allard J F Hosman, A Rogier T Donders, Martin H Pouw, John F Ditunno Jr, Armin Curt, Alexander C H Geurts, Hendrik Van de Meent, for the EM-SCI Study Group
Summary Lancet 2011; 377: 1004–10 Published Online March 4, 2011 DOI:10.1016/S01406736(10)62276-3 See Comment page 972 Spine Unit, Department of Orthopaedics (J J van Middendorp MD, A J F Hosman MD, M H Pouw MD); Department of Epidemiology, Biostatistics and HTA (A R T Donders PhD); and Department of Rehabilitation Medicine (Prof A C H Geurts MD, H Van de Meent MD), Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands; Department of Rehabilitation Medicine, Jefferson Medical College, Thomas Jefferson University, Philadelphia, PA, USA (Prof J F Ditunno Jr MD); and Spinal Cord Injury Center, Balgrist University Hospital, Zurich, Switzerland (Prof A Curt MD) Correspondence to: Dr Joost J van Middendorp, Department of Orthopaedics, PO Box 9101, 6500 HB Nijmegen, Netherlands [email protected]
1004
Background Traumatic spinal cord injury is a serious disorder in which early prediction of ambulation is important to counsel patients and to plan rehabilitation. We developed a reliable, validated prediction rule to assess a patient’s chances of walking independently after such injury. Methods We undertook a longitudinal cohort study of adult patients with traumatic spinal cord injury, with early (within the first 15 days after injury) and late (1-year follow-up) clinical examinations, who were admitted to one of 19 European centres between July, 2001, and June, 2008. A clinical prediction rule based on age and neurological variables was derived from the international standards for neurological classification of spinal cord injury with a multivariate logistic regression model. Primary outcome measure 1 year after injury was independent indoor walking based on the Spinal Cord Independence Measure. Model performances were quantified with respect to discrimination (area under receiver-operating-characteristics curve [AUC]). Temporal validation was done in a second group of patients from July, 2008, to December, 2009. Findings Of 1442 patients with spinal cord injury, 492 had available outcome measures. A combination of age (<65 vs ≥65 years), motor scores of the quadriceps femoris (L3), gastrocsoleus (S1) muscles, and light touch sensation of dermatomes L3 and S1 showed excellent discrimination in distinguishing independent walkers from dependent walkers and non-walkers (AUC 0·956, 95% CI 0·936–0·976, p<0·0001). Temporal validation in 99 patients confirmed excellent discriminating ability of the prediction rule (AUC 0·967, 0·939–0·995, p<0·0001). Interpretation Our prediction rule, including age and four neurological tests, can give an early prognosis of an individual’s ability to walk after traumatic spinal cord injury, which can be used to set rehabilitation goals and might improve the ability to stratify patients in interventional trials. Funding Internationale Stiftung für Forschung in Paraplegie.
Introduction Traumatic spinal cord injury has a profound effect on patients’ physical and psychosocial wellbeing. Although the frequency of such injury is low at 10·4–83 cases per million people worldwide, this devastating disorder imposes a substantial burden on the health-care system.1 Despite advances in basic research into spinal cord repair that show promise, no treatment that results in major neurological or functional recovery is available.2 After a spinal cord injury, a reliable prognosis of a patient’s potential functional outcome is essential for counselling and to design a personalised rehabilitation programme.3 During rehabilitation, recovery of the ability to walk is a high priority for such patients.4 However, no prediction rule for the ability to walk independently after traumatic spinal cord injury is available. In this study we analysed data from a prospective, longitudinal, multicentre cohort study5 of a representative European population with spinal cord injury to develop an accurate and simple clinical prediction rule for a patient’s ability to walk independently. Age at injury6 and variables from the international standards for neurological classification of spinal cord injury7 are used in the clinical algorithm. We tested the
reproducibility and validity of the rule to predict an individual’s ability to walk independently after injury on a second group of patients.
Methods Study design and patient population Since July, 2001, 19 centres (five centres originally) have gathered a standardised dataset of neurological and functional outcomes of patients with spinal cord injury as part of the European Multicenter Study on Human Spinal Cord Injury (EM-SCI).5 Data for neurological and functional status were collected prospectively, per protocol, within the first 15 days and at months 1, 3, 6, and 12 after injury. Because no proven effective treatment is available,2 treatment regimens are not standardised within the EM-SCI network. Details of applied treatments were not recorded systematically, but ranged from nonoperative interventions to very early (<6 h after injury) surgical stabilisation and decompression of the spinal cord. Dependent on level and severity of injury, individually tailored rehabilitation programmes varied in focus and intensity. From the EM-SCI dataset we extracted data for all adult (≥18 years) patients with acute traumatic spinal cord www.thelancet.com Vol 377 March 19, 2011
Articles
injury, including conus medullaris and cauda equina injuries, who were admitted between July, 2001, and June, 2008. Patients who were unable to cooperate with physical examination because of cognitive impairment (as assessed by the examiner), who had a peripheral nerve lesion, or who had neuropathy or polyneuropathy were not included in the EM-SCI database. Polyneuropathy was tested by measurement of ulnar and tibial sensory nerve conduction velocity. Patients with medical records showing frequent causes of polyneuropathy (eg, diabetes mellitus), and those without a complete neurological assessment within the first 15 days after injury were excluded from the analysis. The multicentre follow-up study was done in accordance with the ethics standards in the updated version of the 1964 Declaration of Helsinki. The study protocol was approved by the local ethics committees of all participating centres, and the patients gave informed, oral consent before entering the study.
Prognostic variables For the prognostic model we considered patients’ age and variables from their initial neurological examination. Older patients (≥65 years) with spinal cord injury have less potential to translate neurological improvements into functional recovery than do younger patients.6 Therefore, age was categorised into two groups: patients younger than 65 years and those aged 65 years and older. Neurological examinations were done according to the international standards for classification of spinal cord injuries.7 Examination included motor score testing (graded on a five-point scale adapted from the Medical Research Council scale), light touch sensory (LTS) and pinprick sensory (PPS) testing (0=absent, 1=impaired, and 2=normal), and sacral sparing scores, including voluntary anal contraction and anal sensation (0=absent and 1=present).7 Muscle testing was done in the supine position. Because the PPS scores and the LTS scores are highly correlated,8 we included only one of the two sensory scoring systems in the initial model.9 Because an LTS score of 0 means that light touch sensation is absent and a PPS score of 0 means that there could be local sensation, but the separation of dull and sharp sensation is absent,7 we used only the LTS scoring system in analysis because we thought it to be the least prone to error. To validate this approach the final model was tested with the addition of PPS scores. For every patient, we included only the best scores of each level (ie, right or left) of the lower extremity and sacral scores for analysis.8 Clinical assessments were done by trained and certified neurologists and rehabilitation physicians with at least 1 year of experience in examination of patients with spinal cord injury. Motor and sensory scores were recorded in the electronic EM-SCI database and the quality and correctness of the data were monitored centrally by a data quality manager. Patients’ American Spinal Injury Association/ International Spinal Cord Society neurological standard www.thelancet.com Vol 377 March 19, 2011
Panel 1: American Spinal Injury Association/International Spinal Cord Society neurological standard scale7 Grade A No motor or sensory function is preserved in the sacral segments S4–S5 Grade B Sensory but not motor function is preserved below the neurological level and includes the sacral segments S4–S5 Grade C Motor function is preserved below the neurological level, and more than half of key muscles below the neurological level have a muscle grade of less than 3 Grade D Motor function is preserved below the neurological level, and at least half of key muscles below the neurological level have a muscle grade of 3 or more Grade E Motor and sensory function are normal
Panel 2: Spinal Cord Independence Measure item 12—mobility indoors11,12 0: Requires total assistance 1: Needs electric wheelchair or partial assistance to operate manual wheelchair 2: Moves independently in manual wheelchair 3: Requires supervision while walking (with or without devices) 4: Walks with a walking frame or crutches (swing) 5: Walks with crutches or two canes (reciprocal walking) 6: Walks with one cane 7: Needs leg orthosis only 8: Walks without walking aids We applied a cutoff SCIM (Spinal Cord Independence Measure ) mobility score to differentiate between patients who are unable to walk or are dependent on assistance while walking (scores 0–3) and those who are able to walk independently (scores 4–8).
scale (AIS) grades were computed automatically according to the international standards (panel 1).7 Because the aim of this study was to introduce a simple clinical prediction rule with minimum burden on patients and maximum time efficiency for physicians, we did not include aggregated neurological scores (eg, total lower extremity motor score10) in analyses.
Outcome assessment The ability to walk independently 1 year after injury was the primary functional outcome. The Spinal Cord Independence Measure indoor mobility item (SCIM item 12, ability to walk <10 m) was assessed and analysed for this purpose.11,12 The SCIM indoor mobility item ranges from total assistance, to wheelchair use, to walking with aids, to walking without aids, and has 1005
Articles
Derivation group (n=1282)
Temporal validation group (n=389)
Setting
19 European SCI centres
13 European SCI Centres
Inclusion period
July, 2001, to June, 2008
July, 2008, to December, 2009
Adults with complete initial neurological examination within the first 2 weeks after traumatic spinal cord injury*
640 (50%)
214 (55%)
Sex (male)
504 (79%)
169 (79%)
Mean age at injury in years (SD, range)
44 (17, 18–92)
≥65 years
108 (17%)
Mean timing of examination in days after injury (SD, range) Examination <72 h after injury
7·7 (4·7, 0–15) 123 (19%)
47 (19, 18–89) 49 (23%) 8·0 (4·6, 0–15) 34 (16%)
Severity of initial neurological deficit AIS grade A
314 (49%)
85 (40%)
AIS grade B
88 (14%)
26 (12%)
AIS grade C
96 (15%)
46 (21%)
AIS grade D
142 (22%)
57 (27%)
Patients with tetraplegia
341 (53%)
114 (53%)
1-year follow-up measure
374 (59%)
54 (25%)
Only 6-month follow-up measure
118 (18%)
45 (21%)
Individuals who can walk independently†
200 (41%)
43 (43%)
Statistical analysis
Outcome measurements
Data are n (%) unless otherwise stated. AIS=American Spinal Injury Association/International Spinal Cord Society neurological standard scale.7 *Number used to calculate proportions for other characteristics. †% is proportion of patients with available follow-up data.
Table 1: Baseline characteristics
1442 patients with spinal cord injury 56 aged <18 years 104 non-traumatic spinal cord injury 1282 adult patients with traumatic spinal cord injury 578 no neurological examination within the first 15 days after injury 64 measures not testable 640 with a complete initial neurological examination
148 without 6-month or 12-month follow-up measurements
374 with 12-month follow-up measurements
118 with 6-month follow-up measurements only
492 with available outcome measures
Figure 1: Selection of patients
shown excellent reliability and construct validity in patients with spinal cord injury.12 To distinguish between individuals who could walk indoors 1006
independently and those who could not, a cutoff SCIM indoor mobility score was applied; scores 0–3 were grouped and defined as unable to walk or dependent on assistance while walking and scores 4–8 were grouped and defined as able to walk independently (panel 2).8 To gain insight into the prospects of a patient being able to walk outdoors independently, we did an ancillary correlation analysis between the SCIM indoor mobility outcomes and ambulation outcomes for moderate distance (10–100 m; SCIM item 13) and outdoors (>100 m; SCIM item 14).11,12 In spinal cord injury research, 1-year follow-up measurements are generally thought to be representative for the assessment of long-term outcomes.13 For patients without 1-year follow-up measurements, 6-month followup measurements were used, as previously validated.8 Physicians, physiotherapists, and occupational therapists who assessed the SCIM measurements were not masked to the initial neurological examination results.
A descriptive analysis of patients’ characteristics was done with absolute and relative frequencies for qualitative variables and means (SD) for quantitative variables. Positive and negative predictive values were calculated from contingency tables with 95% CIs with the binomial exact method. Because the neurological candidate predictors included in analysis are highly correlated, several models with almost equivalent performances can be constructed. We applied an exhaustive model search in which all logistic regression models, with a maximum of seven predictor variables, were assessed. The Akaike information criterion was calculated for each model to assess the goodness-of-fit.14 The smaller the number, the more accurate the model. We identified the most accurate models, including the model with the lowest Akaike information criterion and those with a maximum of four points more. If any of these models had almost equivalent performances we selected the best one on the basis of the number of variables included (the smaller the better) and its ease of use in clinical settings. The relative weighting of every variable included in the final model was based on each variable’s β value in logistic regression analysis. We calculated predicted probabilities on the basis of these weighted values. The performance of each prediction rule was quantified by its discriminatory ability, which was defined as the area under receiver-operating-characteristics curve (AUC). This curve shows a model’s ability to discriminate between patients who can walk independently after 1 year and those who cannot.15 Calibration of predictions was assessed graphically by plotting recorded frequencies against predicted probabilities. Several ancillary analyses were done. First, the potential additional predictive value of PPS scores, the timing of examination (≤24 h, <72 h, or <15 days after injury), and www.thelancet.com Vol 377 March 19, 2011
Articles
the level of injury (tetraplegia or paraplegia) were examined by the addition of these variables separately to the final model.3,16–19 Second, the AUC of the newly derived prediction rule was compared with the AUC of the AIS grading system.7 Third, we calculated the agreement between dichotomous SCIM indoor mobility outcomes and moderate distance and outdoor mobility outcomes with the kappa statistic (κ). Finally, for temporal validation, the performance of the clinical prediction rule was assessed for individuals with traumatic spinal cord injury who were included in the EM-SCI network between July, 2008, and December, 2009. No alteration in the prognostic score was allowed after temporal validation began. Statistical analyses were done with the SPSS software package version 16.0.02 and the R software package version 2.10.1.
Range of Weighted Minimum test scores coefficient score Age ≥65 years
0–1
–10
–10
0
Motor score L3
0–5
2
0
10
Motor score S1
0–5
2
0
10
Light touch score L3
0–2
5
0
10
Light touch score S1
0–2
5
0
10
–10
40
Total
Only the best score of each motor score or light touch score (ie, right or left) should be applied for the prediction rule (see Methods).
Table 2: Clinical prediction rule variables
100
Role of the funding source
Results Between July, 2001, and June, 2008, 1442 patients with spinal cord injury were admitted to one of 19 EM-SCI centres. Of 1282 adult patients with traumatic injury in the study population (table 1), 640 had completely documented neurological examinations assessed within the first 15 days after injury and were included for analysis (table 1; figure 1). The clinical characteristics of individuals included in the analysis were much the same as those of individuals excluded (webappendix p 1). Ambulation outcome measures were available in 492 patients (77%, figure 1). The clinical characteristics of patients with 1-year follow-up measurements were much the same as those of patients with 6-month follow-up measurements and of patients without follow-up measurements (webappendix p 2). After logistic regression analysis, 11 different models consisting of age and four neurological predictors were most significantly related to ambulation outcomes (webappendix pp 3–6 shows complete datasets of the best models). The final model was selected on the basis of its simplicity of use and included age (dichotomised at 65 years) and four neurological predictors: quadriceps femoris muscle grade (L3), gastrocsoleus muscle grade (S1), LTS at L3, and LTS at S1. We estimated the probability of an individual being able to walk independently 1 year after traumatic spinal cord injury with the weighted coefficients of the final prediction rule (table 2), with a minimum total score of –10 and a maximum total score of 40. To calculate the probability of being able to walk independently with this prediction rule score, we used the following equation: e–3·273+0·267×score/1 + e–3·279+0·267×score. Figure 2 provides a graphical representation of the equation. www.thelancet.com Vol 377 March 19, 2011
90
80
Probability of walking independently (%)
The sponsor of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Maximum score
70
60
50
40
30
20
10
0 –10
–5
0
5
10 15 20 Prediction rule score
25
30
35
40
Figure 2: Probability of walking independently 1 year after injury based on the prediction rule score The shaded area around the curve is the 95% CI of the prediction rule based on the regression model. The dotted lines are a visual aid to determine the probability of walking independently.
The prediction rule distinguished well between those patients who were able to walk independently and those who were not (AUC 0·956, 95% CI 0·936–0·976, p<0·0001; webappendix p 8). To visualise the calibration of the prediction rule, the total sample was divided into four groups that contained roughly the same number of patients (figure 3). Ancillary analyses showed that neither level of injury (p=0·659) nor timing of examination (p=0·312) had a significant additional value with respect to prediction of an individual’s ability to walk independently. The addition of aggregated lumbosacral PPS scores to the final model did not significantly improve its fit (p=0·339). However, after applying a backward selection we noted one significant additional effect for PPS at L5 (p=0·017). The
See Online for webappendix
1007
Articles
A
B
Probability of walking independently (%)
1·0
0·8
0·6
0·4
0·2
0 –10
0
10 20 Prediction rule score
30
40
–10
0
10 20 Prediction rule score
30
40
Figure 3: Calibration plots of the prediction rule scores divided into four intervals (A) Data from the 492 patients in the derivation group. (B) Data from the 99 patients in the validation group. The size of each point corresponds to the number of patients in the interval and the vertical bars are the 95% CIs. The vertical stripes at the lower horizontal border represent the prediction rule scores of patients who were not able to walk independently. The vertical stripes at the upper horizontal border represent the prediction rule scores of patients who were able to walk independently.
AUC of the model with the addition of PPS scores at L5 was slightly higher than was the AUC of the prediction rule without this variable (0·959 [95% CI 0·940–0·978] vs 0·956). Table 3 shows the predictive values of the AIS grading system. The accuracy of the prediction rule was significantly higher (change in AUC: 0·058, p<0·0001, 95% CI 0·030–0·086) than was the accuracy of the AIS grading system (AUC: 0·898, 0·867–0·928, p<0·0001). The prediction rule had a clear additional clinical value for the prediction of an individual’s ability to walk independently in each of the AIS grades (webappendix p 9). We recorded highly significant correlations of SCIM item 12 with SCIM items 13 (κ=0·962, p<0·0001) and 14 (κ=0·862, p<0·0001; webappendix p 7). Between July, 2008, and December, 2009, 389 adults with traumatic spinal cord injury were admitted to one of 13 EM-SCI centres. 214 patients with completely documented neurological examinations assessed within the first 15 days after injury were included in analysis (table 1). Because analysis was done before some 1-year follow-up measurements could be recorded, 1-year followup data were available for a smaller proportion of patients in the validation group than in the derivation group (table 1). The discriminating ability of the prediction rule in the validation group was excellent (AUC 0·967, 95% CI 0·939–0·995, p<0·0001; webappendix p 8). Figure 3 shows the calibration of the prediction rule with data from patients in the validation group. Although, because of the smaller sample size, deviations from the predicted probability of the four intervals were more apparent in the validation group than they were in the derivation group, the calibration was very good. The addition of PPS scores 1008
N (%)
Negative predictive value (% [95% CI])
Positive predictive value (% [95% CI])
AIS grade A
240 (49)
91·7 (87·4–94·8)
8·3 (5·2–12·6)
AIS grade B
66 (13)
60·6 (47·8–72·4)
39·4 (27·6–52·2)
AIS grade C
76 (16)
38·2 (27·3–50·0)
61·8 (50·0–72·8)
AIS grade D
110 (22)
2·7 (0·6–7·8)
97·3 (92·2–99·4)
AIS=American Spinal Injury Association/International Spinal Cord Society neurological standard scale.
Table 3: The predictive value of the AIS grading system to discriminate between the ability to walk independently or not 1 year after injury
at L5 (with weighting derived from the derivation set) to the prediction rule resulted in a slightly lower AUC (0·964, 95% CI 0·935–0·994, p<0·0001) compared with the AUC of the prediction rule alone.
Discussion We have developed a simple clinical prediction rule derived from data from a large prospective European database that can be used by physicians to counsel patients with traumatic spinal cord injury and their families during the initial phase after injury. On the basis of age and four clinical neurological parameters, a patient’s long-term probability of walking independently after injury can be calculated more accurately than it can with the widely used AIS grading system. Studies10,20 have shown that lower extremity motor scores (at times combined with sensory tests) are better than AIS grades alone to predict the likelihood of independent walking www.thelancet.com Vol 377 March 19, 2011
Articles
after traumatic spinal cord injury. Our prediction rule not only accords with these previous clinical data, but provides a statistically reliable basis for prediction of walking after such injury with an efficient and simple clinical examination. Many neurological variables have been assessed for their predictive value of ambulation outcomes.7,8,20–23 Several studies have shown the prognostic value of the early assessment of only one neurological predictor, such as strength of the quadriceps femoris,24 strength of the hip flexors,25 or anal sensation,20,21 but, except for a clinical trial20 and a European database,8 these studies have used small samples. Use of multivariate prognostic models to determine outcomes after neurotrauma (eg, traumatic brain injury) that include large samples and apply external validation have gained increased recognition,26 but such models have not been applied to traumatic spinal cord injury. Ours is an accurate and well validated prediction rule for walking after traumatic spinal cord injury. Although our prediction rule is more accurate and less time consuming than the AIS grading system, to do accurate and reliable assessments of the four neurological tests, a physician must have experience in the physical examination of patients with traumatic spinal cord injury.27 Furthermore, for assessment of injury severity and effectiveness of treatments, the international standards for neurological classification of spinal cord injury are the reference standard.27 Neurophysiological variables such as somatosensory evoked potentials have been assessed for their prognostic value on ambulation outcomes,22 but they are time consuming to test and are therefore not suitable for inclusion in a simple prediction rule. Nonetheless, neurophysiological assessments can be of value in patients who cannot participate in a reliable physical examination. Variables that are highly correlated with others contribute little independent information and can be excluded before the development of a prognostic model.9 By contrast with earlier reports,21 a high-volume study8 from the EM-SCI consortium showed that sacral PPS and LTS scores have a similar discriminative ability for prediction of an individual’s ability to walk after traumatic spinal cord injury. Because we wanted the prediction rule to be as simple as possible, we included only LTS scores in the initial model. Although the addition of the PPS at L5 to the prediction rule resulted in a slightly higher AUC in the derivation group, its inclusion resulted in a marginally lower AUC in the validation group. This occurrence was probably because, with the addition of PPS at L5, the model was overfitted to the dataset from which it was derived. Overall, we think that the exclusion of PPS scores before development of a model is a valid approach. Because many of the EM-SCI centres are referral centres, most (81%) of the neurological assessments had not been done within 72 h after injury. There is no consensus about the difference between the prognostic value of immediate www.thelancet.com Vol 377 March 19, 2011
(<24 h) versus subacute (<72 h) examinations.16 A post-hoc analysis in our study population showed that the timing of examination (<24 h, <72 h, or <15 days after injury) did not have a pronounced effect on the accuracy of the prediction rule. Furthermore, whereas Kirshblum and colleagues3 postulated that patients with incomplete tetraplegia are less likely to be able to walk independently than are patients with incomplete paraplegia, we noted no difference in outcome between patients with tetraplegia and those with paraplegia. A dichotomisation of SCIM item 12 was applied as the primary functional outcome measure.8,11,12 The present study accords with previous studies,28 showing that the SCIM indoor mobility outcome is strongly correlated with moderate and outdoor distance outcomes. Our prediction rule, however, can be applied to predict the ability to walk independently for indoor distances only. Strengths of our study include the prospectively collected data in a large European population, the availability of validated and detailed information about patients’ initial neurological impairments assessed by trained and certified physicians, the use of a well validated clinical outcome measure for ambulation (SCIM), and a temporal validation of the derived clinical prediction rule. Nonetheless, several potential limitations of our study exist. Although the applied dichotomous outcome is easy to use, it does not provide detailed information about a patient’s quality of walking. Furthermore, because some EM-SCI centres are specialised rehabilitation centres, acute-phase measurements were absent for many patients. Nonetheless, the clinical characteristics of patients who were excluded were much the same as for those who were included (webappendix p 1). Details of patients lost to follow-up (eg, mortality) have not been documented, which might have resulted in an overoptimistic prediction model. Before application of the prediction rule in clinical practice, an external validation study is needed to assess its generalisability.29 Moreover, the clinical efficacy of the prediction rule also needs to be established by investigation of whether its use results in more efficient use of rehabilitation resources and improved psychological wellbeing of patients with spinal cord injury.3,30 Finally, although no effective treatment that results in major neurological or functional recovery is available, future effective treatment strategies might necessitate a reassessment of the prediction rule’s accuracy.31 Contributors AJFH, AC, ACHG, and HVdM are all senior authors, managed the project, and obtained funding. JJvM and ARTD did data analysis and the preparation of the final report. All authors contributed to the writing of the paper and read and approved the final version. Conflicts of interest We declare that we have no conflicts of interest. Acknowledgments This work was supported by a grant from the Internationale Stiftung für Forschung in Paraplegie (IFP), Zürich, Switzerland. All spinal cord injury centres participating in the EM-SCI network contributed to the study.
1009
Articles
References 1 Wyndaele M, Wyndaele JJ. Incidence, prevalence and epidemiology of spinal cord injury: what learns a worldwide literature survey? Spinal Cord 2006; 44: 523–29. 2 Tator CH. Review of treatment trials in human spinal cord injury: issues, difficulties, and recommendations. Neurosurgery 2006; 59: 957–82. 3 Kirshblum SC, Priebe MM, Ho CH, Scelza WM, Chiodo AE, Wuermser LA. Spinal cord injury medicine: 3—Rehabilitation phase after acute spinal cord injury. Arch Phys Med Rehabil 2007; 88: S62–70. 4 Ditunno PL, Patrick M, Stineman M, Ditunno JF Jr. Who wants to walk? Preferences for recovery after SCI: a longitudinal and cross-sectional study. Spinal Cord 2008; 46: 500–06. 5 van Hedel HJ, Wirz M, Dietz V. Standardized assessment of walking capacity after spinal cord injury: the European network approach. Neurol Res 2008; 30: 61–73. 6 Furlan JC, Fehlings MG. The impact of age on mortality, impairment, and disability among adults with acute traumatic spinal cord injury. J Neurotrauma 2009; 26: 1707–17. 7 American Spinal Injury Association: International standards for neurological classification of spinal cord injury, revised 2002. Chicago, IL: American Spinal Injury Association; 2002. 8 van Middendorp JJ, Hosman AJ, Pouw MH, Van de Meent H. Is determination between complete and incomplete traumatic spinal cord injury clinically relevant? Validation of the ASIA sacral sparing criteria in a prospective cohort of 432 patients. Spinal Cord 2009; 47: 809–16. 9 Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ 2009; 338: b604. 10 Marino RJ, Graves DE. Metric properties of the ASIA motor score: subscales improve correlation with functional activities. Arch Phys Med Rehabil 2004; 85: 1804–10. 11 Catz A, Itzkovich M, Steinberg F, et al. The Catz-Itzkovich SCIM: a revised version of the spinal cord independence measure. Disabil Rehabil 2001; 23: 263–68. 12 Catz A, Itzkovich M, Tesio L, et al. A multicenter international study on the spinal cord independence measure, version III: Rasch psychometric validation. Spinal Cord 2007; 45: 275–91. 13 Kirshblum S, Millis S, McKinley W, Tulsky D. Late neurologic recovery after traumatic spinal cord injury. Arch Phys Med Rehabil 2004; 85: 1811–17. 14 Akaike H. A new look at the statistical model identification: system identification and time-series analysis. IEEE Trans Automat Contr 1974; 19: 716–23. 15 Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148: 839–43. 16 Kirshblum SC, O’Connor KC. Predicting neurologic recovery in traumatic cervical spinal cord injury. Arch Phys Med Rehabil 1998; 79: 1456–66.
1010
17
18
19
20
21
22
23
24
25 26
27
28
29
30
31
Penrod LE, Hegde SK, Ditunno JF Jr. Age effect on prognosis for functional recovery in acute, traumatic central cord syndrome. Arch Phys Med Rehabil 1990; 71: 963–8. Burns SP, Golding DG, Rolle WA Jr, Graziani V, Ditunno JF Jr. Recovery of ambulation in motor-incomplete tetraplegia. Arch Phys Med Rehabil 1997; 78: 1169–72. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837–45. Oleson CV, Burns AS, Ditunno JF, Geisler FH, Coleman WP. Prognostic value of pinprick preservation in motor complete, sensory incomplete spinal cord injury. Arch Phys Med Rehabil 2005; 86: 988–92. Crozier KS, Graziani V, Ditunno JF Jr, Herbison GJ. Spinal cord injury: prognosis for ambulation based on sensory examination in patients who are initially motor complete. Arch Phys Med Rehabil 1991; 72: 119–21. Curt A, Dietz V. Ambulatory capacity in spinal cord injury: significance of somatosensory evoked potentials and ASIA protocol in predicting outcome. Arch Phys Med Rehabil 1997; 78: 39–43. Waters RL, Adkins R, Yakura J, Vigil D. Prediction of ambulatory performance based on motor scores derived from standards of the American spinal injury association. Arch Phys Med Rehabil 1994; 75: 756–60. Crozier KS, Cheng LL, Graziani V, Zorn G, Herbison G, Ditunno JF Jr. Spinal cord injury: prognosis for ambulation based on quadriceps recovery. Paraplegia 1992; 30: 762–67. Hussey RW, Stauffer ES. Spinal cord injury: requirements for ambulation. Arch Phys Med Rehabil 1973; 54: 544–47. Lingsma HF, Roozenbeek B, Steyerberg EW, Murray GD, Maas AI. Early prognosis in traumatic brain injury: from prophecies to predictions. Lancet Neurol 2010; 9: 543–54. Steeves JD, Lammertse D, Curt A, et al. Guidelines for the conduct of clinical trials for spinal cord injury (SCI) as developed by the ICCP panel: clinical trial outcome measures. Spinal Cord 2007; 45: 206–21. van Hedel HJ, Dietz V. Walking during daily life can be validly and responsively assessed in subjects with a spinal cord injury. Neurorehabil Neural Repair 2009; 23: 117–24. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ 2009; 338: b605. Widerström-Noga EG, Felipe-Cuervo E, Broton JG, Duncan RC, Yezierski RP. Perceived difficulty in dealing with consequences of spinal cord injury. Arch Phys Med Rehabil 1999; 80: 580–86. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009; 338: b606.
www.thelancet.com Vol 377 March 19, 2011
A clinical prediction rule for ambulation outcomes after traumatic spinal cord injury: a longitudinal cohort study Joost J van Middendorp, Allard J F Hosman, A Rogier T Donders, Martin H Pouw, John F Ditunno Jr, Armin Curt, Alexander C H Geurts, Hendrik Van de Meent, for the EM-SCI Study Group
Summary Lancet 2011; 377: 1004–10 Published Online March 4, 2011 DOI:10.1016/S01406736(10)62276-3 See Comment page 972 Spine Unit, Department of Orthopaedics (J J van Middendorp MD, A J F Hosman MD, M H Pouw MD); Department of Epidemiology, Biostatistics and HTA (A R T Donders PhD); and Department of Rehabilitation Medicine (Prof A C H Geurts MD, H Van de Meent MD), Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands; Department of Rehabilitation Medicine, Jefferson Medical College, Thomas Jefferson University, Philadelphia, PA, USA (Prof J F Ditunno Jr MD); and Spinal Cord Injury Center, Balgrist University Hospital, Zurich, Switzerland (Prof A Curt MD) Correspondence to: Dr Joost J van Middendorp, Department of Orthopaedics, PO Box 9101, 6500 HB Nijmegen, Netherlands [email protected]
1004
Background Traumatic spinal cord injury is a serious disorder in which early prediction of ambulation is important to counsel patients and to plan rehabilitation. We developed a reliable, validated prediction rule to assess a patient’s chances of walking independently after such injury. Methods We undertook a longitudinal cohort study of adult patients with traumatic spinal cord injury, with early (within the first 15 days after injury) and late (1-year follow-up) clinical examinations, who were admitted to one of 19 European centres between July, 2001, and June, 2008. A clinical prediction rule based on age and neurological variables was derived from the international standards for neurological classification of spinal cord injury with a multivariate logistic regression model. Primary outcome measure 1 year after injury was independent indoor walking based on the Spinal Cord Independence Measure. Model performances were quantified with respect to discrimination (area under receiver-operating-characteristics curve [AUC]). Temporal validation was done in a second group of patients from July, 2008, to December, 2009. Findings Of 1442 patients with spinal cord injury, 492 had available outcome measures. A combination of age (<65 vs ≥65 years), motor scores of the quadriceps femoris (L3), gastrocsoleus (S1) muscles, and light touch sensation of dermatomes L3 and S1 showed excellent discrimination in distinguishing independent walkers from dependent walkers and non-walkers (AUC 0·956, 95% CI 0·936–0·976, p<0·0001). Temporal validation in 99 patients confirmed excellent discriminating ability of the prediction rule (AUC 0·967, 0·939–0·995, p<0·0001). Interpretation Our prediction rule, including age and four neurological tests, can give an early prognosis of an individual’s ability to walk after traumatic spinal cord injury, which can be used to set rehabilitation goals and might improve the ability to stratify patients in interventional trials. Funding Internationale Stiftung für Forschung in Paraplegie.
Introduction Traumatic spinal cord injury has a profound effect on patients’ physical and psychosocial wellbeing. Although the frequency of such injury is low at 10·4–83 cases per million people worldwide, this devastating disorder imposes a substantial burden on the health-care system.1 Despite advances in basic research into spinal cord repair that show promise, no treatment that results in major neurological or functional recovery is available.2 After a spinal cord injury, a reliable prognosis of a patient’s potential functional outcome is essential for counselling and to design a personalised rehabilitation programme.3 During rehabilitation, recovery of the ability to walk is a high priority for such patients.4 However, no prediction rule for the ability to walk independently after traumatic spinal cord injury is available. In this study we analysed data from a prospective, longitudinal, multicentre cohort study5 of a representative European population with spinal cord injury to develop an accurate and simple clinical prediction rule for a patient’s ability to walk independently. Age at injury6 and variables from the international standards for neurological classification of spinal cord injury7 are used in the clinical algorithm. We tested the
reproducibility and validity of the rule to predict an individual’s ability to walk independently after injury on a second group of patients.
Methods Study design and patient population Since July, 2001, 19 centres (five centres originally) have gathered a standardised dataset of neurological and functional outcomes of patients with spinal cord injury as part of the European Multicenter Study on Human Spinal Cord Injury (EM-SCI).5 Data for neurological and functional status were collected prospectively, per protocol, within the first 15 days and at months 1, 3, 6, and 12 after injury. Because no proven effective treatment is available,2 treatment regimens are not standardised within the EM-SCI network. Details of applied treatments were not recorded systematically, but ranged from nonoperative interventions to very early (<6 h after injury) surgical stabilisation and decompression of the spinal cord. Dependent on level and severity of injury, individually tailored rehabilitation programmes varied in focus and intensity. From the EM-SCI dataset we extracted data for all adult (≥18 years) patients with acute traumatic spinal cord www.thelancet.com Vol 377 March 19, 2011
Articles
injury, including conus medullaris and cauda equina injuries, who were admitted between July, 2001, and June, 2008. Patients who were unable to cooperate with physical examination because of cognitive impairment (as assessed by the examiner), who had a peripheral nerve lesion, or who had neuropathy or polyneuropathy were not included in the EM-SCI database. Polyneuropathy was tested by measurement of ulnar and tibial sensory nerve conduction velocity. Patients with medical records showing frequent causes of polyneuropathy (eg, diabetes mellitus), and those without a complete neurological assessment within the first 15 days after injury were excluded from the analysis. The multicentre follow-up study was done in accordance with the ethics standards in the updated version of the 1964 Declaration of Helsinki. The study protocol was approved by the local ethics committees of all participating centres, and the patients gave informed, oral consent before entering the study.
Prognostic variables For the prognostic model we considered patients’ age and variables from their initial neurological examination. Older patients (≥65 years) with spinal cord injury have less potential to translate neurological improvements into functional recovery than do younger patients.6 Therefore, age was categorised into two groups: patients younger than 65 years and those aged 65 years and older. Neurological examinations were done according to the international standards for classification of spinal cord injuries.7 Examination included motor score testing (graded on a five-point scale adapted from the Medical Research Council scale), light touch sensory (LTS) and pinprick sensory (PPS) testing (0=absent, 1=impaired, and 2=normal), and sacral sparing scores, including voluntary anal contraction and anal sensation (0=absent and 1=present).7 Muscle testing was done in the supine position. Because the PPS scores and the LTS scores are highly correlated,8 we included only one of the two sensory scoring systems in the initial model.9 Because an LTS score of 0 means that light touch sensation is absent and a PPS score of 0 means that there could be local sensation, but the separation of dull and sharp sensation is absent,7 we used only the LTS scoring system in analysis because we thought it to be the least prone to error. To validate this approach the final model was tested with the addition of PPS scores. For every patient, we included only the best scores of each level (ie, right or left) of the lower extremity and sacral scores for analysis.8 Clinical assessments were done by trained and certified neurologists and rehabilitation physicians with at least 1 year of experience in examination of patients with spinal cord injury. Motor and sensory scores were recorded in the electronic EM-SCI database and the quality and correctness of the data were monitored centrally by a data quality manager. Patients’ American Spinal Injury Association/ International Spinal Cord Society neurological standard www.thelancet.com Vol 377 March 19, 2011
Panel 1: American Spinal Injury Association/International Spinal Cord Society neurological standard scale7 Grade A No motor or sensory function is preserved in the sacral segments S4–S5 Grade B Sensory but not motor function is preserved below the neurological level and includes the sacral segments S4–S5 Grade C Motor function is preserved below the neurological level, and more than half of key muscles below the neurological level have a muscle grade of less than 3 Grade D Motor function is preserved below the neurological level, and at least half of key muscles below the neurological level have a muscle grade of 3 or more Grade E Motor and sensory function are normal
Panel 2: Spinal Cord Independence Measure item 12—mobility indoors11,12 0: Requires total assistance 1: Needs electric wheelchair or partial assistance to operate manual wheelchair 2: Moves independently in manual wheelchair 3: Requires supervision while walking (with or without devices) 4: Walks with a walking frame or crutches (swing) 5: Walks with crutches or two canes (reciprocal walking) 6: Walks with one cane 7: Needs leg orthosis only 8: Walks without walking aids We applied a cutoff SCIM (Spinal Cord Independence Measure ) mobility score to differentiate between patients who are unable to walk or are dependent on assistance while walking (scores 0–3) and those who are able to walk independently (scores 4–8).
scale (AIS) grades were computed automatically according to the international standards (panel 1).7 Because the aim of this study was to introduce a simple clinical prediction rule with minimum burden on patients and maximum time efficiency for physicians, we did not include aggregated neurological scores (eg, total lower extremity motor score10) in analyses.
Outcome assessment The ability to walk independently 1 year after injury was the primary functional outcome. The Spinal Cord Independence Measure indoor mobility item (SCIM item 12, ability to walk <10 m) was assessed and analysed for this purpose.11,12 The SCIM indoor mobility item ranges from total assistance, to wheelchair use, to walking with aids, to walking without aids, and has 1005
Articles
Derivation group (n=1282)
Temporal validation group (n=389)
Setting
19 European SCI centres
13 European SCI Centres
Inclusion period
July, 2001, to June, 2008
July, 2008, to December, 2009
Adults with complete initial neurological examination within the first 2 weeks after traumatic spinal cord injury*
640 (50%)
214 (55%)
Sex (male)
504 (79%)
169 (79%)
Mean age at injury in years (SD, range)
44 (17, 18–92)
≥65 years
108 (17%)
Mean timing of examination in days after injury (SD, range) Examination <72 h after injury
7·7 (4·7, 0–15) 123 (19%)
47 (19, 18–89) 49 (23%) 8·0 (4·6, 0–15) 34 (16%)
Severity of initial neurological deficit AIS grade A
314 (49%)
85 (40%)
AIS grade B
88 (14%)
26 (12%)
AIS grade C
96 (15%)
46 (21%)
AIS grade D
142 (22%)
57 (27%)
Patients with tetraplegia
341 (53%)
114 (53%)
1-year follow-up measure
374 (59%)
54 (25%)
Only 6-month follow-up measure
118 (18%)
45 (21%)
Individuals who can walk independently†
200 (41%)
43 (43%)
Statistical analysis
Outcome measurements
Data are n (%) unless otherwise stated. AIS=American Spinal Injury Association/International Spinal Cord Society neurological standard scale.7 *Number used to calculate proportions for other characteristics. †% is proportion of patients with available follow-up data.
Table 1: Baseline characteristics
1442 patients with spinal cord injury 56 aged <18 years 104 non-traumatic spinal cord injury 1282 adult patients with traumatic spinal cord injury 578 no neurological examination within the first 15 days after injury 64 measures not testable 640 with a complete initial neurological examination
148 without 6-month or 12-month follow-up measurements
374 with 12-month follow-up measurements
118 with 6-month follow-up measurements only
492 with available outcome measures
Figure 1: Selection of patients
shown excellent reliability and construct validity in patients with spinal cord injury.12 To distinguish between individuals who could walk indoors 1006
independently and those who could not, a cutoff SCIM indoor mobility score was applied; scores 0–3 were grouped and defined as unable to walk or dependent on assistance while walking and scores 4–8 were grouped and defined as able to walk independently (panel 2).8 To gain insight into the prospects of a patient being able to walk outdoors independently, we did an ancillary correlation analysis between the SCIM indoor mobility outcomes and ambulation outcomes for moderate distance (10–100 m; SCIM item 13) and outdoors (>100 m; SCIM item 14).11,12 In spinal cord injury research, 1-year follow-up measurements are generally thought to be representative for the assessment of long-term outcomes.13 For patients without 1-year follow-up measurements, 6-month followup measurements were used, as previously validated.8 Physicians, physiotherapists, and occupational therapists who assessed the SCIM measurements were not masked to the initial neurological examination results.
A descriptive analysis of patients’ characteristics was done with absolute and relative frequencies for qualitative variables and means (SD) for quantitative variables. Positive and negative predictive values were calculated from contingency tables with 95% CIs with the binomial exact method. Because the neurological candidate predictors included in analysis are highly correlated, several models with almost equivalent performances can be constructed. We applied an exhaustive model search in which all logistic regression models, with a maximum of seven predictor variables, were assessed. The Akaike information criterion was calculated for each model to assess the goodness-of-fit.14 The smaller the number, the more accurate the model. We identified the most accurate models, including the model with the lowest Akaike information criterion and those with a maximum of four points more. If any of these models had almost equivalent performances we selected the best one on the basis of the number of variables included (the smaller the better) and its ease of use in clinical settings. The relative weighting of every variable included in the final model was based on each variable’s β value in logistic regression analysis. We calculated predicted probabilities on the basis of these weighted values. The performance of each prediction rule was quantified by its discriminatory ability, which was defined as the area under receiver-operating-characteristics curve (AUC). This curve shows a model’s ability to discriminate between patients who can walk independently after 1 year and those who cannot.15 Calibration of predictions was assessed graphically by plotting recorded frequencies against predicted probabilities. Several ancillary analyses were done. First, the potential additional predictive value of PPS scores, the timing of examination (≤24 h, <72 h, or <15 days after injury), and www.thelancet.com Vol 377 March 19, 2011
Articles
the level of injury (tetraplegia or paraplegia) were examined by the addition of these variables separately to the final model.3,16–19 Second, the AUC of the newly derived prediction rule was compared with the AUC of the AIS grading system.7 Third, we calculated the agreement between dichotomous SCIM indoor mobility outcomes and moderate distance and outdoor mobility outcomes with the kappa statistic (κ). Finally, for temporal validation, the performance of the clinical prediction rule was assessed for individuals with traumatic spinal cord injury who were included in the EM-SCI network between July, 2008, and December, 2009. No alteration in the prognostic score was allowed after temporal validation began. Statistical analyses were done with the SPSS software package version 16.0.02 and the R software package version 2.10.1.
Range of Weighted Minimum test scores coefficient score Age ≥65 years
0–1
–10
–10
0
Motor score L3
0–5
2
0
10
Motor score S1
0–5
2
0
10
Light touch score L3
0–2
5
0
10
Light touch score S1
0–2
5
0
10
–10
40
Total
Only the best score of each motor score or light touch score (ie, right or left) should be applied for the prediction rule (see Methods).
Table 2: Clinical prediction rule variables
100
Role of the funding source
Results Between July, 2001, and June, 2008, 1442 patients with spinal cord injury were admitted to one of 19 EM-SCI centres. Of 1282 adult patients with traumatic injury in the study population (table 1), 640 had completely documented neurological examinations assessed within the first 15 days after injury and were included for analysis (table 1; figure 1). The clinical characteristics of individuals included in the analysis were much the same as those of individuals excluded (webappendix p 1). Ambulation outcome measures were available in 492 patients (77%, figure 1). The clinical characteristics of patients with 1-year follow-up measurements were much the same as those of patients with 6-month follow-up measurements and of patients without follow-up measurements (webappendix p 2). After logistic regression analysis, 11 different models consisting of age and four neurological predictors were most significantly related to ambulation outcomes (webappendix pp 3–6 shows complete datasets of the best models). The final model was selected on the basis of its simplicity of use and included age (dichotomised at 65 years) and four neurological predictors: quadriceps femoris muscle grade (L3), gastrocsoleus muscle grade (S1), LTS at L3, and LTS at S1. We estimated the probability of an individual being able to walk independently 1 year after traumatic spinal cord injury with the weighted coefficients of the final prediction rule (table 2), with a minimum total score of –10 and a maximum total score of 40. To calculate the probability of being able to walk independently with this prediction rule score, we used the following equation: e–3·273+0·267×score/1 + e–3·279+0·267×score. Figure 2 provides a graphical representation of the equation. www.thelancet.com Vol 377 March 19, 2011
90
80
Probability of walking independently (%)
The sponsor of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Maximum score
70
60
50
40
30
20
10
0 –10
–5
0
5
10 15 20 Prediction rule score
25
30
35
40
Figure 2: Probability of walking independently 1 year after injury based on the prediction rule score The shaded area around the curve is the 95% CI of the prediction rule based on the regression model. The dotted lines are a visual aid to determine the probability of walking independently.
The prediction rule distinguished well between those patients who were able to walk independently and those who were not (AUC 0·956, 95% CI 0·936–0·976, p<0·0001; webappendix p 8). To visualise the calibration of the prediction rule, the total sample was divided into four groups that contained roughly the same number of patients (figure 3). Ancillary analyses showed that neither level of injury (p=0·659) nor timing of examination (p=0·312) had a significant additional value with respect to prediction of an individual’s ability to walk independently. The addition of aggregated lumbosacral PPS scores to the final model did not significantly improve its fit (p=0·339). However, after applying a backward selection we noted one significant additional effect for PPS at L5 (p=0·017). The
See Online for webappendix
1007
Articles
A
B
Probability of walking independently (%)
1·0
0·8
0·6
0·4
0·2
0 –10
0
10 20 Prediction rule score
30
40
–10
0
10 20 Prediction rule score
30
40
Figure 3: Calibration plots of the prediction rule scores divided into four intervals (A) Data from the 492 patients in the derivation group. (B) Data from the 99 patients in the validation group. The size of each point corresponds to the number of patients in the interval and the vertical bars are the 95% CIs. The vertical stripes at the lower horizontal border represent the prediction rule scores of patients who were not able to walk independently. The vertical stripes at the upper horizontal border represent the prediction rule scores of patients who were able to walk independently.
AUC of the model with the addition of PPS scores at L5 was slightly higher than was the AUC of the prediction rule without this variable (0·959 [95% CI 0·940–0·978] vs 0·956). Table 3 shows the predictive values of the AIS grading system. The accuracy of the prediction rule was significantly higher (change in AUC: 0·058, p<0·0001, 95% CI 0·030–0·086) than was the accuracy of the AIS grading system (AUC: 0·898, 0·867–0·928, p<0·0001). The prediction rule had a clear additional clinical value for the prediction of an individual’s ability to walk independently in each of the AIS grades (webappendix p 9). We recorded highly significant correlations of SCIM item 12 with SCIM items 13 (κ=0·962, p<0·0001) and 14 (κ=0·862, p<0·0001; webappendix p 7). Between July, 2008, and December, 2009, 389 adults with traumatic spinal cord injury were admitted to one of 13 EM-SCI centres. 214 patients with completely documented neurological examinations assessed within the first 15 days after injury were included in analysis (table 1). Because analysis was done before some 1-year follow-up measurements could be recorded, 1-year followup data were available for a smaller proportion of patients in the validation group than in the derivation group (table 1). The discriminating ability of the prediction rule in the validation group was excellent (AUC 0·967, 95% CI 0·939–0·995, p<0·0001; webappendix p 8). Figure 3 shows the calibration of the prediction rule with data from patients in the validation group. Although, because of the smaller sample size, deviations from the predicted probability of the four intervals were more apparent in the validation group than they were in the derivation group, the calibration was very good. The addition of PPS scores 1008
N (%)
Negative predictive value (% [95% CI])
Positive predictive value (% [95% CI])
AIS grade A
240 (49)
91·7 (87·4–94·8)
8·3 (5·2–12·6)
AIS grade B
66 (13)
60·6 (47·8–72·4)
39·4 (27·6–52·2)
AIS grade C
76 (16)
38·2 (27·3–50·0)
61·8 (50·0–72·8)
AIS grade D
110 (22)
2·7 (0·6–7·8)
97·3 (92·2–99·4)
AIS=American Spinal Injury Association/International Spinal Cord Society neurological standard scale.
Table 3: The predictive value of the AIS grading system to discriminate between the ability to walk independently or not 1 year after injury
at L5 (with weighting derived from the derivation set) to the prediction rule resulted in a slightly lower AUC (0·964, 95% CI 0·935–0·994, p<0·0001) compared with the AUC of the prediction rule alone.
Discussion We have developed a simple clinical prediction rule derived from data from a large prospective European database that can be used by physicians to counsel patients with traumatic spinal cord injury and their families during the initial phase after injury. On the basis of age and four clinical neurological parameters, a patient’s long-term probability of walking independently after injury can be calculated more accurately than it can with the widely used AIS grading system. Studies10,20 have shown that lower extremity motor scores (at times combined with sensory tests) are better than AIS grades alone to predict the likelihood of independent walking www.thelancet.com Vol 377 March 19, 2011
Articles
after traumatic spinal cord injury. Our prediction rule not only accords with these previous clinical data, but provides a statistically reliable basis for prediction of walking after such injury with an efficient and simple clinical examination. Many neurological variables have been assessed for their predictive value of ambulation outcomes.7,8,20–23 Several studies have shown the prognostic value of the early assessment of only one neurological predictor, such as strength of the quadriceps femoris,24 strength of the hip flexors,25 or anal sensation,20,21 but, except for a clinical trial20 and a European database,8 these studies have used small samples. Use of multivariate prognostic models to determine outcomes after neurotrauma (eg, traumatic brain injury) that include large samples and apply external validation have gained increased recognition,26 but such models have not been applied to traumatic spinal cord injury. Ours is an accurate and well validated prediction rule for walking after traumatic spinal cord injury. Although our prediction rule is more accurate and less time consuming than the AIS grading system, to do accurate and reliable assessments of the four neurological tests, a physician must have experience in the physical examination of patients with traumatic spinal cord injury.27 Furthermore, for assessment of injury severity and effectiveness of treatments, the international standards for neurological classification of spinal cord injury are the reference standard.27 Neurophysiological variables such as somatosensory evoked potentials have been assessed for their prognostic value on ambulation outcomes,22 but they are time consuming to test and are therefore not suitable for inclusion in a simple prediction rule. Nonetheless, neurophysiological assessments can be of value in patients who cannot participate in a reliable physical examination. Variables that are highly correlated with others contribute little independent information and can be excluded before the development of a prognostic model.9 By contrast with earlier reports,21 a high-volume study8 from the EM-SCI consortium showed that sacral PPS and LTS scores have a similar discriminative ability for prediction of an individual’s ability to walk after traumatic spinal cord injury. Because we wanted the prediction rule to be as simple as possible, we included only LTS scores in the initial model. Although the addition of the PPS at L5 to the prediction rule resulted in a slightly higher AUC in the derivation group, its inclusion resulted in a marginally lower AUC in the validation group. This occurrence was probably because, with the addition of PPS at L5, the model was overfitted to the dataset from which it was derived. Overall, we think that the exclusion of PPS scores before development of a model is a valid approach. Because many of the EM-SCI centres are referral centres, most (81%) of the neurological assessments had not been done within 72 h after injury. There is no consensus about the difference between the prognostic value of immediate www.thelancet.com Vol 377 March 19, 2011
(<24 h) versus subacute (<72 h) examinations.16 A post-hoc analysis in our study population showed that the timing of examination (<24 h, <72 h, or <15 days after injury) did not have a pronounced effect on the accuracy of the prediction rule. Furthermore, whereas Kirshblum and colleagues3 postulated that patients with incomplete tetraplegia are less likely to be able to walk independently than are patients with incomplete paraplegia, we noted no difference in outcome between patients with tetraplegia and those with paraplegia. A dichotomisation of SCIM item 12 was applied as the primary functional outcome measure.8,11,12 The present study accords with previous studies,28 showing that the SCIM indoor mobility outcome is strongly correlated with moderate and outdoor distance outcomes. Our prediction rule, however, can be applied to predict the ability to walk independently for indoor distances only. Strengths of our study include the prospectively collected data in a large European population, the availability of validated and detailed information about patients’ initial neurological impairments assessed by trained and certified physicians, the use of a well validated clinical outcome measure for ambulation (SCIM), and a temporal validation of the derived clinical prediction rule. Nonetheless, several potential limitations of our study exist. Although the applied dichotomous outcome is easy to use, it does not provide detailed information about a patient’s quality of walking. Furthermore, because some EM-SCI centres are specialised rehabilitation centres, acute-phase measurements were absent for many patients. Nonetheless, the clinical characteristics of patients who were excluded were much the same as for those who were included (webappendix p 1). Details of patients lost to follow-up (eg, mortality) have not been documented, which might have resulted in an overoptimistic prediction model. Before application of the prediction rule in clinical practice, an external validation study is needed to assess its generalisability.29 Moreover, the clinical efficacy of the prediction rule also needs to be established by investigation of whether its use results in more efficient use of rehabilitation resources and improved psychological wellbeing of patients with spinal cord injury.3,30 Finally, although no effective treatment that results in major neurological or functional recovery is available, future effective treatment strategies might necessitate a reassessment of the prediction rule’s accuracy.31 Contributors AJFH, AC, ACHG, and HVdM are all senior authors, managed the project, and obtained funding. JJvM and ARTD did data analysis and the preparation of the final report. All authors contributed to the writing of the paper and read and approved the final version. Conflicts of interest We declare that we have no conflicts of interest. Acknowledgments This work was supported by a grant from the Internationale Stiftung für Forschung in Paraplegie (IFP), Zürich, Switzerland. All spinal cord injury centres participating in the EM-SCI network contributed to the study.
1009
Articles
References 1 Wyndaele M, Wyndaele JJ. Incidence, prevalence and epidemiology of spinal cord injury: what learns a worldwide literature survey? Spinal Cord 2006; 44: 523–29. 2 Tator CH. Review of treatment trials in human spinal cord injury: issues, difficulties, and recommendations. Neurosurgery 2006; 59: 957–82. 3 Kirshblum SC, Priebe MM, Ho CH, Scelza WM, Chiodo AE, Wuermser LA. Spinal cord injury medicine: 3—Rehabilitation phase after acute spinal cord injury. Arch Phys Med Rehabil 2007; 88: S62–70. 4 Ditunno PL, Patrick M, Stineman M, Ditunno JF Jr. Who wants to walk? Preferences for recovery after SCI: a longitudinal and cross-sectional study. Spinal Cord 2008; 46: 500–06. 5 van Hedel HJ, Wirz M, Dietz V. Standardized assessment of walking capacity after spinal cord injury: the European network approach. Neurol Res 2008; 30: 61–73. 6 Furlan JC, Fehlings MG. The impact of age on mortality, impairment, and disability among adults with acute traumatic spinal cord injury. J Neurotrauma 2009; 26: 1707–17. 7 American Spinal Injury Association: International standards for neurological classification of spinal cord injury, revised 2002. Chicago, IL: American Spinal Injury Association; 2002. 8 van Middendorp JJ, Hosman AJ, Pouw MH, Van de Meent H. Is determination between complete and incomplete traumatic spinal cord injury clinically relevant? Validation of the ASIA sacral sparing criteria in a prospective cohort of 432 patients. Spinal Cord 2009; 47: 809–16. 9 Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: developing a prognostic model. BMJ 2009; 338: b604. 10 Marino RJ, Graves DE. Metric properties of the ASIA motor score: subscales improve correlation with functional activities. Arch Phys Med Rehabil 2004; 85: 1804–10. 11 Catz A, Itzkovich M, Steinberg F, et al. The Catz-Itzkovich SCIM: a revised version of the spinal cord independence measure. Disabil Rehabil 2001; 23: 263–68. 12 Catz A, Itzkovich M, Tesio L, et al. A multicenter international study on the spinal cord independence measure, version III: Rasch psychometric validation. Spinal Cord 2007; 45: 275–91. 13 Kirshblum S, Millis S, McKinley W, Tulsky D. Late neurologic recovery after traumatic spinal cord injury. Arch Phys Med Rehabil 2004; 85: 1811–17. 14 Akaike H. A new look at the statistical model identification: system identification and time-series analysis. IEEE Trans Automat Contr 1974; 19: 716–23. 15 Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148: 839–43. 16 Kirshblum SC, O’Connor KC. Predicting neurologic recovery in traumatic cervical spinal cord injury. Arch Phys Med Rehabil 1998; 79: 1456–66.
1010
17
18
19
20
21
22
23
24
25 26
27
28
29
30
31
Penrod LE, Hegde SK, Ditunno JF Jr. Age effect on prognosis for functional recovery in acute, traumatic central cord syndrome. Arch Phys Med Rehabil 1990; 71: 963–8. Burns SP, Golding DG, Rolle WA Jr, Graziani V, Ditunno JF Jr. Recovery of ambulation in motor-incomplete tetraplegia. Arch Phys Med Rehabil 1997; 78: 1169–72. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837–45. Oleson CV, Burns AS, Ditunno JF, Geisler FH, Coleman WP. Prognostic value of pinprick preservation in motor complete, sensory incomplete spinal cord injury. Arch Phys Med Rehabil 2005; 86: 988–92. Crozier KS, Graziani V, Ditunno JF Jr, Herbison GJ. Spinal cord injury: prognosis for ambulation based on sensory examination in patients who are initially motor complete. Arch Phys Med Rehabil 1991; 72: 119–21. Curt A, Dietz V. Ambulatory capacity in spinal cord injury: significance of somatosensory evoked potentials and ASIA protocol in predicting outcome. Arch Phys Med Rehabil 1997; 78: 39–43. Waters RL, Adkins R, Yakura J, Vigil D. Prediction of ambulatory performance based on motor scores derived from standards of the American spinal injury association. Arch Phys Med Rehabil 1994; 75: 756–60. Crozier KS, Cheng LL, Graziani V, Zorn G, Herbison G, Ditunno JF Jr. Spinal cord injury: prognosis for ambulation based on quadriceps recovery. Paraplegia 1992; 30: 762–67. Hussey RW, Stauffer ES. Spinal cord injury: requirements for ambulation. Arch Phys Med Rehabil 1973; 54: 544–47. Lingsma HF, Roozenbeek B, Steyerberg EW, Murray GD, Maas AI. Early prognosis in traumatic brain injury: from prophecies to predictions. Lancet Neurol 2010; 9: 543–54. Steeves JD, Lammertse D, Curt A, et al. Guidelines for the conduct of clinical trials for spinal cord injury (SCI) as developed by the ICCP panel: clinical trial outcome measures. Spinal Cord 2007; 45: 206–21. van Hedel HJ, Dietz V. Walking during daily life can be validly and responsively assessed in subjects with a spinal cord injury. Neurorehabil Neural Repair 2009; 23: 117–24. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ 2009; 338: b605. Widerström-Noga EG, Felipe-Cuervo E, Broton JG, Duncan RC, Yezierski RP. Perceived difficulty in dealing with consequences of spinal cord injury. Arch Phys Med Rehabil 1999; 80: 580–86. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 2009; 338: b606.
www.thelancet.com Vol 377 March 19, 2011