- Email: [email protected]
Confirmation and extension of the validity of the Multiple Sclerosis Walking Scale-12 (MSWS-12)
Journal of the Neurological Sciences 268 (2008) 69 – 73 www.elsevier.com/locate/jns
Confirmation and extension of the validity of the Multiple Sclerosis Walking Scale-12 (MSWS-12) Robert W. Motl ⁎, Erin M. Snook Department of Kinesiology and Community Health, University of Illinois at Urbana–Champaign, United States Received 27 July 2007; received in revised form 31 October 2007; accepted 2 November 2007 Available online 3 December 2007
Abstract Objective: The Multiple Sclerosis Walking Scale-12 (MSWS-12) is a 12-item patient-rated measure of the impact of MS on walking. As validation of scores from a measure is an ongoing and evolving process, the provision of additional evidence is warranted that confirms and extends the validity of inferences from scores on the MSWS-12. Method: Participants (N = 133) were recruited through support group meetings of the Greater Illinois and Indiana State Chapters of the National Multiple Sclerosis Society, and wore an accelerometer for a seven-day period and completed the MSWS-12 and other outcome measures. Results: Confirmatory factor analysis indicated that a single-factor model provided an adequate fit for MSWS-12 scores. MSWS-12 scores demonstrated strong evidence of internal consistency. The correlations between MSWS-12 scores with scores from other scales, including an accelerometer, were consistent with our a priori hypotheses. Conclusion: We provide evidence that both confirms and extends the validity of inferences from scores of the MSWS-12 as a measure of the impact of MS on walking in a community-based sample of individuals with MS. © 2007 Elsevier B.V. All rights reserved. Keywords: Multiple sclerosis; Validity; Walking
1. Introduction Walking ability is often used by researchers and clinicians as a measure of disease progression in individuals with MS. There are multiple measures of walking that have been included in clinical trials and practice, such as Timed Walk Tests (e.g., 25-ft or 6-min walk tests), quantitative movement analysis (i.e., gait kinematics), Ambulation Index, and the Expanded Disability Status Scale (EDSS) [1]. Importantly, all of those measures have limitations as assessments of walking ability, [1] thereby serving as the impetus for developing the ⁎ Corresponding author. University of Illinois at Urbana–Champaign, Department of Kinesiology and Community Health, 350 Freer Hall, 906 South Goodwin Ave., Urbana, IL, 61801, United States. Tel.: +1 217 265 0886; fax: +1 217 244 7322. E-mail address: [email protected] (R.W. Motl). 0022-510X/$ - see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.jns.2007.11.003
Multiple Sclerosis Walking Scale-12 (MSWS-12) [2]. The MSWS-12 is a 12-item patient-rated measure of the impact of MS on walking that was developed using standard methods of test construction and then validated in community and hospital-residing samples of individuals with MS [2]. The initial evidence for the validity of MSWS-12 scores was subsequently confirmed in community-residing and hospital outpatient samples of individuals with MS [3]. As validation of scores from a measure is an ongoing and evolving process, [4,5] the provision of additional evidence is warranted that further establishes the validity of inferences from scores on the MSWS-12. Therefore, we conducted analyses that both confirmed and extended the validity of inferences from the MSWS-12 in a community-based sample of individuals with MS. We initially tested the factorial or structural validity of a uni-dimensional model for MSWS-12 scores and then confirmed the previously reported pattern of correlations
70
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73
between scores from MSWS-12 with the EDSS and subscales of the Multiple Sclerosis Impact Scale (MSIS-29) [6]. We then extended previous research by first examining the correlation between scores from the MSWS-12 with an objective measure of walking provided by accelerometry and second examining the pattern of correlations between MSWS-12 scores with walking and non-walking related subscales of the Multiple Sclerosis Related Symptom Checklist (MS-RS) [7] and Performance Scales (PS) [8]. 2. Methods
(ActiGraph single-axis accelerometer, model 7164 version, Health One Technology, Fort Walton Beach, FL) was included as an objective measure of ambulation [1] that provided a summary measure of movement counts accumulated per day across a seven-day period [11,12]. The MS-SR was included as a measure of the frequency of occurrence of 26 symptoms during the previous four weeks, and the items yielded five subscales of Motor, Brainstem, Sensory, Mental/ Emotional, and Elimination symptoms [7]. The PS consisted of 8 items that reflected subscales of Mobility, Hand Function, Vision, Fatigue, Cognition, Bladder/Bowel, Sensory, and Spasticity [8].
2.1. Participants 2.3. Procedure Participants (N = 133) were recruited through support group meetings of the Greater Illinois and Indiana State Chapters of the National Multiple Sclerosis Society. The mean age of the sample was 51.1 years (SD = 11.1) and the sample was primarily female (n = 104, 78%), Caucasian (n = 123, 93%), employed (n = 71, 53%), married (n = 91, 68%) and well educated (some college education n = 36, 27%; college graduate n = 61, 46%). The mean duration of time since MS diagnosis was 12.0 years (SD = 9.0). The sample consisted of 85 individuals with relapsing–remitting MS, 52 individuals with secondary progressive MS, and 6 individual with primary progressive MS. 2.2. Measures Walking ability was measured using the MSWS-12 [2]. We included a self-reported version of the EDSS [9] and the MSIS-29 as a patient-rated measure of the physical and psychological impact of MS [6,10]. The accelerometer
The procedure was approved by the institutional review board at the University of Illinois at Urbana-Champaign and all participants provided written informed consent. Participants completed a battery of questionnaires that included the MSWS-12, EDSS, MSIS-29, MS-SR, and PS and wore an accelerometer for 7 consecutive days. Participants received $10 remuneration for completing the study. 2.4. Data analysis Descriptive data are presented as mean ± standard deviation. We tested the factorial validity of a uni-dimensional or single-factor model for MSWS-12 scores using confirmatory factor analysis with the robust weighted least squares estimator and polychoric correlations in Mplus version 3.0 [13]. The fit of the model was based on combinatory rules of standardized root mean squared residual (SRMS) ≤ .08 and comparative fit index (CFI) ≥ .95 [14]. The internal
Table 1 Descriptive statistics for the measures in the sample of 133 individuals with multiple sclerosis Variable a
MSWS-12 EDSS a MSIS-29 Psychological a MSIS-29 Physical a Accelerometer b MS-SR Motor a MS-SR Brainstem a MS-SR Sensory a MS-SR Mental/Emotional a MS-SR Elimination a PS Mobility a PS Hand Function a PS Vision a PS Fatigue a PS Cognition a PS Bladder/Bowel a PS Sensory a PS Spasticity a
Mean score
Standard deviation
Range
Median (IQR)
44.5 4.9 52.8 22.4 194,940 14.2 5.2 8.6 4.4 7.5 3.0 1.6 1.4 2.8 1.9 2.0 1.7 1.6
24.4 2.0 16.8 8.0 107,822 6.7 3.4 5.1 3.6 5.1 2.0 1.3 1.3 1.2 1.2 1.5 1.2 1.3
0–80 1–8 22–92 9–43 39,993–691,233 0–35 0–17 0–20 0–15 0–20 0–7 0–7 0–7 0–5 0–5 0–7 0–5 0–5
48.3 (40.8) 5.5 (3) 51 (26) 21 (12) 177,165 (126,229) 14 (9) 5 (8) 9 (8) 4 (4) 7 (7) 3 (3) 1 (1) 1 (1) 3 (2) 2 (2) 2 (2) 1 (2) 1 (2)
Note: IQR = Inter-quartile range; MSWS-12 = Multiple Sclerosis Walking Scale-12; EDSS = Expanded Disability Status Scale; MSIS-29 = Multiple Sclerosis Impact Scale; MS-SR = Multiple Sclerosis Related Symptom Checklist; PS = Performance Scales. a Lower scores are better. b Higher scores are better.
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73 Table 2 Correlations among MSWS-12 scores with scores from other measures in the entire sample and in sub-samples based on major ranges of EDSS scores Variable
Entire sample EDSS scores EDSS scores (N = 133) between 1–4.5 between 5–8 (n = 52) (n = 81)
EDSS .80 (.78) MSIS-29 Psychological .36 (.36) MSIS-29 Physical .77 (.78) Accelerometer − .64 (−.68) MS-SR Motor .67 (.66) MS-SR Brainstem .27 (.23) MS-SR Sensory .35 (.34) MS-SR Mental/Emotional .21 (.23) MS-SR Elimination .45 (.47) PS Mobility .75 (.77) PS Hand Function .25 (.33) PS Vision .13 (.14) PS Fatigue .34 (.34) PS Cognition .07 (.06) PS Bladder/Bowel .28 (.39) PS Sensory .34 (.35) PS Spasticity .31 (.32)
.71 (.72) .56 (.58) .74 (.75) −.51 (− .57) .69 (.70) .46 (.44) .46 (.48) .32 (.35) .29 (.27) .70 (.73) .27 (.47) .15 (.29) .39 (.38) .26 (.24) .19 (.31) .42 (.42) .32 (.31)
.33 (.39) .26 (.26) .57 (.62) − .48 (− .46) .35 (.38) .09 (.07) .23 (.25) .05 (.02) .33 (.35) .41 (.47) .11 (.10) .19 (.13) .13 (.14) − .08 (− .02) − .01 (.21) .29 (.32) .18 (.13)
Note: Correlation coefficients reported as r(ρ). MSWS-12 = Multiple Sclerosis Walking Scale-12; EDSS = Expanded Disability Status Scale; MSIS-29 = Multiple Sclerosis Impact Scale; MS-SR = Multiple Sclerosis Related Symptom Checklist; PS = Performance Scales.
consistency of MSWS-12 scores and relationships between scores from the multiple measures were estimated using Cronbach's coefficient alpha and Pearson product-moment correlations (r) in SPSS, version 15 (SPSS, Chicago, IL). We further examined the relationships using Spearman correlations (ρ), which provide a non-parametric version of a Pearson correlation coefficient based on ranks of the data rather than the actual values. The correlation analyses were conducted with the entire sample and by sub-samples of individuals with EDSS scores between 1–4.5 (i.e., individuals with MS who are fully ambulatory; n = 52) and EDSS scores between 5–8 (i.e., individuals with MS who have ambulatory impairment; n = 81). Cohen's guidelines of .1, .3, and .5 were used for judging the magnitude of the correlations as small, moderate, and large, respectively [15]. We expected large correlations between MSWS-12 scores with EDSS scores and accelerometer counts as well as the physical impact subscale of the MSIS-29, Motor subscale of the MSSR, and Mobility subscale of the PS (convergent evidence of construct validity). We expected small and moderate correlations between MSWS-12 scores with the psychological impact subscale of the MSIS-29 and the remaining nonwalking related subscales of the MS-SR and PS (divergent evidence of construct validity). We performed one final analysis that involved regressing MSWS-12 scores on EDSS scores and other strong correlates of MSWS-12 scores, identified in the correlation analyses, using hierarchical multiple regression. This analysis provided an examination of EDSS scores plus a linear combination of other variables for capturing maximum variation in MSWS-12 scores.
71
3. Results 3.1. Descriptive statistics Mean scores, standard deviations, and ranges of scores for the measures included in the study are presented in Table 1. 3.2. Factorial validity The single-factor model provided an adequate fit for MSWS-12 scores (χ2 = 88.71, df = 18, p b .001, SRMR = 0.04, CFI = 0.99) and all factor loadings were statistically significant (p b .001) and strong in magnitude (mean factor loading = .92, range = .84–.97). 3.3. Internal consistency The internal consistency of scores from the MSWS-12 was acceptable (α = .97). 3.4. Bivariate correlation analysis: confirming previous validity evidence The correlations between MSWS-12 scores and scores from the EDSS and MSIS-29 for the entire sample and subsamples based on major ranges of EDSS scores are provided in Table 2. With the entire sample, there were strong correlations between MSWS-12 scores with EDSS scores (r = .80, p = .0001; ρ = .78, p = .0001) and scores from the physical impact subscale of the MSIS-29 (r = .77, p = .0001; ρ = .78, p = .0001). There was a small correlation between MSWS-12 scores with scores from the psychological impact subscale of the MSIS-29 (r = .36, p = .0001; ρ = .36, p = .0001). The pattern of correlations in the overall sample is consistent with previous research [2,3] and further confirms the existing validity evidence for the MSWS-12. The analyses within the sub-samples indicated a pattern of correlations that was similar across the two groups that differed based on major ranges of EDSS scores (i.e., stronger correlations between MSWS-12 and MSIS-29 Physical than MSIS-29 Psychological scores), but the magnitude of the
Fig. 1. Bivariate scatter plot of scores from the MSWS-12 and accelerometer counts.
72
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73
Table 3 Summary of hierarchical regression analysis for variables predicting MSWS-12 scores Step
Variable
B
EDSS
10.77
SE B β
Step 1
R2
.93
.80⁎
EDSS 2.47 1.13 Accelerometer −2.97 1.41 MSIS-29 0.55 0.15 Physical MS-SR Motor 0.90 0.32 PS Mobility 2.99 1.36
.18⁎ − .13⁎ .33⁎
Step 2
R2 Change F Change
.64 .64
133.03⁎
.84 .20
21.59⁎
.23⁎ .18⁎
Note. ⁎p b .05. B = Unstandardized coefficient. SE B = Standard error of unstandardized coefficient. Β = Standardized coefficient. MSWS-12 = Multiple Sclerosis Walking Scale-12; EDSS = Expanded Disability Status Scale; MSIS-29 = Multiple Sclerosis Impact Scale; MS-SR = Multiple Sclerosis Related Symptom Checklist; PS = Performance Scales.
correlations was consistently smaller in the sub-sample with EDSS scores between 5–8. 3.5. Bivariate correlation analysis: extending previous validity evidence The correlations between MSWS-12 scores and scores from the MS-SR and PS for the entire sample and subsamples based on major ranges of EDSS scores are provided in Table 2. Within the entire sample, there was a strong correlation between MSWS-12 scores with accelerometer counts (r = –.64, p = .0001; ρ = –.68, p = .0001), and the bivariate plot is provided in Fig. 1. Regarding the subscales of the MS-SR, there was a strong correlation between scores from the MSWS-12 and Motor subscale (r = .67, p = .0001; ρ = .66, p = .0001), whereas there were small to moderate correlations between scores from the MSWS-12 and the Brainstem (r = .27, p = .004; ρ = .23, p = .01), Sensory (r = .35, p = .0001; ρ = .34, p = .0001), Mental (r = .21, p = .02; ρ = .23, p = .01), and Elimination (r = .45, p = .0001; ρ = .47, p = .0001) subscales. Regarding the subscales of the PS, there was a strong correlation between scores from the MSWS-12 and Mobility subscale (r = .75, p = .0001; ρ = .77, p = .0001), and there were small to moderate correlations between scores from the MSWS-12 and the Hand Function (r = .25, p = .01; ρ = .33, p = .0001), Vision (r = .13, p = .15; ρ = .14, p = .12), Fatigue (r = .34, p = .0001; ρ = .34, p = .0001), Cognition (r = .07, p = .44; ρ = 06, p = .52), Bladder/Bowel (r = .28, p = .008; ρ = .39, p = .0001), Sensory (r = .34, p = .0001; ρ = .35, p = .0001), and Spasticity (r = .31, p = .001; ρ = .32, p = .001) subscales. Within the sub-samples, the analyses indicated a pattern of correlations that was similar across the two groups that differed based on major ranges of EDSS scores (i.e., stronger correlations between MSWS-12 and MS-SR Motor and PS Mobility than the remaining MS-SR and PS Subscales, respectively), but, again, the magnitude of the correlations was consistently smaller in the sub-sample with EDSS scores between 5–8.
3.6. Multiple regression analysis The results of the multiple regression analysis are provided in Table 3. EDSS scores explained 64% of variation in MSWS-12 scores in Step 1. EDSS, accelerometer, MSIS-29 Physical, MS-SR Motor, and PS Mobility scores were all significant associated with MSWS-12 scores in Step 2. Those variables explained 84% of variation in MSWS-12 scores; the linear combination of five variables accounted for 20% more variation in MSWS-12 scores than EDSS scores alone. 4. Discussion Validation of scores from a measure is an ongoing and evolving process [5]. Therefore, we conducted analyses that both confirmed and extended the validity of inferences from MSWS-12 scores in a community-based sample of individuals with MS. The confirmatory factor analysis provided novel evidence that a uni-dimensional or single-factor model provided an adequate fit for MSWS-12 scores. This indicates that scores from MSWS-12 items can be summed into a composite or overall measure of walking ability in persons with MS. Our initial correlation analyses confirmed the previously reported pattern of correlations between MSWS-12 scores and scores from the EDSS and subscales of the MSIS-29. For example, we reported a correlation of r = .80 between EDSS and MSWS-12 scores and other studies have reported correlation coefficients ranging between r = .65 [2] and r = .84 [3]. We reported correlation coefficients of r = .77 and r = .36 between the MSIS-12 and physical and psychological subscales of the MSIS-29, respectively, and this too is consistent with previous research [2,3]. One novel feature of our correlation analyses was the observation of a similar pattern of relationships between MSWS-12 and MSIS-29 scores in sub-samples generated based on major ranges of EDSS scores. Therefore, such evidence confirms and extends the previously reported convergent and divergent validity of MSWS-12 scores as a measure of walking in individuals with MS. We subsequently extended previous research by first examining the correlation between MSWS-12 scores with movement counts from an objective measure of walking afforded by accelerometry. There was a strong correlation between MSWS-12 scores with accelerometer counts of r = −.64 such that individuals who were more ambulatory based on accumulation of daily activity counts reported less impact of MS on walking based on lower MSWS-12 scores; this relationship was similar in magnitude within the sub-samples that differed by major ranges of EDSS scores. Importantly, accelerometers have been considered as a “gold standard” measure of ambulation in individuals with neurological diseases, [1] and therefore accelerometers serve as an important outcome variable for extending the validity of MSWS-12 scores. Although the
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73
correlation between MSWS-12 scores and accelerometer counts was weaker than the correlation between MSWS-12 and EDSS scores, the EDSS likely correlates better with MSWS-12 scores because it taps mobility and sensory, cognitive, mood, and fatigue factors compared with the motor focused assessment by the accelerometer. The second method of extending previous validity evidence involved studying the pattern of correlations between scores from walking and non-walking related subscales of the MS-RS and PS with MSWS-12 scores; such analyses represent an additional means of providing convergent and divergent evidence of construct validity of MSWS-12 scores. There were strong relationships between MSWS-12 scores with the Motor subscale of the MS-SR and the Mobility subscale of the PS such that those with lower MSWS-12 scores reported less frequent motor symptoms and less disability associated with mobility impairments. By comparison, there were small to moderate associations between MSWS-12 scores with non-walking subscales of the MS-SR and PS. The pattern of correlations was similar across the two groups that differed based on major ranges of EDSS scores, although the magnitude of the correlations was consistently smaller in the sub-sample with higher EDSS scores. The differential pattern of associations between MSWS-12 scores with walking and non-walking related subscales of the MS-SR and PS in the overall sample and sub-samples that differed based on major ranges of EDSS scores provides new convergent and divergent evidence of construct validity of MSWS-12 scores. Scores from several variables (e.g., EDSS, MSIS-29 Physical, Accelerometer, MS-SR Motor, and PS Mobility) individually had large correlations with MSWS-12 scores. Those variables individually accounted for between 41 and 64% of the variation in MSWS-12 scores. Within our multiple regression analysis, we were able to account for 84% of variation in MSWS-12 scores based on a linear combination of EDSS, MSIS-29 Physical, Accelerometer, MS-SR Motor, and PS Mobility scores. Hence, the EDSS together with other ambulatory and non-ambulatory measures are capable of explaining more variation in MSWS-12 scores than any single variable such as the EDSS. This study is not without limitations. One limitation is the demographic composition of our sample. Indeed, our sample consisted mostly of Caucasian women, and, although this is consistent with the demographic features of communityresiding samples used for previous validation of the MSWS12, [2,3] it highlights the importance of establishing validity evidence in more diverse samples of individuals with MS. Another limitation is that we did not further establish evidence of responsiveness based on drug (e.g., IV steroid treatment) or rehabilitation (e.g., physical therapy) intervention therapies, and the use of the MSWS-12 as an outcome
73
in rehabilitation interventions is an important line of future research. One final limitation is that we did not collect 25-ft Timed Walk Test data for determining the comparative value of collecting accelerometer data for further understanding MSWS-12 scores and walking ability in MS. Despite those limitations, we provide evidence that both confirms and extends the validity of inferences from scores of the MSWS-12 as a measure of the impact of MS on walking in a community-based sample of individuals with MS. Perhaps future researchers should examine the possible utility of including the MSWS-12 in a composite measure such as the Multiple Sclerosis Functional Composite, particularly given that MSWS-12 scores have been more responsive to IV steroid treatment than the 25-ft Timed Walk Test [2]. References [1] Pearson OR, Busse ME, van Deursen RW, Wiles CM. Quantification of walking mobility in neurological disorders. QJM 2004;97:463–75. [2] Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ. Measuring the impact of MS on walking ability: the 12-item MS Walking Scale (MSWS-12). Neurology 2003;60:31–6. [3] McGuigan C, Hutchinson M. Confirming the validity and responsiveness of the Multiple Sclerosis Walking Scale-12 (MSWS-12). Neurology 2004;62:2103–5. [4] Cronbach LJ, Meehl PE. Construct validation in psychological tests. Psychol Bull 1955;52:281–302. [5] Messick S. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am Psychol 1995;50:741–9. [6] Hobart JC, Lamping DL, Fitzpatrick R, Riazi A, Thompson AJ. The Multiple Sclerosis Impact Scale (MSIS-29): a new patient-based outcome measure. Brain 2001;124:962–73. [7] Gulick EE. Model confirmation of the MS-Related Symptom Checklist. Nurs Res 1989;38:147–53. [8] Schwartz CE, Vollmer T, Lee H. Reliability and validity of two selfreport measures of impairment and disability for MS. Neurology 1999;52:63–70. [9] Goodin DS. A questionnaire to assess neurological impairment in multiple sclerosis. Mult Scler 1998;4:444–51. [10] Riazi A, Hobart JC, Lamping DL, Fitzpatrick R, Thompson A. Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry 2002;73: 701–4. [11] Gosney JL, Scott JA, Snook EM, Motl RW. Physical activity and multiple sclerosis: Validity of self-report and objective measures. Fam Commun Health 2007;30:144–50. [12] Motl RW, McAuley E, Snook EM, Scott JA. Validity of physical activity measures in ambulatory individuals with multiple sclerosis. Disabil Rehabil 2006;28:1151–6. [13] Muthén L.K, Muthén BO. Mplus. Los Angeles: Author. 1998–2004. [14] Hu L, Bentler PM. Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 1999;6:1–55. [15] Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
Confirmation and extension of the validity of the Multiple Sclerosis Walking Scale-12 (MSWS-12) Robert W. Motl ⁎, Erin M. Snook Department of Kinesiology and Community Health, University of Illinois at Urbana–Champaign, United States Received 27 July 2007; received in revised form 31 October 2007; accepted 2 November 2007 Available online 3 December 2007
Abstract Objective: The Multiple Sclerosis Walking Scale-12 (MSWS-12) is a 12-item patient-rated measure of the impact of MS on walking. As validation of scores from a measure is an ongoing and evolving process, the provision of additional evidence is warranted that confirms and extends the validity of inferences from scores on the MSWS-12. Method: Participants (N = 133) were recruited through support group meetings of the Greater Illinois and Indiana State Chapters of the National Multiple Sclerosis Society, and wore an accelerometer for a seven-day period and completed the MSWS-12 and other outcome measures. Results: Confirmatory factor analysis indicated that a single-factor model provided an adequate fit for MSWS-12 scores. MSWS-12 scores demonstrated strong evidence of internal consistency. The correlations between MSWS-12 scores with scores from other scales, including an accelerometer, were consistent with our a priori hypotheses. Conclusion: We provide evidence that both confirms and extends the validity of inferences from scores of the MSWS-12 as a measure of the impact of MS on walking in a community-based sample of individuals with MS. © 2007 Elsevier B.V. All rights reserved. Keywords: Multiple sclerosis; Validity; Walking
1. Introduction Walking ability is often used by researchers and clinicians as a measure of disease progression in individuals with MS. There are multiple measures of walking that have been included in clinical trials and practice, such as Timed Walk Tests (e.g., 25-ft or 6-min walk tests), quantitative movement analysis (i.e., gait kinematics), Ambulation Index, and the Expanded Disability Status Scale (EDSS) [1]. Importantly, all of those measures have limitations as assessments of walking ability, [1] thereby serving as the impetus for developing the ⁎ Corresponding author. University of Illinois at Urbana–Champaign, Department of Kinesiology and Community Health, 350 Freer Hall, 906 South Goodwin Ave., Urbana, IL, 61801, United States. Tel.: +1 217 265 0886; fax: +1 217 244 7322. E-mail address: [email protected] (R.W. Motl). 0022-510X/$ - see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.jns.2007.11.003
Multiple Sclerosis Walking Scale-12 (MSWS-12) [2]. The MSWS-12 is a 12-item patient-rated measure of the impact of MS on walking that was developed using standard methods of test construction and then validated in community and hospital-residing samples of individuals with MS [2]. The initial evidence for the validity of MSWS-12 scores was subsequently confirmed in community-residing and hospital outpatient samples of individuals with MS [3]. As validation of scores from a measure is an ongoing and evolving process, [4,5] the provision of additional evidence is warranted that further establishes the validity of inferences from scores on the MSWS-12. Therefore, we conducted analyses that both confirmed and extended the validity of inferences from the MSWS-12 in a community-based sample of individuals with MS. We initially tested the factorial or structural validity of a uni-dimensional model for MSWS-12 scores and then confirmed the previously reported pattern of correlations
70
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73
between scores from MSWS-12 with the EDSS and subscales of the Multiple Sclerosis Impact Scale (MSIS-29) [6]. We then extended previous research by first examining the correlation between scores from the MSWS-12 with an objective measure of walking provided by accelerometry and second examining the pattern of correlations between MSWS-12 scores with walking and non-walking related subscales of the Multiple Sclerosis Related Symptom Checklist (MS-RS) [7] and Performance Scales (PS) [8]. 2. Methods
(ActiGraph single-axis accelerometer, model 7164 version, Health One Technology, Fort Walton Beach, FL) was included as an objective measure of ambulation [1] that provided a summary measure of movement counts accumulated per day across a seven-day period [11,12]. The MS-SR was included as a measure of the frequency of occurrence of 26 symptoms during the previous four weeks, and the items yielded five subscales of Motor, Brainstem, Sensory, Mental/ Emotional, and Elimination symptoms [7]. The PS consisted of 8 items that reflected subscales of Mobility, Hand Function, Vision, Fatigue, Cognition, Bladder/Bowel, Sensory, and Spasticity [8].
2.1. Participants 2.3. Procedure Participants (N = 133) were recruited through support group meetings of the Greater Illinois and Indiana State Chapters of the National Multiple Sclerosis Society. The mean age of the sample was 51.1 years (SD = 11.1) and the sample was primarily female (n = 104, 78%), Caucasian (n = 123, 93%), employed (n = 71, 53%), married (n = 91, 68%) and well educated (some college education n = 36, 27%; college graduate n = 61, 46%). The mean duration of time since MS diagnosis was 12.0 years (SD = 9.0). The sample consisted of 85 individuals with relapsing–remitting MS, 52 individuals with secondary progressive MS, and 6 individual with primary progressive MS. 2.2. Measures Walking ability was measured using the MSWS-12 [2]. We included a self-reported version of the EDSS [9] and the MSIS-29 as a patient-rated measure of the physical and psychological impact of MS [6,10]. The accelerometer
The procedure was approved by the institutional review board at the University of Illinois at Urbana-Champaign and all participants provided written informed consent. Participants completed a battery of questionnaires that included the MSWS-12, EDSS, MSIS-29, MS-SR, and PS and wore an accelerometer for 7 consecutive days. Participants received $10 remuneration for completing the study. 2.4. Data analysis Descriptive data are presented as mean ± standard deviation. We tested the factorial validity of a uni-dimensional or single-factor model for MSWS-12 scores using confirmatory factor analysis with the robust weighted least squares estimator and polychoric correlations in Mplus version 3.0 [13]. The fit of the model was based on combinatory rules of standardized root mean squared residual (SRMS) ≤ .08 and comparative fit index (CFI) ≥ .95 [14]. The internal
Table 1 Descriptive statistics for the measures in the sample of 133 individuals with multiple sclerosis Variable a
MSWS-12 EDSS a MSIS-29 Psychological a MSIS-29 Physical a Accelerometer b MS-SR Motor a MS-SR Brainstem a MS-SR Sensory a MS-SR Mental/Emotional a MS-SR Elimination a PS Mobility a PS Hand Function a PS Vision a PS Fatigue a PS Cognition a PS Bladder/Bowel a PS Sensory a PS Spasticity a
Mean score
Standard deviation
Range
Median (IQR)
44.5 4.9 52.8 22.4 194,940 14.2 5.2 8.6 4.4 7.5 3.0 1.6 1.4 2.8 1.9 2.0 1.7 1.6
24.4 2.0 16.8 8.0 107,822 6.7 3.4 5.1 3.6 5.1 2.0 1.3 1.3 1.2 1.2 1.5 1.2 1.3
0–80 1–8 22–92 9–43 39,993–691,233 0–35 0–17 0–20 0–15 0–20 0–7 0–7 0–7 0–5 0–5 0–7 0–5 0–5
48.3 (40.8) 5.5 (3) 51 (26) 21 (12) 177,165 (126,229) 14 (9) 5 (8) 9 (8) 4 (4) 7 (7) 3 (3) 1 (1) 1 (1) 3 (2) 2 (2) 2 (2) 1 (2) 1 (2)
Note: IQR = Inter-quartile range; MSWS-12 = Multiple Sclerosis Walking Scale-12; EDSS = Expanded Disability Status Scale; MSIS-29 = Multiple Sclerosis Impact Scale; MS-SR = Multiple Sclerosis Related Symptom Checklist; PS = Performance Scales. a Lower scores are better. b Higher scores are better.
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73 Table 2 Correlations among MSWS-12 scores with scores from other measures in the entire sample and in sub-samples based on major ranges of EDSS scores Variable
Entire sample EDSS scores EDSS scores (N = 133) between 1–4.5 between 5–8 (n = 52) (n = 81)
EDSS .80 (.78) MSIS-29 Psychological .36 (.36) MSIS-29 Physical .77 (.78) Accelerometer − .64 (−.68) MS-SR Motor .67 (.66) MS-SR Brainstem .27 (.23) MS-SR Sensory .35 (.34) MS-SR Mental/Emotional .21 (.23) MS-SR Elimination .45 (.47) PS Mobility .75 (.77) PS Hand Function .25 (.33) PS Vision .13 (.14) PS Fatigue .34 (.34) PS Cognition .07 (.06) PS Bladder/Bowel .28 (.39) PS Sensory .34 (.35) PS Spasticity .31 (.32)
.71 (.72) .56 (.58) .74 (.75) −.51 (− .57) .69 (.70) .46 (.44) .46 (.48) .32 (.35) .29 (.27) .70 (.73) .27 (.47) .15 (.29) .39 (.38) .26 (.24) .19 (.31) .42 (.42) .32 (.31)
.33 (.39) .26 (.26) .57 (.62) − .48 (− .46) .35 (.38) .09 (.07) .23 (.25) .05 (.02) .33 (.35) .41 (.47) .11 (.10) .19 (.13) .13 (.14) − .08 (− .02) − .01 (.21) .29 (.32) .18 (.13)
Note: Correlation coefficients reported as r(ρ). MSWS-12 = Multiple Sclerosis Walking Scale-12; EDSS = Expanded Disability Status Scale; MSIS-29 = Multiple Sclerosis Impact Scale; MS-SR = Multiple Sclerosis Related Symptom Checklist; PS = Performance Scales.
consistency of MSWS-12 scores and relationships between scores from the multiple measures were estimated using Cronbach's coefficient alpha and Pearson product-moment correlations (r) in SPSS, version 15 (SPSS, Chicago, IL). We further examined the relationships using Spearman correlations (ρ), which provide a non-parametric version of a Pearson correlation coefficient based on ranks of the data rather than the actual values. The correlation analyses were conducted with the entire sample and by sub-samples of individuals with EDSS scores between 1–4.5 (i.e., individuals with MS who are fully ambulatory; n = 52) and EDSS scores between 5–8 (i.e., individuals with MS who have ambulatory impairment; n = 81). Cohen's guidelines of .1, .3, and .5 were used for judging the magnitude of the correlations as small, moderate, and large, respectively [15]. We expected large correlations between MSWS-12 scores with EDSS scores and accelerometer counts as well as the physical impact subscale of the MSIS-29, Motor subscale of the MSSR, and Mobility subscale of the PS (convergent evidence of construct validity). We expected small and moderate correlations between MSWS-12 scores with the psychological impact subscale of the MSIS-29 and the remaining nonwalking related subscales of the MS-SR and PS (divergent evidence of construct validity). We performed one final analysis that involved regressing MSWS-12 scores on EDSS scores and other strong correlates of MSWS-12 scores, identified in the correlation analyses, using hierarchical multiple regression. This analysis provided an examination of EDSS scores plus a linear combination of other variables for capturing maximum variation in MSWS-12 scores.
71
3. Results 3.1. Descriptive statistics Mean scores, standard deviations, and ranges of scores for the measures included in the study are presented in Table 1. 3.2. Factorial validity The single-factor model provided an adequate fit for MSWS-12 scores (χ2 = 88.71, df = 18, p b .001, SRMR = 0.04, CFI = 0.99) and all factor loadings were statistically significant (p b .001) and strong in magnitude (mean factor loading = .92, range = .84–.97). 3.3. Internal consistency The internal consistency of scores from the MSWS-12 was acceptable (α = .97). 3.4. Bivariate correlation analysis: confirming previous validity evidence The correlations between MSWS-12 scores and scores from the EDSS and MSIS-29 for the entire sample and subsamples based on major ranges of EDSS scores are provided in Table 2. With the entire sample, there were strong correlations between MSWS-12 scores with EDSS scores (r = .80, p = .0001; ρ = .78, p = .0001) and scores from the physical impact subscale of the MSIS-29 (r = .77, p = .0001; ρ = .78, p = .0001). There was a small correlation between MSWS-12 scores with scores from the psychological impact subscale of the MSIS-29 (r = .36, p = .0001; ρ = .36, p = .0001). The pattern of correlations in the overall sample is consistent with previous research [2,3] and further confirms the existing validity evidence for the MSWS-12. The analyses within the sub-samples indicated a pattern of correlations that was similar across the two groups that differed based on major ranges of EDSS scores (i.e., stronger correlations between MSWS-12 and MSIS-29 Physical than MSIS-29 Psychological scores), but the magnitude of the
Fig. 1. Bivariate scatter plot of scores from the MSWS-12 and accelerometer counts.
72
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73
Table 3 Summary of hierarchical regression analysis for variables predicting MSWS-12 scores Step
Variable
B
EDSS
10.77
SE B β
Step 1
R2
.93
.80⁎
EDSS 2.47 1.13 Accelerometer −2.97 1.41 MSIS-29 0.55 0.15 Physical MS-SR Motor 0.90 0.32 PS Mobility 2.99 1.36
.18⁎ − .13⁎ .33⁎
Step 2
R2 Change F Change
.64 .64
133.03⁎
.84 .20
21.59⁎
.23⁎ .18⁎
Note. ⁎p b .05. B = Unstandardized coefficient. SE B = Standard error of unstandardized coefficient. Β = Standardized coefficient. MSWS-12 = Multiple Sclerosis Walking Scale-12; EDSS = Expanded Disability Status Scale; MSIS-29 = Multiple Sclerosis Impact Scale; MS-SR = Multiple Sclerosis Related Symptom Checklist; PS = Performance Scales.
correlations was consistently smaller in the sub-sample with EDSS scores between 5–8. 3.5. Bivariate correlation analysis: extending previous validity evidence The correlations between MSWS-12 scores and scores from the MS-SR and PS for the entire sample and subsamples based on major ranges of EDSS scores are provided in Table 2. Within the entire sample, there was a strong correlation between MSWS-12 scores with accelerometer counts (r = –.64, p = .0001; ρ = –.68, p = .0001), and the bivariate plot is provided in Fig. 1. Regarding the subscales of the MS-SR, there was a strong correlation between scores from the MSWS-12 and Motor subscale (r = .67, p = .0001; ρ = .66, p = .0001), whereas there were small to moderate correlations between scores from the MSWS-12 and the Brainstem (r = .27, p = .004; ρ = .23, p = .01), Sensory (r = .35, p = .0001; ρ = .34, p = .0001), Mental (r = .21, p = .02; ρ = .23, p = .01), and Elimination (r = .45, p = .0001; ρ = .47, p = .0001) subscales. Regarding the subscales of the PS, there was a strong correlation between scores from the MSWS-12 and Mobility subscale (r = .75, p = .0001; ρ = .77, p = .0001), and there were small to moderate correlations between scores from the MSWS-12 and the Hand Function (r = .25, p = .01; ρ = .33, p = .0001), Vision (r = .13, p = .15; ρ = .14, p = .12), Fatigue (r = .34, p = .0001; ρ = .34, p = .0001), Cognition (r = .07, p = .44; ρ = 06, p = .52), Bladder/Bowel (r = .28, p = .008; ρ = .39, p = .0001), Sensory (r = .34, p = .0001; ρ = .35, p = .0001), and Spasticity (r = .31, p = .001; ρ = .32, p = .001) subscales. Within the sub-samples, the analyses indicated a pattern of correlations that was similar across the two groups that differed based on major ranges of EDSS scores (i.e., stronger correlations between MSWS-12 and MS-SR Motor and PS Mobility than the remaining MS-SR and PS Subscales, respectively), but, again, the magnitude of the correlations was consistently smaller in the sub-sample with EDSS scores between 5–8.
3.6. Multiple regression analysis The results of the multiple regression analysis are provided in Table 3. EDSS scores explained 64% of variation in MSWS-12 scores in Step 1. EDSS, accelerometer, MSIS-29 Physical, MS-SR Motor, and PS Mobility scores were all significant associated with MSWS-12 scores in Step 2. Those variables explained 84% of variation in MSWS-12 scores; the linear combination of five variables accounted for 20% more variation in MSWS-12 scores than EDSS scores alone. 4. Discussion Validation of scores from a measure is an ongoing and evolving process [5]. Therefore, we conducted analyses that both confirmed and extended the validity of inferences from MSWS-12 scores in a community-based sample of individuals with MS. The confirmatory factor analysis provided novel evidence that a uni-dimensional or single-factor model provided an adequate fit for MSWS-12 scores. This indicates that scores from MSWS-12 items can be summed into a composite or overall measure of walking ability in persons with MS. Our initial correlation analyses confirmed the previously reported pattern of correlations between MSWS-12 scores and scores from the EDSS and subscales of the MSIS-29. For example, we reported a correlation of r = .80 between EDSS and MSWS-12 scores and other studies have reported correlation coefficients ranging between r = .65 [2] and r = .84 [3]. We reported correlation coefficients of r = .77 and r = .36 between the MSIS-12 and physical and psychological subscales of the MSIS-29, respectively, and this too is consistent with previous research [2,3]. One novel feature of our correlation analyses was the observation of a similar pattern of relationships between MSWS-12 and MSIS-29 scores in sub-samples generated based on major ranges of EDSS scores. Therefore, such evidence confirms and extends the previously reported convergent and divergent validity of MSWS-12 scores as a measure of walking in individuals with MS. We subsequently extended previous research by first examining the correlation between MSWS-12 scores with movement counts from an objective measure of walking afforded by accelerometry. There was a strong correlation between MSWS-12 scores with accelerometer counts of r = −.64 such that individuals who were more ambulatory based on accumulation of daily activity counts reported less impact of MS on walking based on lower MSWS-12 scores; this relationship was similar in magnitude within the sub-samples that differed by major ranges of EDSS scores. Importantly, accelerometers have been considered as a “gold standard” measure of ambulation in individuals with neurological diseases, [1] and therefore accelerometers serve as an important outcome variable for extending the validity of MSWS-12 scores. Although the
R.W. Motl, E.M. Snook / Journal of the Neurological Sciences 268 (2008) 69–73
correlation between MSWS-12 scores and accelerometer counts was weaker than the correlation between MSWS-12 and EDSS scores, the EDSS likely correlates better with MSWS-12 scores because it taps mobility and sensory, cognitive, mood, and fatigue factors compared with the motor focused assessment by the accelerometer. The second method of extending previous validity evidence involved studying the pattern of correlations between scores from walking and non-walking related subscales of the MS-RS and PS with MSWS-12 scores; such analyses represent an additional means of providing convergent and divergent evidence of construct validity of MSWS-12 scores. There were strong relationships between MSWS-12 scores with the Motor subscale of the MS-SR and the Mobility subscale of the PS such that those with lower MSWS-12 scores reported less frequent motor symptoms and less disability associated with mobility impairments. By comparison, there were small to moderate associations between MSWS-12 scores with non-walking subscales of the MS-SR and PS. The pattern of correlations was similar across the two groups that differed based on major ranges of EDSS scores, although the magnitude of the correlations was consistently smaller in the sub-sample with higher EDSS scores. The differential pattern of associations between MSWS-12 scores with walking and non-walking related subscales of the MS-SR and PS in the overall sample and sub-samples that differed based on major ranges of EDSS scores provides new convergent and divergent evidence of construct validity of MSWS-12 scores. Scores from several variables (e.g., EDSS, MSIS-29 Physical, Accelerometer, MS-SR Motor, and PS Mobility) individually had large correlations with MSWS-12 scores. Those variables individually accounted for between 41 and 64% of the variation in MSWS-12 scores. Within our multiple regression analysis, we were able to account for 84% of variation in MSWS-12 scores based on a linear combination of EDSS, MSIS-29 Physical, Accelerometer, MS-SR Motor, and PS Mobility scores. Hence, the EDSS together with other ambulatory and non-ambulatory measures are capable of explaining more variation in MSWS-12 scores than any single variable such as the EDSS. This study is not without limitations. One limitation is the demographic composition of our sample. Indeed, our sample consisted mostly of Caucasian women, and, although this is consistent with the demographic features of communityresiding samples used for previous validation of the MSWS12, [2,3] it highlights the importance of establishing validity evidence in more diverse samples of individuals with MS. Another limitation is that we did not further establish evidence of responsiveness based on drug (e.g., IV steroid treatment) or rehabilitation (e.g., physical therapy) intervention therapies, and the use of the MSWS-12 as an outcome
73
in rehabilitation interventions is an important line of future research. One final limitation is that we did not collect 25-ft Timed Walk Test data for determining the comparative value of collecting accelerometer data for further understanding MSWS-12 scores and walking ability in MS. Despite those limitations, we provide evidence that both confirms and extends the validity of inferences from scores of the MSWS-12 as a measure of the impact of MS on walking in a community-based sample of individuals with MS. Perhaps future researchers should examine the possible utility of including the MSWS-12 in a composite measure such as the Multiple Sclerosis Functional Composite, particularly given that MSWS-12 scores have been more responsive to IV steroid treatment than the 25-ft Timed Walk Test [2]. References [1] Pearson OR, Busse ME, van Deursen RW, Wiles CM. Quantification of walking mobility in neurological disorders. QJM 2004;97:463–75. [2] Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ. Measuring the impact of MS on walking ability: the 12-item MS Walking Scale (MSWS-12). Neurology 2003;60:31–6. [3] McGuigan C, Hutchinson M. Confirming the validity and responsiveness of the Multiple Sclerosis Walking Scale-12 (MSWS-12). Neurology 2004;62:2103–5. [4] Cronbach LJ, Meehl PE. Construct validation in psychological tests. Psychol Bull 1955;52:281–302. [5] Messick S. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am Psychol 1995;50:741–9. [6] Hobart JC, Lamping DL, Fitzpatrick R, Riazi A, Thompson AJ. The Multiple Sclerosis Impact Scale (MSIS-29): a new patient-based outcome measure. Brain 2001;124:962–73. [7] Gulick EE. Model confirmation of the MS-Related Symptom Checklist. Nurs Res 1989;38:147–53. [8] Schwartz CE, Vollmer T, Lee H. Reliability and validity of two selfreport measures of impairment and disability for MS. Neurology 1999;52:63–70. [9] Goodin DS. A questionnaire to assess neurological impairment in multiple sclerosis. Mult Scler 1998;4:444–51. [10] Riazi A, Hobart JC, Lamping DL, Fitzpatrick R, Thompson A. Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples. J Neurol Neurosurg Psychiatry 2002;73: 701–4. [11] Gosney JL, Scott JA, Snook EM, Motl RW. Physical activity and multiple sclerosis: Validity of self-report and objective measures. Fam Commun Health 2007;30:144–50. [12] Motl RW, McAuley E, Snook EM, Scott JA. Validity of physical activity measures in ambulatory individuals with multiple sclerosis. Disabil Rehabil 2006;28:1151–6. [13] Muthén L.K, Muthén BO. Mplus. Los Angeles: Author. 1998–2004. [14] Hu L, Bentler PM. Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 1999;6:1–55. [15] Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.