Responsiveness and Predictive Validity of the Participation Measure–3 Domains, 4 Dimensions in Survivors of Stroke

Archives of Physical Medicine and Rehabilitation journal homepage: www.archives-pmr.org Archives of Physical Medicine and Rehabilitation 2019;-:-------

ORIGINAL RESEARCH

Responsiveness and Predictive Validity of the Participation Measuree3 Domains, 4 Dimensions in Survivors of Stroke Feng-Hang Chang, ScD, OTR/L,a,b Pengsheng Ni, MD, MPHc From the aGraduate Institute of Injury Prevention and Control, College of Public Health, Taipei Medical University, Taipei, Taiwan; b Department of Physical Medicine and Rehabilitation, School of Medicine, Taipei Medical University, Taipei, Taiwan; and cDepartment of Health Law, Policy & Management, Boston University School of Public Health, Boston, Massachusetts.

Abstract Objectives: To examine the responsiveness and predictive validity of the Participation Measuree3 Domains, 4 Dimensions (PM-3D4D) in people receiving outpatient rehabilitation following stroke. Design: Prospective cohort observational study. Setting: Outpatient rehabilitation settings. Participants: Volunteer patients (NZ269) with stroke (mean age
SD [y], 55.36
12.46; 70.26% male). Interventions: Not applicable. Main Outcome Measures: The PM-3D4D was designed to measure 3 domains (Productivity, Social, and Community) and 4 dimensions (Diversity, Frequency, Desire for change, and Difficulty) of participation in individuals with rehabilitation needs. All participants completed the PM-3D4D, the Participation Assessment with Recombined Tools-Objective (PART-O), the Participation Measure for Post-Acute Care (PM-PAC), and the EuroQol-5-Dimension (EQ-5D) at the baseline assessment and again following 3 months of outpatient rehabilitation. Results: Significant mean changes in scores were observed for most of the PM-3D4D subscales, with the largest score change observed in the Difficulty subscale (standardized response meanZ0.57w0.88). The minimal detectable change and meaningful clinically important differences were calculated for each subscale. The Frequency and Difficulty dimensions of the PM-3D4D demonstrated significantly greater responsiveness than the PART-O and PM-PAC, respectively. The baseline PM-3D4D scores, except for Desire for change subscales, were significantly correlated with the PART-O, PM-PAC, and EQ-5D scores after 3 months of rehabilitation. Conclusions: This study provides evidence supporting the responsiveness and predictive validity of the PM-3D4D in survivors of stroke. Among all subscales of the PM-3D4D, the Difficulty dimensional scale demonstrated the greatest responsiveness. The Desire for change dimension of the PM-3D4D showed less responsiveness, and we recommend that it be used as a goal-setting tool rather than an outcome measure. The PM-3D4D can potentially be used to predict participation outcomes and the health-related quality of life following rehabilitation interventions. Archives of Physical Medicine and Rehabilitation 2019;-:------ª 2019 by the American Congress of Rehabilitation Medicine

Stroke is the leading cause of disability globally.1 Following the onset of stroke, survivors often experience a wide range of restrictions including participating in different community activities. Many of these survivors have to deal with issues such as loss of Supported by the Ministry of Science and Technology, Taiwan (grant no. MOST105-2628-B038-003-MY3), the Ministry of Health and Welfare, Taiwan (grant no. MOHW105-TDU-B-212133018), and National Health Research Institutes, Taiwan (grant no. NHRI-EX108-10819PC). The funding sources had no influence on the study design or findings. Disclosures: none.

employment, marital breakup, and failure to manage family care responsibilities, as well as numerous other life challenges.2 Participation in society is therefore an urgent goal of rehabilitation not only for the survivors of stroke themselves, but also for their family and society.3 However, measuring participation has always been a great challenge.4 In the previous literature, numerous limitations of existing participation measures were identified, including a lack of a clear and consistent operational definition of participation

0003-9993/19/$36 - see front matter ª 2019 by the American Congress of Rehabilitation Medicine https://doi.org/10.1016/j.apmr.2019.06.018

2

F.-H. Chang, P. Ni

and no regard for the multidimensional nature of participation.5 Most widely used participation measures only address 1 dimension of participation. For example, the Participation Assessment with Recombined Tools-Objective (PART-O)6 only assesses the frequency of participation, while measures such as the Participation Measure for Post-Acute Care (PM-PAC)7 only assess a person’s perceived participation restrictions. These tools may reflect 1 part of participation but are unlikely to provide a complete picture of the complex construct of participation and are unable to detect improvements in different domains of a person’s participation.8 In response to these limitations, the Participation Measuree3 Domains, 4 Dimensions (PM-3D4D) was developed.9 The PM3D4D was designed to measure 3 domains (Productivity, Social, and Community) and 4 dimensions (Diversity, Frequency, Desire for change, and Difficulty) of participation in individuals with rehabilitation needs.9 The measure showed excellent construct validity, multidimensionality, item-fit, test-retest reliability, and concordance between different administration modes.9-11 The Frequency and Difficulty scales of the PM3D4D, respectively, demonstrated moderate-to-strong correlations with the PART-O and PM-PAC.11 However, like many existing participation measures,12 some critical psychometric properties, including the responsiveness and predictive validity of the PM-3D4D, have not been examined. Responsiveness refers to the ability of an instrument to detect a meaningful change over time and is an essential property for instruments intended to be used to assess changes over time.13,14 Predictive validity refers to the ability of an instrument to predict future outcomes.15 With no evidence on the responsiveness or predictive validity, we cannot know whether the PM-3D4D can be used to evaluate the effectiveness of interventions in clinical settings or whether it can be used to predict individuals’ future functioning. The purpose of this study was to examine the responsiveness and predictive validity of the PM-3D4D in people receiving outpatient rehabilitation after stroke. We hypothesized that the PM-3D4D would demonstrate significant and similar levels of responsiveness with the existing participation measures of the PART-O and PM-PAC, and would have predictive validity for predicting participation and health-related quality of life (HRQoL) outcomes in people receiving outpatient rehabilitation after stroke.

List of abbreviations: CI EQ-5D HRQoL MDC MDC90 MCID PART-O PM-3D4D PM-PAC SRM USER-Participation

confidence interval EuroQol-5-Dimension health-related quality of life minimal detectable change minimal detectable change with a confidence level of 90% meaningful clinically important difference Participation Assessment with Recombined Tools-Objective Participation Measuree3 Domains, 4 Dimensions Participation Measure for Post-Acute Care standardized response mean Utrecht Scale for Evaluation of Rehabilitation Participation

Methods Participants This study is part of a cohort study which aimed to monitor functional changes in patients receiving rehabilitation in 2016-2018. A sample of volunteer participants who were admitted to the outpatient rehabilitation program of 1 of 4 hospitals in the greater Taipei area was recruited for the study. Inclusion criteria for the current study included (1) being 20 years of age; (2) being able to understand Mandarin; (3) having a primary diagnosis of stroke; (4) being able to provide informed consent; (5) scoring 18 on the Montreal Cognitive Assessment; and (6) being admitted to an outpatient rehabilitation program and expecting to continue it for at least 3 months. All participants were receiving multidisciplinary rehabilitation interventions 2-4 times per week, including physical and occupational therapy, which focused on improving activity and participation performance.

Data collection Participants meeting the eligible criteria completed the baseline assessment of a self-reported demographic questionnaire, the PM-3D4D, PART-O, PM-PAC, and the EuroQol-5-Dimension (EQ-5D) in a typical clinical appointment. Clinical data were collected by research staff from medical records. After receiving 3 months of outpatient rehabilitation, participants were again asked to complete the outcome measures. Study protocols were approved by the institutional review boards of a university and all 4 sites.

Measures Participation measure The PM-3D4D is a self-reported measure designed to evaluate 3 domains of participation, Productivity, Social, and Community, rated on 4 dimensional scales of Diversity (calculated as the number of items in which the respondent answers “yes” to the question, “Did you do this activity?” divided by the total number of items), Frequency (calculated by summing the response scores to the question, “How often did you do this activity?” divided by the total number of items), Desire for change (calculated as the number of items in which the respondent answers “yes” to the question, “Would you like to change your current participation in this activity?” divided by the total number of items), and Difficulty (calculated by summing the response scores to the question, “What was the level of difficulty in participating in this type of activity?”).9 The original PM-3D4D consisted of 19 items.9-11 In response to limitations of the original 19-item PM-3D4D (ceiling effects and poor person reliability),9 5 more items were added to the measure, and the Rasch model was used to recalibrate the item parameters within each subscale of the Difficulty dimension (supplemental tables S1 and S2, available online only at http:// www.archives-pmr.org/), for a total of 24 items (appendix 1). The summed score of each subscale was converted into a 0-100 scale based on the calibrated item parameters (appendix 2). The total difficulty dimensional score was calculated as the average of the 3 subscale scores. For the Diversity, Frequency, and Desire for change dimensions, raw scores were summed for each subscale. www.archives-pmr.org

Responsiveness of participation measures Participation assessment with recombined tools The PART-O is a 17-item participation measure that is widely used in populations with traumatic brain injury.6, 16 It measures the frequency of participation in 3 domains of Productivity, Social relations, and Community (out and about). Items are scored on a 0 (never participate) to 5 (almost always participate) rating scale. Higher scores indicate greater participation.6 Participation Measure for Post-Acute Care The PM-PAC was designed to assess perceived restrictions of participation.7 The revised version of the PM-PAC consists of 17 items covering 3 domains of participation: Productivity, Social, and Community.17 Each item is scored from 1 (not at all limited) to 5 (extremely limited), with higher scores indicating greater participation restrictions. The measure demonstrated good psychometric properties in a population with spinal cord injury.17 EuroQol-5-dimension The EQ-5D is a 5-item scale assessing the HRQoL in 5 dimensions (Mobility, Self-care, Usual activities, Pain, and Anxiety/depression) with 3 levels (1-no problems to 3-extreme problems). Summary index scores (ranging from -0.111 to 1) for the EQ-5D were calculated using a Japanese value set, with higher scores indicating a better quality of life.18 The psychometric properties of the EQ-5D have been validated in a Taiwanese population.19

Data analysis Responsiveness The responsiveness of the PM-3D4D was compared to reference measures, the PART-O and PM-PAC, since these 2 measures assess the same 3-domain construct of participation as does the PM-3D4D. Descriptive statistics were calculated for each measure at the baseline and after 3 months of rehabilitation. A paired t test was used to examine whether mean score changes significantly differed from 0 in each measure. The responsiveness of the PM-3D4D was assessed using the standardized response measure (SRM). The SRM was calculated using the equation: SRMZDifference in means/SD of the difference. Values of the SRM of 0.2, 0.5, and 0.8 were respectively considered small, moderate, and large changes. To compare the responsiveness of the PM-3D4D with the PART-O and PM-PAC, the bootstrap method was used to calculate the 95% confidence interval (CI) for differences in SRM estimates of the 3 measures.20 Any 95% CI that did not include 0 was considered to indicate a statistically significant difference in the responsiveness across the measures. Minimal detectable change The minimal detectable change (MDC) is the smallest change in score that can be detected beyond random error. The MDC with a confidence level of 90% (MDC90) was calculated as follows: MDC90Z1.64 x baseline SD x square root of (2 x [1eintraclass correlation coefficient]), where 1.64 is the z score associated with the 90% confidence level. This is estimated using a

www.archives-pmr.org

3 two-way mixed model with independent data from 200 patients, who filled out the measure twice, 2 weeks apart. Changes of  MDC90 were considered to be true changes due to an intervention effect.21 Minimal clinically important difference The meaningful clinically important difference (MCID) is the smallest improvement in score reflecting a clinically meaningful change. The distribution-based MCID was estimated. The half SD of the baseline score was calculated as an indicator of a clinically important change and was used as the minimal threshold of the clinically important difference for all subscales.22 Predictive validity The predictive validity of the PM-3D4D was determined by examining associations between PM-3D4D scores at the baseline and PART-O, PM-PAC, and EQ-5D scores postrehabilitation using Spearman correlation coefficient (r). The PART-O and PM-PAC demonstrated moderate-to-strong concurrent correlations with the PM-3D4D Frequency and Difficulty scales, respectively (r values of 0.41-0.82).11 We expected that the baseline PM-3D4D would also have moderate-to-strong correlations with the PART-O and PM-PAC postrehabilitation. Additionally, we expected a moderate association (rZ0.30-0.50) between baseline PM-3D4D and postrehabilitation EQ-5D scores, since participation is considered one of the predictors of the HRQoL.23, 24

Results Of the 269 participants enrolled in the study, 233 participants completed the 3-month rehabilitation program and postrehabilitation assessment. No missing data were found in the assessments. Thirty-six participants (mean age
SD [y], 54.31
8.36; 33.3% female) dropped out of the study due to moving, transferring to a different hospital or clinic, unavailability, or other undescribed reasons. Demographic characteristics of participants who completed the baseline assessments and those who dropped out are shown in table 1. No significant difference in characteristics were observed between the 2 samples.

Responsiveness Table 2 presents the mean score of each PM-3D4D subscale at the baseline and postrehabilitation with corresponding mean changes in scores. Significant mean changes in scores were observed in all PM-3D4D subscales except for the Social-Diversity subscale (mean changeZ0.01, PZ.05). Among the 4 dimensions of the PM-3D4D, the largest score changes were observed in the Difficulty dimensional subscales (SRMZ0.59-1.05); the Diversity and Frequency dimensional subscales showed small to moderate changes in scores (Diversity: SRMZ0.07-0.39; Frequency: SRMZ0.27-0.67); the Desire for change dimensional subscales showed small changes in score (SRMZ-0.21 to 0.14), and the directions of score changes were mixed. The mean changes in scores of the PART-O and PM-PAC are also presented in table 2. Significant mean changes in scores were observed in subscales of both measures except for the Productivity subscales. Both measures demonstrated small to moderate score

4

F.-H. Chang, P. Ni Table 1

Participant characteristics Mean
SD or Frequency (%)

Characteristic

Baseline Sample (NZ269)

Lost to Follow-Up (NZ36), P Value*

Age (y) Sex, male National Institutes of Health Stroke Scale Modified Rankin Scale Time since stroke (mo) Educational level Less than high school High school graduate College graduate or more Marital status Married/with a partner Single/widowed/separated/divorced Living with whom Living alone Living at home with family/friends Living in a nursing home With a personal care assistant, yes Barthel Index (0w100) Montreal Cognitive Assessment (0w30)

55.36
12.46 189 (70.26) 10.23
5.03 3
0.86 3.68
1.39

54.31
8.36, PZ.46 24 (66.67) 10.11
5.04, PZ.88 3
0.93, PZ1.00 3.89
1.43, PZ.33

127 (47.21) 76 (28.25) 66 (24.54)

14 (38.89) 11 (30.56) 11 (30.56), PZ.52

190 (70.63) 79 (29.37)

28 (77.78) 8 (22.22), PZ.43

7 (2.60) 246 (91.45) 16 (5.95) 96 (35.69) 71.77
18.04 23.67
3.47

1 (2.78) 33 (91.67) 2 (5.56), PZ.99 9 (25.00) 76.25
15.23, PZ.11 24.37
3.41, PZ.19

* P value of testing the demographic variables for subjects with and those without 3 months of rehabilitation. A 2-group t test or chi-square/Fisher exact test was respectively used for examining group differences for continuous or categorical variables.

changes between the 2 time points (SRMZ0.04-0.53 for PART-O and -0.02 to 0.58 for PM-PAC). Statistical comparisons of the responsiveness levels of the PM-3D4D, PART-O, and PM-PAC are shown in table 3. The overall PM-3D4D Frequency dimension demonstrated significantly greater responsiveness than did the PART-O (SRM differenceZ0.26, 95% CIZ0.1-0.43). However, no significant difference was found between their subscales except for the Productivity subscales (SRM differenceZ0.23, 95% CIZ0.090.39). The overall score and all subscales of the PM-3D4D Difficulty dimension demonstrated significantly greater responsiveness than the PM-PAC and its corresponding subscales (SRM differencesZ0.38-0.71) except for the Social subscale (SRM differenceZ0.15, 95% CIZ-0.05 to 0.36). Statistical comparisons of the SRMs for the 3 measures are also demonstrated (fig 1).

MDC and MCID The MDC90 estimates of all subscales of PM-3D4D are reported in table 4. The MDC90 estimates for each subscale were 0.07-11.24. Proportions of participants who had a positive change that exceeded the MDC90 estimates were 9.01%-24.89% in the Diversity subscales, 3.43%-14.59% in the Frequency subscales, 0.43%-1.29% in the Desire for change subscales, and 17.17%-27.90% in the Difficulty subscales. MCID estimates for each subscale were 0.07-74.17. Proportions of participants who had a positive change that exceeded the MCID estimates were 9.01%-24.89% in the Diversity subscales, 8.15%-14.59% in the Frequency subscales, 2.58%-13.3% in the Desire for change subscales, and 18.45%-33.48% in the Difficulty subscales.

Predictive validity Baseline scores on the PM-3D4D Frequency and Difficulty dimensions showed fair to moderate correlations with the PARTO and PM-PAC scores postrehabilitation, respectively (r values of 0.33-0.71, P<.0001) (table 5). Baseline scores on the PM3D4D Diversity, Frequency, and Difficulty dimensions also showed fair to moderate correlations with the EQ-5D index score at postrehabilitation (r values of 0.30-0.56, P<.0001). The PM3D4D Desire for change dimensional scores did not show significant correlations with the EQ-5D score, except that the community subscale had a weak correlation with the EQ-5D (rZ0.23, PZ.0005).

Discussion Responsiveness is a critical property for any outcome measure intended to be used in research or practice to detect changes over time or to determine the efficacy of an intervention. This is the first study to examine the responsiveness of the PM-3D4D and compare it to the responsiveness of other existing participation measures. Findings of the present study provide empirical evidence supporting the responsiveness of the PM-3D4D. Study results indicated that except for the Desire for change subscales, the PM-3D4D was responsive to change. This finding is encouraging since determining how to sensitively detect intervention effects on participation has always been a challenge for rehabilitation practitioners.12 Even though participation is a common goal of patients undergoing rehabilitation, there is a limited number of standardized outcome measures with established evidence that support the use of such measures to assess the effectiveness of rehabilitation

www.archives-pmr.org

Responsiveness of participation measures

5

Table 2 Mean score changes and responsiveness to improvement of the Participation Measuree3 Domains, 4 Dimensions, Participation Assessment with Recombined Tools-Objective, and Participation Measure for Post-Acute Care Measure/Domain PM-3D4D Diversity Social Community Productivity All Frequency Social Community Productivity All Desire for change Social Community Productivity All Difficulty Social Community Productivity All PART-O Social Community Productivity All PM-PAC Social Community Productivity All

Mean
SD at the Baseline (NZ269)

Mean
SD at Post-Rehab (NZ233)

Mean
SD of Score Change

SRM (95% CI)

0.83
0.17 0.32
0.15 0.25
0.20 0.43
0.13

0.85
0.14 0.36
0.15 0.27
0.20 0.46
0.13

0.01
0.07 0.02
0.07* 0.02
0.12* 0.02
0.05*

0.07 0.35 0.14 0.39

(-0.06 to 0.2) (0.24-0.45) (0.01-0.27) (0.28-0.5)

2.80
0.78 0.85
0.46 0.60
0.58 1.27
0.48

2.93
0.71 0.95
0.47 0.68
0.58 1.38
0.48

0.07
* 0.06
* 0.08
* 0.07
*

0.30 0.45 0.27 0.67

(0.18-0.43) (0.33-0.58) (0.15-0.39) (0.52-0.82)

0.10
0.13 0.12
0.10 0.12
0.14 0.11
0.08

0.11
0.14 0.14
0.12 0.10
0.13 0.12
0.09

-0.01
0.1* -0.02
0.1* 0.01
0.08* -0.01
0.06*

-0.14 -0.21 0.14 -0.18

70.23
16.16 59.81
13.57 62.08
17.43 64.04
14.33

76.06
16.26 62.91
13.61 65.48
17.38 68.15
14.3

5.84
7.05* 3.10
4.24* 3.39
5.73* 4.11
3.93*

0.83 0.73 0.59 1.05

(0.7-0.97) (0.6-0.88) (0.47-0.71) (0.91-1.20)

2.83
0.71 0.80
0.40 0.60
0.69 1.41
0.41

2.93
0.73 0.98
0.45 0.64
0.73 1.51
0.45

0.10
0.29* 0.15
0.29* 0.02
0.53 0.09
0.22*

0.33 0.53 0.04 0.40

(0.22-0.44) (0.41-0.67) (-0.09 to 0.16) (0.28-0.53)

19.37
3.98 28.51
7.55 13.67
11.28 61.55
17.72

20.32
3.57 29.49
6.53 12.52
7.75 62.32
13.57

0.94
1.61* 0.85
3.46* -0.10
5.4 1.68
7.35*

0.58 0.24 -0.02 0.23

(0.47-0.71) (0.1-0.42) (-0.12 to 0.16) (0.08-0.43)

(0.23) * (0.14) * (0.29) * (0.1) *

(-0.26 to -0.01) (-0.33 to -0.09) (0.02-0.27) (-0.32 to -0.05)

* The mean score change significantly differs from 0 (P<.05).

interventions on participation outcomes.12,25 Most existing participation measures are insufficiently sensitive to intervention effects, possibly due to participation goals usually varying among patients and participation outcomes possibly being influenced by various factors.25 As a result, changes in participation outcomes are difficult to detect. The PM-3D4D has demonstrated its ability to monitor changes in participation, possibly because the PM-3D4D consists of items that cover life situations highly relevant to rehabilitation patients, and changes in these life situations during rehabilitation can be effectively detected by these items. Nevertheless, not all subscales of the PM-3D4D demonstrated good responsiveness. The Diversity, Frequency, and Desire for change subscales showed small to moderate score changes. Moreover, only a limited percentage of participants exceeded the thresholds of the MDC and MCID of these subscales. That is, even though most of these subscales were generally responsive to change at a certain level, only a small proportion of participants had meaningful improvements in these participation areas. This result is consistent with the literature of testing the responsiveness of the Utrecht Scale for Evaluation of

www.archives-pmr.org

Rehabilitation (USER)-Participation26 in a multi-diagnostic group of patients in outpatient rehabilitation programs, in which the responsiveness of the Restriction scale was greatest among the 3 dimensional scales of the USER-Participation:

Table 3 Comparisons of the standard response measure among the 3 participation measures SRM Difference* (95% CI) Domain

PM-3D4D Frequency Dimension vs PART-O

PM-3D4D Difficulty Dimension vs PM-PAC

Social Community Productivity All

-0.03 -0.08 0.23y 0.26y

0.15 0.38y 0.57y 0.71y

(-0.19 to 0.15) (-0.23 to 0.07) (0.09-0.39) (0.1-0.43)

(-0.05 to 0.36) (0.12-0.6) (0.23-0.75) (0.41-0.95)

* A positive difference in the SRM means PM-3D4D had greater responsiveness than the PART-O or PM-PAC, otherwise it means it had lower responsiveness than the others. y Significant difference between the 2 measures.

6

F.-H. Chang, P. Ni Table 5 Predictive validity of the Participation Measuree3 Domains, 4 Dimensions Postrehabilitation Spearman Correlation (P Value) Baseline PM-3D4D

Fig 1 Comparison of the standardized response measure of the Participation Measuree3 Domains, 4 Dimensions and its subscales with the Participation Assessment with Recombined Tools-Objective and the Participation Measure for Post-Acute Care.

Frequency, Restriction, and Satisfaction (SRMZ0.21, 0.54, and 0.39, respectively)12. Those findings along with our findings imply that individuals’ subjective appraisal of perceived difficulty and restrictions on participation are potentially the most responsive dimensional scales to interventions compared to other participation dimensional scales. The Desire for change dimension of the PM-3D4D generally showed the least responsiveness, and the directions of score changes were mixed. This finding is not surprising since the Desire for change dimension asks about a number of areas in which individuals wish to make a difference. While individuals’

Table 4 Minimal detectable change and minimal clinically important difference estimates of the Participation Measuree3 Domains, 4 Dimensions MDC90* PM-3D4D Diversity Social Community Productivity All Frequency Social Community Productivity All Desire for change Social Community Productivity All Difficulty Social Community Productivity All

Score

MCID %

Score

%

0.11 0.08 0.10 0.07

9.01 24.89 15.45 15.45

0.09 0.08 0.1 0.07

9.01 24.89 15.45 15.45

0.60 0.24 0.30 0.25

3.43 10.73 14.59 7.73

0.39 0.23 0.29 0.24

8.15 10.73 14.59 8.15

0.24 0.25 0.23 0.19

0.86 1.29 0.86 0.43

0.12 0.1 0.1 0.09

10.3 5.15 13.3 2.58

11.24 7.04 8.09 5.76

19.31 17.17 19.31 27.90

8.08 6.79 8.72 7.17

33.48 18.45 19.31 20.17

* MDC90 and MDC at the 90% confidence level.

Diversity scores Social Community Productivity Total Frequency scores Social Community Productivity Total Desire for change scores Social Community Productivity Total Difficulty scores Social Community Productivity Total

PART-O

0.33 0.58 0.50 0.60

PM-PAC

EQ-5D Index

(<.0001)* (<.0001)* (<.0001)* (<.0001)*

0.31 0.46 0.30 0.46

(<.0001)* (<.0001)* (<.0001)* (<.0001)*

0.42 0.49 0.33 0.52

(<.0001)* (<.0001)* (<.0001)* (<.0001)*

-0.03 0.23 -0.06 0.12 0.71 0.37 0.60 0.66

(<.001)* (<.001)* (<.001)* (<.001)*

0.55 0.52 0.58 0.59

(.63) (.0005) (.35) (.08) (<.0001)* (<.0001)* (<.0001)* (<.0001)*

* A correlation was significant at P<.002 (PZ.05 corrected with the Bonferroni method).

actual participation performance can improve during rehabilitation, they may wish to make changes in more or fewer areas due to various reasons. Based on our findings, we suggest clinicians consider using this dimension as a goal-setting tool to describe specific areas in which individuals want to make changes and record whether their goals have been achieved or changed instead of as an outcome indicator to evaluate intervention effectiveness. That is, items for which a patient desire to make changes should be treated as individual goals for rehabilitation instead of being scored as a scale. Clinicians may use the Desire for change dimensional scales as supplementary scales in addition to the other dimensional scales. Further discussion is needed in the process of improving this measure. The responsiveness of the PM-3D4D was compared to 2 other participation measures: PART-O and PM-PAC. Both PART-O and PM-PAC showed small to moderate score changes. Our findings indicated that the responsiveness of the Frequency and Difficulty dimensions of the PM-3D4D was, at least to some extent, greater than that of the PART-O and PM-PAC. In fact, the responsiveness of the Frequency and Difficulty dimensions of the PM-3D4D was potentially greater than that of various other existing participation measures. Van der Zee et al25 tested and reviewed the responsiveness of several existing participation measures, such as the USER-Participation, the Impact on Participation and Autonomy,27 the Frenchay Activities Index,28 and the London Handicap Scale,29 and found that all of the measures demonstrated small to moderate changes in participation. Comparing the existing participation measures, the PM-3D4D has the potential to be a

www.archives-pmr.org

Responsiveness of participation measures more-suitable option for monitoring changes in participation outcomes. In line with the evidence that supports the concurrent validity of the PM-3D4D,11 our findings further support the predictive validity of the PM-3D4D by demonstrating that PM-3D4D scores at admission to outpatient rehabilitation wards predicted postrehabilitation participation outcomes as measured by the PARTO and PM-PAC. Moreover, the PM-3D4D Diversity, Frequency, and Difficulty dimensional scores also predicted the EQ-5D score, indicating that patients with a stroke history and better participation performance at admission would possibly have a more-favorable HRQoL following their outpatient rehabilitation. This result is consistent with the literature which reveals that participation is a significant predictor of the HRQoL among community-dwelling survivors of stroke.23 However, the PM3D4D Desire for change dimensional scores had minimal predictive power for predicting the EQ-5D score, suggesting that clinicians should not use the number of areas in which participants desire to make changes at admission to predict their postrehabilitation HRQoL.

7 to test the responsiveness of the PM-3D4D in populations with more chronic conditions.

Conclusions This study provides preliminary evidence supporting the responsiveness and predictive validity of the PM-3D4D in patients with a stroke history who were undergoing outpatient rehabilitation services. The study results suggest that most of the PM-3D4D subscales showed significant and even greater responsiveness than that of 2 other participation measures: the PART-O and PM-PAC. However, the Desire for change dimension of the PM-3D4D showed less responsiveness. The Desire for change dimension scale may be considered a goal-setting tool to describe areas where individuals want to make changes rather than as an outcome indicator to evaluate intervention effectiveness. This study also defined the MDC and MCID of the PM3D4D, which will provide clinicians with important indices to determine whether a patient has meaningful improvement. Future research based on a larger and more diverse sample is warranted to verify these findings.

Study limitations

Keywords

This study has several limitations. First, the study only recruited a convenience sample of people with a stroke. The SRM and MCID may vary across populations since they are dependent on participants’ characteristics. Future studies need to test application of the PM-3D4D in larger samples of different populations to validate our findings. Second, although all participants in the study received 3 months of multidisciplinary outpatient rehabilitation services, the intensity and contents of the services each person received may have been individualized and varied, which may have resulted in differences in participation improvement, and the responsiveness of the participation measures may have been underestimated. Future research should consider replicating this study by examining the responsiveness of the PM-3D4D in people participating in a more homogeneous intervention to see whether the same responsiveness results are sustained. Finally, participants in this study were mostly in the postacute stage of their illness. Future studies could be expanded

Outpatients; Psychometrics; Rehabilitation; Social participation; Stroke

www.archives-pmr.org

Corresponding author Feng-Hang Chang, ScD, OTR/L, Graduate Institute of Injury Prevention and Control, College of Public Health, Taipei Medical University, 250 Wu-Hsing Street, Taipei City 11031, Taiwan. E-mail address: [email protected].

Acknowledgment We thank our collaborating hospitals: Taipei Medical University Hospital, Wan Fang Hospital, Shuang Ho Hospital, and National Taiwan University Hospital.

8

F.-H. Chang, P. Ni Appendix 1

PM-3D4D Items and Response Options Items

Social S1. Keep in touch with others through phone, email, instant message, or letter. S2. Get together with friends or family (e.g., visit friends or invite friends to your house for dinner). S3. Engage in sexual activities (e.g., kiss, touch, or have sex). S4. Attend gatherings with friends, family, or social groups (e.g., family gatherings, school reunions, or patient support groups). S5. Connect with your neighbors. S6. Talk to strangers (e.g., ask for directions or order a meal). Community C1. Participate in political activities (e.g., vote in elections or participate in political events). C2. Shop on-line or by phone. C3. Go shopping or run errands. C4. Go to movies, concerts, shows, sports events, lectures, or exhibits. C5. Go out for exercise or to attend outdoor activities (e.g., walk, jog, or do tai chi or yoga). C6. Eat out or go out for a drink. C7. Go out to do a sedentary activity (e.g., read books in a library, write or surf the internet in a coffee shop). C8. Play sports (e.g., play badminton, basketball, table tennis, or baseball). C9. Participate in non-physical group entertaining activities (e.g., playing cards, chess, mahjong, or on-line games). C10. Participate in religious or spiritual activities (e.g., go to a church, temple, or meditation group). C11. Travel in your own country. C12. Travel abroad. Productivity P1. Work for a job (including full-time job, part-time job, sheltered job, or temporary job). P2. Work as a volunteer (e.g., work for a community or charity group without remuneration) or do other unpaid work (e.g., babysit a family member’s child without payment). P3. Receive school education or take courses for personal skills development (e.g., take courses such as languages, computer, music, art, or finance in a community college). P4. Do household work for the family (e.g., make meals, care for pets, do laundry, housecleaning, gardening, or minor household repairs and maintenance for the family). P5. Provide care for minor children or other dependent family members in your household. P6. Get Involved in investment activities (e.g., buy and sell stocks, buy lottery tickets, and collect rent from tenants) Questions and response options for all items In the past 3 months. I. Did you do this II. How often did you do this activity? III. Would you like to change your IV. What was the level of activity? current participation in this activity? difficulty in participating in this type of activity? 1 , Yes 1 , Very difficult 1, Yes Social and Community 0 , Never in past 3 months 0 , No 2 , Moderately difficult 0 , No 1 , Once or twice in past 3 months 3 , A little difficult 2 , Once a month 4 , Not difficult at all 3 , 2 to 3 times a month 4 , Once a week 5 , 2 to 4 times a week 6 , Every day or almost every day Productivity 0 , Less than once a month 1 , 1 to 3 days a month 2 , Once a week 3 , 2 to 4 days a week 4 , 5 or more days a week The latest version of the PM-3D4D and its scoring sheet can be downloaded in PDF format from the website: http://my2.tmu.edu.tw/fhchang/ doc/129540. The instrument is free for research and clinical practice use but not for commercial use. Modifications and language translations of the instrument are not allowed without written permission from the author. For further information, please contact: Feng-Hang Chang, ScD OTR/ L [email protected]. NOTE. The 24-item PM-3D4D was created in response to the limitations of the original 19-item PM-3D4D (eg, ceiling effects and poor person reliability). The measurement properties of the 24-item PM-3D4D were re-evaluated with a sample of 500 patients in rehabilitation outpatient settings. The Rasch partial-credit model was used to calibrate items within each Difficulty dimensional subscale and generate logit scores.

www.archives-pmr.org

Responsiveness of participation measures

9

Appendix 2 The Rasch-based Score Conversion Table of the 24item PM-3D4D Difficulty Dimensional Subscales Raw Score Social 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Community 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

0-100 Score

SE

0 10.45 19.50 29.18 37.81 43.48 47.62 50.94 53.98 56.88 59.86 63.10 66.77 71.05 75.96 81.69 89.50 100

13.34 8.43 7.80 8.36 6.91 5.74 5.04 4.63 4.49 4.49 4.63 4.91 5.25 5.60 6.01 6.63 8.29 13.41

0 9.22 15.36 19.54 22.85 25.62 28.05 30.21 32.23 34.05 35.74 37.36 38.91 40.39 41.81 43.23 44.65 46 47.35 48.69 50.04 51.39 52.74 54.16 55.65 57.13 58.61 60.23 61.99 63.88 65.90 68.26 71.03 74.47 79.47

12.75 7.49 5.74 4.93 4.52 4.18 3.91 3.71 3.58 3.44 3.37 3.24 3.17 3.17 3.10 3.04 3.04 3.04 3.04 3.04 3.04 3.04 3.04 3.10 3.17 3.17 3.24 3.37 3.44 3.64 3.85 4.11 4.52 5.20 6.48 (continued on next column)

www.archives-pmr.org

Appendix 2 (continued ) Raw Score

0-100 Score

SE

47 48 Productivity 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

88.04 100

8.97 13.70

0 12.69 22.08 28.82 34.19 38.72 42.64 46.31 49.72 53.05 56.29 59.54 62.86 66.45 70.29 74.64 80.10 88.12 100

16.47 10.15 8.11 7.08 6.49 5.97 5.63 5.46 5.38 5.30 5.30 5.30 5.38 5.55 5.89 6.40 7.34 9.56 16.21

References 1. World Health Organization. The atlas of heart disease and stroke. Available at: http://www.who.int/cardiovascular_diseases/resources/ atlas/en/. Accessed September 23, 2015. 2. Teasell RW, McRae MP, Finestone HM. Social issues in the rehabilitation of younger stroke patients. Arch Phys Med Rehabil 2000;81: 205-9. 3. Kersten P, Low JT, Ashburn A, George SL, McLellan DL. The unmet needs of young people who have had a stroke: results of a national UK survey. Disabil Rehabil 2002;24:860-6. 4. Whiteneck GG, Dijkers MP. Difficult to measure constructs: conceptual and methodological issues concerning participation and environmental factors. Arch Phys Med Rehabil 2009;90:S22-35. 5. Chang FH, Coster WJ, Helfrich CA. Community participation measures for people with disabilities: a systematic review of content from an international classification of functioning, disability and health perspective. Arch Phys Med Rehabil 2013; 94:771-81. 6. Whiteneck GG, Dijkers MP, Heinemann AW, Bogner JA, Bushnik T, Cicerone KD, et al. Development of the Participation Assessment With Recombined ToolseObjective for Use After Traumatic Brain Injury. Arch Phys Med Rehabil 2011;92:542-51. 7. Gandek B, Sinclair SJ, Jette AM, Ware JE Jr. Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). Am J Phys Med Rehabil 2007;86:57-71. 8. Chang FH, Coster WJ. Conceptualizing the construct of participation in adults with disabilities. Arch Phys Med Rehabil 2014;95: 1791-8. 9. Chang FH, Liou TH, Ni P, Chang KH, Lai CH. Development of the Participation Measuree3 Domains, 4 Dimensions (PM-3D4D): a new outcome measure for rehabilitation. Arch Phys Med Rehabil 2017;98: 286-94. 10. Chang FH. Measuring participation of rehabilitation patients: testretest reliability and mode of administration concordance of the

10

11.

12.

13.

14. 15. 16.

17.

18. 19.

20.

F.-H. Chang, P. Ni Participation Measure–3 Domains, 4 Dimensions (PM-3D4D). Arch Phys Med Rehabil 2017;98:1622-7. Chang FH, Chang KH, Liou TH, Whiteneck GG. Validation of the Participation Measure-3 Domains, 4 Dimensions (PM-3D4D). Arch Phys Med Rehabil 2017;98:2498-506. van der Zee CH. Responsiveness of four participation measures to changes during and after outpatient rehabilitation. J Rehabil Med 2011;43:1003-9. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures statistics and strategies for evaluation. Control Clin Trials 1991;12:S142-58. Liang MH. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Med Care 2000;38:II84-90. Portney LG, Watkins MP. Foundations of clinical research: applications to practice. New Jersey: Prentice Hall; 2008. Wen PS, Waid-Ebbs JK, Graham DP, Helmer DA. Psychometric properties of 2 participation measures in veterans with mild traumatic brain injury. Arch Phys Med Rehabil 2018;99:S86-93. Chang FH, Ni P, Coster WJ, Whiteneck GG, Jette AM. Measurement properties of a modified measure of participation for persons with spinal cord injury. J Spinal Cord Med 2016;39:476-83. Tsuchiya A, Ikeda S, Ikegami N, et al. Estimating an EQ-5D population value set: the case of Japan. Health Econ 2002;11:341-53. Chang TJ, Tarn YH, Hsieh CL, Liou WS, Shaw JW, Chiou XG. Taiwanese version of the EQ-5D: validation in a representative sample of the Taiwanese population. J Formos Med Assoc 2007;106:1023-31. Cohen J. Statistical Power Analysis for the Behavioral Sciences. New York: Lawrence Erlbaum Associates; 1988.

21. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. 22. Rai SK, Yazdany J, Fortin PR, Avin˜a-Zubieta JA. Approaches for estimating minimal clinically important differences in systemic lupus erythematosus. Arthritis Res Ther 2015;17:143. 23. White J, Magin P, Attia J, Sturm J, McElduff P, Carter G. Predictors of health-related quality of life in community-dwelling stroke survivors: a cohort study. Fam Pract 2016;33:382-7. 24. Silva SM, Correˆa FI, Faria CD, Pereira GS, Attie´ EA, Correˆa JC. Reproducibility of the items on the Stroke Specific Quality of Life questionnaire that evaluate the participation component of the International Classification of Functioning, Disability and Health. Disabil Rehabil 2016;38:2413-8. 25. van der Zee CH, Baars-Elsinga A, Visser-Meily JMA, Post MW. Responsiveness of two participation measures in an outpatient rehabilitation setting. Scand J Occup Ther 2013;20:201-8. 26. Post MW, van der Zee CH, Hennink J, Schafrat CG, Visser-Meily JM, van Berlekom SB. Validity of the Utrecht Scale for evaluation of rehabilitation-participation. Disabil Rehabil 2012;34:478-85. 27. Cardol M, de Haan RJ, van den Bos GA, de Jong BA, de Groot IJ. The development of a handicap assessment questionnaire: the Impact on Participation and Autonomy (IPA). Clin Rehabil 1999; 13:411-9. 28. Holbrook M, Skilbeck CE. An activities index for use with stroke patients. Age Ageing 1983;12:166-70. 29. Harwood RH, Rogers A, Dickinson E, Ebrahim S. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care 1994;3:11-6.

www.archives-pmr.org