Reliability and Validity of the Work Instability Scale for Rheumatoid Arthritis

VALUE IN HEALTH 18 (2015) 1008–1015

Available online at www.sciencedirect.com

journal homepage: www.elsevier.com/locate/jval

Reliability and Validity of the Work Instability Scale for Rheumatoid Arthritis Dennis Revicki, PhD1,*, Arijit Ganguli, PhD2, Miriam Kimel, PhD1, Sanjoy Roy, MS2, Naijun Chen, MS2, Shima Safikhani, MPH1, Mary Cifaldi, PhD2 1

Evidera Inc., Bethesda, MD, USA; 2AbbVie Inc., North Chicago, IL, USA

AB STR A CT

Objective: The objective was to evaluate the psychometric properties of the Rheumatoid Arthritis-Work Instability Scale (RA-WIS) in a clinical trial setting. Methods: Secondary analyses were conducted using data from a 56-week, randomized controlled trial of patients with early rheumatoid arthritis (RA). Patient-reported outcome measures included the RA-WIS, the Health Assessment Questionnaire (HAQ), the Rheumatoid Arthritis Quality of Life Questionnaire, and the Global Assessment of Disease Activity and Pain, data for which were collected at baseline and at weeks 12, 16, 24, and 56. Data were analyzed for reliability, validity, and responsiveness. Results: Among 148 patients whose data were analyzed, more than half were women (56.1%) with a mean age of 46.8 years. On average, patients experienced RA symptoms for 8.7 months; the mean 28-Joint Disease Activity Score (DAS28) was 5.9, and the mean HAQ – Disability Index was 1.3. The RA-WIS demonstrated excellent internal consistency and test-retest reliability (α ¼ 0.89 and intraclass correlation coefficient ¼ 0.91, respectively). At baseline and week 24, moderate to strong

correlations were seen between RA-WIS total scores and the HAQ, the Global Assessment of Disease Activity, and the Pain Rheumatoid Arthritis Quality of Life Questionnaire, ranging from 0.47 to 0.81 (all P o 0.0001). Mean RA-WIS total scores and work disability risk levels discriminated between clinical severity scores on the DAS28, the HAQ – Disability Index, and the Physician Global Assessment of Disease Activity (all P o 0.05). Mean baseline to week 24 RA-WIS total change scores were significantly different among American College of Rheumatology responder groups (P r 0.0001) and between DAS28 remission status groups (P o 0.001). Conclusions: These findings provide evidence supporting the reliability, validity, and responsiveness of the RA-WIS for evaluating work disability in patients with RA in a clinical trial setting. Keywords: reliability and validity, rheumatoid arthritis, work disability, work instability.

Introduction

The Rheumatoid Arthritis-Work Instability Scale (RA-WIS) was developed to assess the level of risk for work disability in patients with RA, providing clinicians with an effective screening tool to facilitate early, appropriate referral for job retention measures [19]. Initial work to develop the RA-WIS was conducted in patients with early RA and included 1) qualitative interviews to identify themes relevant to patients in relation to maintaining work; 2) assessment of criterion validity against the criterion standard (i.e., full vocational assessments); and 3) Rasch analysis to assess discriminative properties and unidimensionality [19]. The initial research on the RA-WIS provides evidence of content and criterion validity and good test-retest reliability [19–21]. Additional studies have supported the reliability, validity, and responsiveness of the RA-WIS in people with RA [22–25]. Beaton et al. [22] demonstrated the relationship between the RA-WIS and other work productivity measures cross-sectionally and over 12 months.

Approximately 20% of the patients with early rheumatoid arthritis (RA) become permanently work disabled during the first 2 to 3 years of disease [1–5], and the frequency of work disability increases over time with approximately one-third of all patients becoming work disabled within 10 years of diagnosis [5,6]. In early RA, aggressive treatment has been shown to reduce functional disability [7–12] and positively affect employment outcomes [7,13–18]. Work instability (WI), a less frequently evaluated construct, relates to an individual’s ability to fulfill normal work tasks and refers to individuals having to make adjustments or job changes due to a mismatch between their functional capabilities and job demands [19]. WI is viewed as a transient, potentially reversible state [19] in which individuals are vulnerable to job loss and interventions are of greatest preventive importance in relation to work.

Copyright & 2015, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc.

Conflict of interest: D. Revicki, M. Kimel, and S. Safikhani are employed by Evidera, which provides consulting and other research services to pharmaceutical, device, government, and nongovernment organizations. In this salaried position, D. Revicki, M. Kimel, and S. Safikhani work with various companies and organizations. They receive no payment or honoraria directly from these organizations for services rendered. A. Ganguli, N. Chen, and M. Cifaldi are full-time employees of AbbVie. S. Roy is a former employee of AbbVie. * Address correspondence to: Dennis Revicki, Evidera Inc., 7101 Wisconsin Avenue, Suite 1400, Bethesda, MD 20814. E-mail: [email protected]. 1098-3015$36.00 – see front matter Copyright & 2015, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jval.2015.09.2941

VALUE IN HEALTH 18 (2015) 1008–1015

Psychometric evaluation is an ongoing process that begins with qualitative assessment to support content validity and quantitative assessment to support reliability, validity, and ability to detect change. Psychometric evaluation is also a cumulative process by which supportive evidence from multiple studies conducted in different patient populations adds confidence that the measure is psychometrically sound and appropriate for use in multiple settings. The objective of this study was to evaluate the psychometric properties of the RA-WIS in a clinical trial setting in patients with early RA to provide further evidence for the suitability of this measure in general and specifically for use as an end point in clinical trials. This is the first psychometric evaluation of the RA-WIS based on longitudinal data from a clinical trial population comparing treatment in patients with early RA.

Methods Research Design and Study Population This study was a secondary analysis of data from the PRevention Of Work Disability (PROWD) study—a 56-week, randomized, double-blind, comparator-controlled, phase IIIb trial of methotrexate (MTX)-naive patients with early RA. A total of 148 patients were randomized to receive adalimumab 40 mg subcutaneous injections every other week plus weekly oral MTX (7.5–25 mg) (n ¼ 75) or placebo every other week plus weekly oral MTX (n ¼ 73). Patients were eligible to participate in the study if they had RA based on the 1987 revised American College of Rheumatology (ACR) classification criteria [26] with less than 2 years of symptom duration, were 18 years or older, were in paid employment, had self-reported RA-related work impairment, and were MTX/biologic naive. Patients with active infections, including tuberculosis, previous malignancy, and uncontrolled concomitant illness, were excluded. Patients were given the option to discontinue from the trial at week 16 for reasons of loss of employment, imminent employment loss, lack of efficacy or adverse effects (N ¼ 40; adalimumab, n ¼ 15; placebo, n ¼ 25). Among those who discontinued the trial at week 16, 75% did so because of loss of employment [7]. A detailed description of the study design has been previously reported [7].

Patient-Reported Outcome Measures Demographic and medical history data were collected at baseline along with the following patient-reported outcome (PRO) measures: RA-WIS, Health Assessment Questionnaire – Disability Index (HAQ-DI), Patient Global Assessment of Disease Activity, Patient Assessment of Pain (visual analogue scale [VAS]), RA Quality of Life Questionnaire (RAQoL), Work Disability Assessment, and Weekly Work Diary. Data for the RA-WIS, HAQ-DI, RAQoL, and VAS measures were also collected 4 weeks before baseline and at weeks 12, 16, 24, and 56.

Rheumatoid arthritis-work instability scale The RA-WIS is a 23-item questionnaire with a “yes/no” response scale covering a broad range of specific work-related issues with a recall period of “at the moment.” Item scores are summed and range from 0 to 23, with higher scores reflecting greater WI and higher risk for work disability. Scores greater than 17 indicate high risk for work disability, scores between 10 and 17 indicate moderate WI, and scores less than 10 indicate low WI [19].

1009

and range from 0 (no difficulties performing tasks) to 3 (total inability to perform tasks) [27]. Research suggests that a mean score change of –0.22 is considered clinically meaningful in patients with RA [28,29].

Disease assessment VASs Three VAS scales were used to assess disease: 1) Physician Global Assessment of Disease Activity, 2) Patient Global Assessment of Disease Activity, and 3) Patient Assessment of Pain. For assessments of disease activity, either physicians or patients rate patients’ current RA disease activity from 0 (no symptoms) to 100 (very active). For assessment of pain, patients rate their pain within the past week from a score of 0 (no pain) to 100 (severe pain).

RA quality of life questionnaire The RAQoL consists of 30 items designed to assess quality of life (QOL) in people with RA [30]. Each item is in the form of a simple statement (e.g., “I have difficulty dressing”) with a “yes/no” response scale. Each item is given a score of “1” (item affirmed) or “0” (item not affirmed). Item scores are summed to give a total score that ranges from 0 (good QOL) to 30 (poor QOL) [30].

Work disability assessment The Work Disability Assessment evaluates the impact of RA on work, including time in current job (in years), ability to work, total hours normally worked per week, sick leave since onset of RA, modification of job since onset of RA, adaptation of working environment since onset of RA, seeking assistance regarding employment since onset of RA, and seeing a work disability advisor because of RA. A “yes/no” response scale is used for most questions.

Weekly work diary The Weekly Work Diary is a detailed record of patients’ work practices and specifically evaluates the number of hours worked each day of the week and days of the week worked, whether the hours worked were the normal hours of working for each day, the number of working hours lost for each day of the week, and whether the hours of work lost were due to RA. Impact on work performance was assessed using a VAS anchored from 0 to 100.

Demographic Characteristics Data on demographic and social/medical history variables such as date of birth, sex, race, and ethnicity were collected at baseline.

Clinical Outcome Measures Disease activity and joint counts were assessed at screening (week –4), baseline (week 0), and weeks 12, 16, 24, and 56.

Swollen joint counts and tender joint counts Twenty-eight joints were examined for swelling and joint pain/ tenderness. Swelling and joint pain/tenderness were classified as present (“1”), absent (“0”), or replaced (“9”), in the case of endoprosthesis.

Health assessment questionnaire

Morning stiffness of joints

The HAQ-DI consists of 20 questions on activities of daily living with a recall period of “the past week” [27]. Scores are continuous

The average daily duration of morning stiffness (for the preceding week) was noted in minutes, up to a maximum of 720 minutes.

1010

VALUE IN HEALTH 18 (2015) 1008–1015

28-Joint disease activity score and ACR response

Known-groups validity

The Disease Activity Score (DAS) is a combined index that has been developed to measure the disease activity in patients with RA [31]. The 28-Joint DAS (DAS28) is measured using the number of swollen joint counts (SJCs) and tender joint counts (TJCs), erythrocyte sedimentation rate, and a patient’s general health or global disease activity VAS. The DAS calculation provides a number between 0 and 10, indicating how active RA is at that moment. DAS28 scores are interpreted as follows: remission (DAS28 r 2.6), low disease activity (2.6 o DAS28 r 3.2), moderate disease activity (3.2 o DAS28 r 5.1), and high disease activity (DAS28 4 5.1) [32]. The ACR criteria for improvement are based on improvement in both SJCs and TJCs and in three of the following measures: Patient Global Assessment of Disease Activity, Physician Global Assessment of Disease Activity, erythrocyte sedimentation rate, pain scale, and physical function (e.g., HAQ-DI) [33]. Improvement is denoted as ACR 20, ACR 50, or ACR 70, reflecting an improvement of 20%, 50%, or 70% in the outlined parameters.

The ability of RA-WIS total scores to discriminate between groups of patients according to clinical severity was assessed using analysis of variance with Scheffe’s post hoc comparisons to evaluate mean differences between groups at baseline and week 24. Clinical severity categories for the HAQ-DI, DAS28, Physician Global Assessment of Disease Activity, and work hours lost per week because of RA were based on quartile distributions from descriptive analysis. Known-groups validity for the RA-WIS by work disability risk categories was assessed using chi-square statistics. Clinical severity categories were the same as those previously described for the analysis of RA-WIS total scores. RA-WIS risk categories were defined as high, medium, or low.

Statistical Analysis The psychometric analyses of the RA-WIS were prespecified in an a priori statistical analysis plan. The psychometric analyses combined data from both treatment groups and were performed blinded to treatment group status. All statistical tests used a significance level of 0.05 unless otherwise noted.

Descriptive statistics Means and frequencies were used to describe the following variables at baseline: sociodemographic characteristics, clinical factors, and PRO assessments.

Reliability Internal consistency of the RA-WIS items was evaluated using the Kuder-Richardson 20 formula for internal consistency reliability for dichotomous scales. The reproducibility of the RA-WIS total score was assessed using intraclass correlation coefficients (ICCs), paired t tests, and Pearson product-moment correlations between visit 1 (week –4) and visit 2 (week 0). Only patients considered stable were included in this analysis, defined as having less than a 0.22 change in HAQ-DI scores between visit 1 and visit 2. ICC values of more than 0.70 are generally considered acceptable [34].

Construct validity Spearman correlations were calculated to evaluate relationships between the RA-WIS total score and clinical assessments and RAQoL, HAQ-DI, Patient Global Assessment of Disease Activity, and Patient Assessment of Pain at baseline and week 24. The construct of WI for the RA-WIS is based on items that assess specific job effects and emotional and physical effects. Because the RA-WIS measures multiple components of work impact, we anticipated that the construct of WI would be associated with individual measures that assess each of these constructs. We hypothesized that moderate to large correlations will be observed between the RA-WIS total score and the measures of disease severity, pain, and quality of life. Construct validity was supported when the RA-WIS total score was correlated with a clinical or PRO measure with coefficient r 4 0.40, reflecting a moderately strong relationship [35]. Correlation coefficients greater than 0.5 are considered large (strong) and those less than 0.2 are considered small (weak).

Responsiveness An anchor-based analytic approach was used to assess the relationship between changes in clinical and work status and changes in RA-WIS total scores [36]. Mean changes in RA-WIS scores were assessed for two time frames (from baseline to weeks 24 and 56) among the following disease activity groups:

 



Clinical remission (DAS28 o 2.6 at weeks 24 and 56) versus nonresponder (DAS28 Z 2.6 at weeks 24 and 56); Clinical responders (DAS28 response of good [score r3.2], or moderate [score 43.2 and r5.1], based on the European League Against Rheumatism (EULAR) criteria at weeks 24 and 56) versus DAS28 nonresponders; and Clinical responders (ACR responses of 20%, 50%, and 70% at weeks 24 and 56) versus nonresponders (ACR response o20% at weeks 24 and 56).

The RA-WIS total baseline to weeks 24 and 56 change scores were adjusted for age, sex, and baseline RA-WIS total scores and were compared using analysis of covariance models with clinical status as the independent factor.

Results Descriptive Statistics Sociodemographic and clinical characteristics A total of 148 patients with a mean age of 46.8 years were included in the analysis sample. Women (n ¼ 83 [56.1%]) comprised slightly more than half of the sample, and most (n ¼ 141 [95.3%]) were white. A detailed description of patient characteristics has previously been published [7]. On average, patients in this trial experienced RA symptoms for 8.7 months with DAS28 of 5.9, suggesting early and severe disease. Patients experienced some difficulty performing their daily tasks (mean HAQ-DI scores of 1.3), with greater than fourfifths of the patients having medium or high risk of work disability (Table 1). Patients reported losing approximately 5 hours of work per week on average because of RA. Sixty percent reported using sick leave since the onset of RA, and 23.7% required modification to their job after the onset of RA. RA impact on work performance was 50.9 ⫾ 30.5. On the basis of RA-WIS distribution frequency at baseline (N ¼ 145), most of the patients reported experiencing work-related difficulties related to having stamina (86.9%), experiencing stiffness (86.2%), experiencing pressure in the hands (86.9%), or having good and bad days at work (91.7%).

1011

VALUE IN HEALTH 18 (2015) 1008–1015

Table 1 – Baseline patient-reported outcomes. Outcome

Value (N ¼ 148)

RA-WIS,* mean ⫾ SD Work disability risk based on RA-WIS,† n (%) High Medium Low Missing RAQoL,‡ mean ⫾ SD HAQ-DI,§ mean ⫾ SD Patient Global Assessment of Disease Activity,ǁ mean ⫾ SD Patient Assessment of Pain,¶ mean ⫾ SD Time in current job (y), mean ⫾ SD Total hours normally worked per week, mean ⫾ SD Sick leave since onset of RA, n (%) Modification of job since onset of RA, n (%) Adaptation of working environment since onset of RA, n (%) Seeing of work disability advisor because of RA, n (%) Sought assistance regarding employment since onset of RA, n (%) No. of hours worked per week, mean ⫾ SD No. of hours lost per week, mean ⫾ SD No. of hours lost per week because of RA, mean ⫾ SD Work performance affected by RA, mean ⫾ SD

15.3 ⫾ 5.5

Table 2 – Construct validity at baseline and week 24: Correlation between RA-WIS and clinical- and patient-reported measures. Measure

RA-WIS* (N ¼ 148) Week 0

58 (39.2) 66 (44.6) 23 (15.5) 1 (0.7) 16.7 ⫾ 7.4 1.3 ⫾ 0.6 61.5 ⫾ 25.6 62.4 ⫾ 25.0 11.6 ⫾ 12.1 35.7 ⫾ 12.3 89 (60.1) 35 (23.7) 18 (12.2) 5 (3.4) 33 (22.3) 21.4 ⫾ 15.2 5.4 ⫾ 9.1 4.9 ⫾ 9.1 50.9 ⫾ 30.5

HAQ-DI, Health Assessment Questionnaire – disability index; RA, rheumatoid arthritis; RAQoL, Rheumatoid Arthritis Quality of Life; RA-WIS, Rheumatoid Arthritis-Work Instability Scale. * Range is 0–23, with higher scores reflecting greater disability risk. † Categorization of RA-WIS scores: low risk (0–9), medium risk (10– 17), and high risk (18–23). ‡ Range 0–30, with higher scores reflecting worse HRQOL. § Range 0 (no difficulty) to 3 (unable to do). ǁ Range 0 (no symptoms) to 100 (very active). ¶ Range 0 (no pain) to 100 (severe pain).

Psychometric Properties of the RA-WIS Reliability Internal consistency reliability at baseline was 0.89 for the RAWIS. Pearson product-moment correlations between RA-WIS total scores at screening (week –4) and baseline (week 0) were high (r ¼ 0.91) and demonstrated substantial agreement between test-retest scores (ICC ¼ 0.91).

Construct validity Spearman correlation coefficients between RA-WIS total scores and clinical and PRO measures at baseline and week 24 were moderate to strong for the HAQ-DI, Patient Global Assessment of Disease Activity, and Patient Assessment of Pain and RAQoL, ranging from 0.47 to 0.81 (all P o 0.0001) (Table 2). At baseline, correlations with RA-WIS total scores were weaker for duration of morning stiffness, DAS28, Physician Global Assessment of Disease Activity, and TJC, ranging from 0.31 to 0.37 (all P o 0.001). Correlations with most of these measures were moderate at week 24 (0.36 for duration of morning stiffness to 0.55 for DAS28 and TJC; P o 0.05 and P o 0.0005, respectively). Although the correlation between RA-WIS total and SJC scores at baseline was weak and nonsignificant, a moderate, significant correlation was seen at week 24 (r ¼ 0.46; P o 0.0001). Consistently across both time periods, the strongest associations were seen

Duration of morning stiffness HAQ-DI DAS28 Patient Global Assessment of Disease Activity Patient Assessment of Pain Physician Global Assessment of Disease Activity RAQoL SJC28 TJC28 No. of hours lost per week because of RA No. of hours lost per week No. of hours worked per week Effect of RA on work performance

Week 24

0.31† 0.55§ 0.37§ 0.47§

0.36‡ 0.76§ 0.55§ 0.66§

0.55§ 0.31†

0.62§ 0.52§

0.77§ 0.07 0.35§ 0.37§

0.81§ 0.46§ 0.55§ 0.27‡

0.30† –0.09 0.37§

0.09 –0.12 0.64§

DAS28, 28-Joint Disease Activity Score; HAQ-DI, Health Assessment Questionnaire – disability index; RA, rheumatoid arthritis; RA-WIS, Rheumatoid Arthritis-Work Instability Scale; RAQoL, Rheumatoid Arthritis Quality of Life; SJC, swollen joint count; TJC, tender joint count. * Spearman rank correlation. † P o 0.001. ‡ P o 0.05. § P o 0.0001.

between the RA-WIS and the RAQoL, HAQ-DI, Patient Global Assessment of Disease Activity, and Patient Global Assessment of Pain.

Known-groups validity RA-WIS total scores were able to discriminate between clinical severity scores on the HAQ-DI, DAS28, and Physician Global Assessment of Disease Activity and measures of work hours lost because of RA. Mean RA-WIS scores by HAQ-DI categories were as follows: 10.9 (HAQ-DI r0.88), 15.4 (HAQ-DI 40.88–1.38), 16.9 (HAQ-DI 41.38–1.75), and 18.8 (HAQ-DI 41.75) (P o 0.001) (Fig. 1A). Mean RA-WIS scores by DAS28 categories were 3.74 (DAS28 r3.2), 11.82 (DAS28 43.2–5.1), and 16.33 (DAS28 45.1) (P o 0.0001). Similar findings were seen for work hours lost per week because of RA, where mean RA-WIS scores by lost hours categories were 13.87 (no lost hours), 15.50 (r1 day lost per week), and 16.9 (41 day lost per week) (P ¼ 0.0006) (Fig. 1B). The findings were consistent at baseline and week 24. Similar findings were seen when categorizing RA-WIS scores into work disability risk levels (i.e., high, medium, low) (data not shown).

Responsiveness Responsiveness of the RA-WIS to changes in disease activity was evaluated by assessing the relationship between changes in RAWIS total scores and disease activity measures (i.e., ACR responder status and DAS28 remission status) from baseline to week 24 and from baseline to week 56 (Fig. 2A,B). Mean baseline to week 24 RA-WIS total change scores were significantly different among ACR responder groups (i.e., ACR o20%, Z20%–o50%, Z50%–o70%, and Z70%; P r 0.0001). Similar findings were seen for RA-WIS total change scores from baseline to week 56 (Fig. 2A) (P r 0.0001). The greatest differences

1012

VALUE IN HEALTH 18 (2015) 1008–1015

(A)

Known Groups Validity of RA-WIS Total by HAQ Scores

WIS Total Scores

HAQ <=0.88

23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

HAQ>0.88 to 1.38

HAQ>1.38 to 1.75

HAQ>1.75

***

***

Week 0

Week 24 HAQ Categories at Weeks 0 and 24

(B)

Known Groups Validity of RA-WIS Total by Hours Lost Per Week Due to RA No Hours Lost

20

<=1 Day

>1 Day

**

18

**

16

WIS Total Scores

14 12 10 8 6 4 2 0

Week 0

Week 24

Hours Lost Per Week Due to RA Categories at Weeks 0 and 24

Fig. 1. – Known-groups validity of RA-WIS total score by HAQ scores. HAQ, Health Assessment Questionnaire; RA-WIS, Rheumatoid Arthritis-Work Instability Scale. at week 24 were seen between responder groups with improvements ranging from less than 20% to 20% and more to less than 50% in ACR (5.01 points), whereas the greatest differences at week

56 were seen between ACR responder groups with improvements ranging from 20% and more to less than 50% to 50% and more to less than 70% in ACR (4.35 points).

1013

VALUE IN HEALTH 18 (2015) 1008–1015

(A)

Responsiveness of RA-WIS Total Change Score by DAS28 Remission Status DAS28 Remission

DAS28 Non-Remission

Baseline to Week 24***

0

Baseline to Week 56*

-2

WIS Total Change Scores

-4

-6

-6.5 -7.22

-8

-10 -10.7 -12

-11.64

-14 DAS28 Remission Status over 24 and 56 Weeks

Responsiveness of RA-WIS Total Change Scores by ACR Responder Status

(B)

<20%

≥20% to <50%

≥50% to <70%

Baseline to Week 24***

0

≥70%

Baseline to Week 56***

-2 -2.46

-2.73

WIS Total Change Scores

-4

-5.31

-6

-8

-10

-12

-7.74 -9.15

-9.66

-11.31

-11.44

-14 ACR Responder Status over 24 and 56 Weeks

Fig. 2. – Responsiveness of RA-WIS total change score by DAS28 remission status. DAS28, 28-Joint Disease Activity Score; RAWIS, Rheumatoid Arthritis-Work Instability Scale. RA-WIS total change scores from baseline to week 24 were significantly different based on DAS28 remission status, with larger changes seen for patients in remission (DAS28 o 2.6) than

in those not in remission (DAS28 Z 2.6; P o 0.001). Similar findings were seen for RA-WIS total change scores from baseline to week 56 (Fig. 2B) (P r 0.0001). Differences between remission

1014

VALUE IN HEALTH 18 (2015) 1008–1015

and nonremission groups were 5.14 points at week 24 and 3.48 points at week 56.

Interpretation guidelines Responder definitions can be developed on the basis of clinical data from these analyses. For example, there was a –5.3 to –7.7 point RA-WIS score change associated with an improvement of 20% to less than 50% in ACR from baseline to 24 or 56 weeks. There was a –5.4 to –6.1 point RA-WIS point change associated with a moderate response for DAS28 at 24 or 56 weeks. These are relatively large changes, given that the pooled baseline SD for the RA-WIS was 5.5 in the PROWD clinical trial [7]. Based on the current evidence, a responder definition of 5 points seems reasonable. For the responder analyses in the planned clinical trials, a decrease of 5 points in RA-WIS total scores will be used to determine responders.

Discussion Measuring risk for work disability (i.e., WI) provides important information from the patients’ perspective as to their potential to remain in paid work and their experience with treatments used to reduce the impact of RA on work outcomes. Psychometrically sound instruments are necessary for assessing this construct. This study expands on the initial assessment of measurement properties for the RA-WIS conducted by the developers of this instrument [19,23] and other researchers [22–24], providing further supportive evidence that the RA-WIS is a reliable and valid measure of WI in patients with RA. The RA-WIS total scores were responsive to changes in clinical severity based on the ACR criteria for improvement and DAS28 remission status experienced over 24 and 56 weeks. Based on the study findings, the RAWIS is an acceptable measure for assessing work disability risk in clinical studies of RA. Internal consistency reliability of the RA-WIS total score exceeded the recommended criterion of 0.70, considered sufficient for performing group-level comparisons [34,37], and demonstrates a strong relationship among the RA-WIS items. Testretest reliability evaluation demonstrated excellent agreement between RA-WIS total scores over 4 weeks in stable patients (ICC ¼ 0.91), suggesting that scores are replicable. These findings were consistent with previously reported internal consistency reliability of 0.92 [22] and test-retest reliability estimates of 0.89 [19]. Construct validity of the RA-WIS was supported by significant correlations between RA-WIS total scores and RA severity measures at baseline and 24 weeks. RA-WIS total scores were moderately to strongly correlated with clinical and PRO measures such as the HAQ-DI, Patient Global Assessment of Disease Activity, Patient Assessment of Pain, RAQoL, DAS28, TJC, and SJC, suggesting that clinical outcomes such as functional status and pain may have a substantial influence on work disability risk. The magnitude of the correlations for disease-oriented constructs (as described by Beaton et al.) is similar in both studies [22]. The magnitude of the correlations for work-oriented constructs, however, is lower for our study (0.09–0.38 absolute values) compared with those reported previously by Beaton et al. (0.54– 0.73 absolute values). This may be a reflection of how the construct of work impact was operationalized—where instruments used in the present study reflect self-report of the number of work hours lost per week (absenteeism) and instruments used in the previous study reflect self-report of productivity, performance, and satisfaction, which may be more closely associated with the construct assessed by the RA-WIS. Mean RA-WIS total score and work disability risk levels varied between clinical severity scores on the DAS28, HAQ-DI, and Physician Global Assessment of Disease Activity, providing

evidence further supporting construct validity. These results demonstrated that patients with worse clinical outcomes (i.e., more severe disease activity and functional disability) had higher risk for work disability. In addition, patients who lost more hours per week from work because of RA had higher work disability risk. The validity findings for the RA-WIS total score provide further evidence supporting construct validity [22], whereas findings for the work disability risk levels add new evidence regarding the validity of using these levels on the impact of RA on work outcomes. Results of this study suggest that the RA-WIS total score was responsive to clinically meaningful changes in disease activity, confirming and expanding on previous research [22]. On the basis of DAS28 and ACR criteria at weeks 24 and 56, patients who responded to therapy had greater improvements than did those who responded less well and nonresponders, indicating a relationship between changes in clinical outcomes and changes in work disability risk. Based on the ACR and the DAS28 results, it is recommended that an improvement of 5 points or more in RA-WIS total scores may be considered clinically meaningful. This responder definition needs to be confirmed in additional clinical trials in patients with RA. Several limitations need to be considered when interpreting the results of this study. First, no objective or independent indicators of work performance were included in the study. The evidence for construct validity was based on other PROs. Second, the study design allowed patients to discontinue treatment after 16 weeks if they experienced loss of employment, lack of efficacy, or adverse effects. It is very unlikely, however, that the absence of these patients from week 24 would affect the results of the psychometric analyses. Third, the psychometric analyses were based on a clinical trial sample, and therefore the results may not be generalizable to the general population of patients with RA. Clinical trial participants often have different demographic and clinical characteristics than do the general population of patients with early RA. The psychometric results from previous research in clinical practice settings are consistent with the reliability and validity findings reported in the present study [19,22,23].

Conclusions The results of this study suggest that the RA-WIS is reliable, valid, and responsive to change and may be useful in assessing the impact of disease and treatment on work disability risk in patients with early RA. WI assessment may be most valuable in clinical studies of interventions for early RA designed to prevent work disability and work loss. The psychometric evidence suggests that the RA-WIS may be an acceptable end point for clinical trials comparing treatment outcomes for patients with early RA. Source of financial support: The design, conduct, and financial support for this study were provided by AbbVie. AbbVie participated in the interpretation of data, review, and approval of the content of this presentation. R EF E R EN C ES

[1] Allaire S, Wolfe F, Niu J, et al. Evaluation of the effect of anti-tumor necrosis factor agent use on rheumatoid arthritis work disability: the jury is still out. Arthritis Care Res 2008;59:1082–9. [2] Sokka T. Work disability in early rheumatoid arthritis. Clin Exp Rheumatol 2003;21(Suppl):S71–4. [3] Centers for Disease Control and Prevention. About arthritis disabilities and limitations. 2012. Available from: http://www.cdc.gov/arthritis/ data_statistics/disabilities-limitations.htm. [Accessed September 30, 2014].

VALUE IN HEALTH 18 (2015) 1008–1015

[4] Sokka T, Kautiainen H, Pincus T, et al. Work disability remains a major problem in rheumatoid arthritis in the 2000s: data from 32 countries in the QUEST-RA study. Arthritis Res Ther 2010;12:R42. [5] Allaire S, Wolfe F, Niu J, et al. Contemporary prevalence and incidence of work disability associated with rheumatoid arthritis in the US. Arthritis Care Res 2008;59:474–80. [6] Sokka T, Kautiainen H, Möttönen T, et al. Work disability in rheumatoid arthritis 10 years after the diagnosis. J Rheumatol 1999;26:1681–5. [7] Bejarano V, Quinn M, Conaghan PG, et al. Effect of the early use of the anti-tumor necrosis factor adalimumab on the prevention of job loss in patients with early rheumatoid arthritis. Arthritis Rheum 2008;59:1467–74. [8] Breedveld FC, Weisman MH, Kavanaugh AF, et al. The PREMIER study: a multicenter, randomized, double-blind clinical trial of combination therapy with adalimumab plus methotrexate versus methotrexate alone or adalimumab alone in patients with early, aggressive rheumatoid arthritis who had not had previous methotrexate treatment. Arthritis Rheum 2006;54:26–37. [9] Fries JF, Williams CA, Morfeld D, et al. Reduction in long-term disability in patients with rheumatoid arthritis by disease-modifying antirheumatic drug-based treatment strategies. Arthritis Rheum 1996;39:616–22. [10] Goekoop-Ruiterman YP, de Vries-Bouwstra JK, Allaart CF, et al. Clinical and radiographic outcomes of four different treatment strategies in patients with early rheumatoid arthritis (the BeSt study): a randomized, controlled trial. Arthritis Rheum 2008;58(Suppl):S126–35. [11] Krishnan E, Fries JF. Reduction in long-term functional disability in rheumatoid arthritis from 1977 to 1998: a longitudinal study of 3035 patients. Am J Med 2003;115:371–6. [12] Quinn MA, Conaghan PG, O’Connor PJ, et al. Very early treatment with infliximab in addition to methotrexate in early, poor-prognosis rheumatoid arthritis reduces magnetic resonance imaging evidence of synovitis and damage, with sustained benefit after infliximab withdrawal: results from a twelve-month randomized, double-blind, placebo-controlled trial. Arthritis Rheum 2005;52:27–35. [13] Augustsson J, Neovius M, Cullinane-Carli C, et al. Patients with rheumatoid arthritis treated with tumour necrosis factor antagonists increase their participation in the workforce: potential for significant long-term indirect cost gains (data from a population-based registry). Ann Rheum Dis 2010;69:126–31. [14] Kimel M, Cifaldi M, Chen N, et al. Adalimumab plus methotrexate improved SF-36 scores and reduced the effect of rheumatoid arthritis (RA) on work activity for patients with early RA. J Rheumatol 2008;35:206–15. [15] Nordmark B, Blomqvist P, Andersson B, et al. A two-year follow-up of work capacity in early rheumatoid arthritis: a study of multidisciplinary team care with emphasis on vocational support. Scand J Rheumatol 2006;35:7–14. [16] Puolakka K, Kautiainen H, Mottonen T, et al. Early suppression of disease activity is essential for maintenance of work capacity in patients with recent-onset rheumatoid arthritis: five-year experience from the FIN-RACo trial. Arthritis Rheum 2005;52:36–41. [17] van Vollenhoven RF, Cifaldi MA, Ray S, et al. Improvement in work place and household productivity for patients with early rheumatoid arthritis treated with adalimumab plus methotrexate: work outcomes and their correlations with clinical and radiographic measures from a randomized controlled trial companion study. Arthritis Care Res (Hoboken) 2010;62:226–34.

1015

[18] Eriksson JK, Neovius M, Bratt J, et al. Biological vs conventional combination treatment and work loss in early rheumatoid arthritis: a randomized trial. JAMA Intern Med 2013;173:1407–14. [19] Gilworth G, Chamberlain MA, Harvey A, et al. Development of a work instability scale for rheumatoid arthritis. Arthritis Rheum 2003;49:349–54. [20] Gilworth G, Woodhouse A, Tennant A, et al. The impact of rheumatoid arthritis in the workplace. Br J Ther Rehabil 2001;8:342–7. [21] Tang K, Beaton DE, Boonen A, et al. Measures of work disability and productivity: Rheumatoid Arthritis Specific Work Productivity Survey (WPS-RA), Workplace Activity Limitations Scale (WALS), Work Instability Scale for Rheumatoid Arthritis (RA-WIS), Work Limitations Questionnaire (WLQ), and Work Productivity and Activity Impairment Questionnaire (WPAI). Arthritis Care Res (Hoboken) 2011;63(Suppl. 11):S337–49. [22] Beaton DE, Tang K, Gignac MA, et al. Reliability, validity, and responsiveness of five at-work productivity measures in patients with rheumatoid arthritis or osteoarthritis. Arthritis Care Res (Hoboken) 2010;62:28–37. [23] Gilworth G, Emery P, Gossec L, et al. Adaptation and cross-cultural validation of the rheumatoid arthritis work instability scale (RA-WIS). Ann Rheum Dis 2009;68:1686–90. [24] Tang K, Beaton DE, Gignac MA, et al. The Work Instability Scale for rheumatoid arthritis predicts arthritis-related work transitions within 12 months. Arthritis Care Res (Hoboken) 2010;62:1578–87. [25] Tang K, Beaton DE, Lacaille D, et al. Sensibility of five at-work productivity measures was endorsed by patients with osteoarthritis or rheumatoid arthritis. J Clin Epidemiol 2013;66:546–56. [26] Arnett FC, Edworthy SM, Bloch DA, et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum 1988;31:315–24. [27] Fries JF, Ramey DR. “Arthritis specific” global health analog scales assess “generic” health related quality-of-life in patients with rheumatoid arthritis. J Rheumatol 1997;24:1697–702. [28] Kosinski M, Zhao SZ, Dedhiya S, et al. Determining minimally important changes in generic and disease-specific health-related quality of life questionnaires in clinical trials of rheumatoid arthritis. Arthritis Rheum 2000;43:1478–87. [29] Wells GA, Tugwell P, Kraag GR, et al. Minimum important difference between patients with rheumatoid arthritis: the patient’s perspective. J Rheumatol 1993;20:557–60. [30] Whalley D, McKenna SP, de Jong Z, et al. Quality of life in rheumatoid arthritis. Br J Rheumatol 1997;36:884–8. [31] Fransen J, van Riel PL. The Disease Activity Score and the EULAR response criteria. Clin Exp Rheumatol 2005;23(Suppl):S93–9. [32] Disease activity score and DAS28. 2011. Available from: http://www. das-score.nl/das28/en/. [Accessed March 26, 2015]. [33] Felson DT, Anderson JJ, Boers M, et al. American College of Rheumatology. Preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum 1995;38:727–35. [34] Hays RD, Revicki DA. Assessing reliability and validity (including responsiveness). In: Fayers PM, Hays RD, eds. Assessing Quality of Life in Clinical Trials (2nd ed.) Oxford: Oxford University Press, 2005. [35] Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Erlbaum, 1988. [36] Revicki D, Hays RD, Cella D, et al. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102–9. [37] Nunnally J, Bernstein I. Psychometric Theory. New York: McGraw-Hill, 1994.