- Email: [email protected]
Validation of the Spine Oncology Study Group—Outcomes Questionnaire to assess quality of life in patients with metastatic spine disease
Accepted Manuscript Title: Validation of the Spine Oncology Study Group Outcomes Questionnaire to assess quality of life in patients with metastatic spine disease Author: Stein J. Janssen, Teun Teunis, Eva van Dijk, Marco L. Ferrone, John H. Shin, Francis Hornicek, Joseph H. Schwab PII: DOI: Reference:
S1529-9430(15)01197-3 http://dx.doi.org/doi: 10.1016/j.spinee.2015.07.456 SPINEE 56497
To appear in:
The Spine Journal
Received date: Revised date: Accepted date:
9-2-2015 22-5-2015 23-7-2015
Please cite this article as: Stein J. Janssen, Teun Teunis, Eva van Dijk, Marco L. Ferrone, John H. Shin, Francis Hornicek, Joseph H. Schwab, Validation of the Spine Oncology Study Group Outcomes Questionnaire to assess quality of life in patients with metastatic spine disease, The Spine Journal (2015), http://dx.doi.org/doi: 10.1016/j.spinee.2015.07.456. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1
Validation of the Spine Oncology Study Group Outcomes Questionnaire to assess quality of
2
life in patients with metastatic spine disease
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Stein J. Janssen, MD1 Research Fellow [email protected] Teun Teunis, MD1 Research Fellow [email protected] Eva van Dijk, BSc1 Graduate research assistant [email protected] Marco L. Ferrone, MD3 Spine surgeon [email protected] John H. Shin, MD2 Spine surgeon [email protected] Francis Hornicek, MD, MS, PhD1 Chief Orthopaedic Oncology Service [email protected] Joseph H. Schwab, MD, MS1 Instructor in Orthopaedic Surgery [email protected] Affiliation address: 1) Department of Orthopaedic Surgery, Orthopaedic Oncology Service, Massachusetts General Hospital – Harvard Medical School, 55 Fruit Street, Boston, Massachusetts, 02114, United States of America 2) Department of Neurosurgery, Massachusetts General Hospital – Harvard Medical School, 55 Fruit Street, Boston, Massachusetts, 02114, United States of America 3) Department of Orthopaedic Surgery, Spine Service, Brigham and Women’s Hospital – Harvard Medical School, 75 Francis Street, Boston, MA 02115, United States of America Corresponding author & request for reprints: Stein J. Janssen, MD
1
Page 1 of 30
1 2 3 4 5 6
Address: Room 3.946, Yawkey building, Massachusetts General Hospital 55 Fruit Street, Boston, MA 02114 Telephone: +1-617-726-1569 Fax: +1-617-643-1274 E-Mail: [email protected] Abstract:
7
Background context: General questionnaires are often used to assess quality of life in patients
8
with spine metastases, while a disease-specific survey did not exist until recently. The Spine
9
Oncology Study Group has developed a questionnaire (SOSG-OQ) to measure quality of life in
10
these patients. However, a scoring system was not developed and the questionnaire was not
11
validated in a group of patients, nor was it compared to other general quality of life
12
questionnaires such as the EuroQol 5 Dimensions (EQ-5D) questionnaire.
13
Purpose: Our primary null hypothesis is that there is no association between the SOSG-OQ and
14
EQ-5D. Our secondary null hypothesis is that there is no difference in coverage and internal
15
consistency between the SOSG-OQ and EQ-5D. We also assess coverage, consistency, and
16
validity of the domains within the SOSG-OQ.
17
Study Design/Setting: Survey study from a tertiary care spine referral center.
18
Patient Sample: 82 patients with spine metastases, myeloma, or lymphoma.
19
Outcome Measures: The SOSG-OQ (27 questions, 6 domains) score ranges from 0 to 80 with a
20
higher score indicating worse quality of life. The EQ-5D (5 questions, 5 domains) index score
21
ranges from 0 to 1 with a higher score indicating better quality of life.
22
Methods: The association between the SOSG-OQ and EQ-5D index score was assessed using
23
the Spearman rank correlation. Instrument coverage and precision were assessed by determining
24
item completion rate, median score with range, and floor and ceiling effect. Internal consistency
25
was assessed using Cronbach alpha. Multitrait analysis and exploratory factor analysis were used
2
Page 2 of 30
1
to analyze properties of the individual domains in the SOSG-OQ. No grants or funds were
2
received for this study.
3
Results: The Spearman rank correlation between the SOSG-OQ and EQ-5D questionnaire was
4
high (r = -0.83, P < 0.001). Internal consistency of the SOSG-OQ (0.92, 95% CI: 0.89 – 0.94)
5
was higher as compared to the internal consistency of the EQ-5D (0.73, 95% CI: 0.63 – 0.84; P <
6
0.001). The SOSG-OQ score had no floor or ceiling effect indicating good coverage (median 30,
7
range 3 – 64), while the EQ-5D had a ceiling effect of 10% (median 0.71, range 0.05 – 1).
8
Conclusions: In conclusion, our study proposes a scoring methodology –after reversing 4
9
inversely scored items– for the SOSG-OQ and demonstrates that the questionnaire is a valid tool
10
for the assessment of quality of life in patients with metastatic spine disease. The SOSG-OQ is
11
superior to the EQ-5D in terms of coverage and internal consistency, but consists of more
12
questions.
13
Keywords: spine oncology study group; quality of life; metastases; spine; validity; eq-5d;
14
questionnaire; survey
15
Abbreviations: SOSG-OQ: Spine Oncology Study Group – Outcomes Questionnaire; EQ-5D:
16
EuroQol 5 Dimensions questionnaire.
17 18 19
Introduction
20
Quality of life is an important outcome measure for patients with metastatic spine disease.1-5
21
Some advocate the use of general quality of life questionnaires to allow for comparison across
22
diseases and calculation of cost effectiveness.5 Others prefer disease-specific tools as these
23
discriminate better between patients, are more sensitive to change within a patient, are easier to
3
Page 3 of 30
1
understand for a patient, and result in better coverage of the construct being measured.6-9 A
2
systematic review by Street et al. demonstrated that many different tools are used to assess
3
outcome in patients with metastatic spine disease.6 However, most tools are clinician reported
4
outcomes and measure only one aspect of quality of life (e.g. ECOG, Frankel scale scores, or
5
ambulatory status).6,10 Only a few studies used patient reported quality of life outcome measures
6
(e.g. EQ-5D, SF-36, QLQ-C30).5,6,11,12 Patient reported outcome measures should be the standard
7
as clinicians’ understanding of the effect of disease on patients’ daily lives is poor.13,14 While
8
general questionnaires have been used to assess quality of life metrics in patients with metastatic
9
spine disease, a disease-specific survey did not exist until recently.5-7
10 11
The Spine Oncology Study Group (SOSG) –an international group of experts– has developed an
12
Outcome Questionnaire to measure quality of life in patients with metastatic spine disease.6 The
13
questionnaire was developed through consensus discussion of the SOSG and followed by a
14
systematic evaluation to examine if all quality of life domains were adequately represented.6
15
Independent health professionals rated the individual questions of the Spine Oncology Study
16
Group Outcomes Questionnaire (SOSG-OQ; Table 1) using the International Classification of
17
Functioning, Disability, and Health (ICF) framework.6,15 The ICF belongs to the World Health
18
Organization’s family of international health classifications and is a framework for measuring
19
health and disability, it is not a measuring instrument itself.15 Analysis of the SOSG-OQ using
20
the ICF framework demonstrated good representation of all 4 domains specified by the ICF to be
21
relevant for measurement of function and disability (body structures, body function, activity and
22
participation, and environmental factors).6 However, a scoring system was not developed and the
4
Page 4 of 30
1
questionnaire was not validated in a group of patients, nor was it compared to other general
2
quality of life questionnaires.
3 4
In this study, we compare the disease-specific SOSG-OQ to the generic EuroQol 5 Dimensions
5
(EQ-5D) quality of life questionnaire in patients with metastatic spine disease. Our primary null
6
hypothesis is that there is no association between the SOSG-OQ and EQ-5D. Our secondary null
7
hypothesis is that there is no difference in coverage –floor and ceiling effect– and internal
8
consistency between the SOSG-OQ and EQ-5D. We also assess coverage, internal consistency,
9
and validity of the domains within the SOSG-OQ.
10 11
Methods
12
Study Design
13
We analyzed a convenience sample of 82 patients with spine metastases, myeloma, or
14
lymphoma, who completed the SOSG-OQ and EQ-5D questionnaires for a quality improvement
15
program at our Orthopaedic Spine Oncology unit. All patients presenting to our outpatient clinic
16
are asked to complete these questionnaires independently before visiting with the surgeon. Data
17
is collected through REDCap (Vanderbilt University, Nashville, Tennessee, United States of
18
America) on a tablet computer.16 REDCap is an online data collection tool that enables
19
researchers to create study-specific surveys to capture participant data securely online.16 One
20
hundred ninety-six patients with metastases, myeloma, or lymphoma of the spine encountered
21
our Orthopaedic Spine Oncology unit between September 2011 and November 2014 and were
22
eligible for this study. Patients were included regardless of previous treatment and disease stage.
23
Eighty-five (43%) patients completed the SOSG-OQ and EQ-5D questionnaire. Three (3.5%)
5
Page 5 of 30
1
patients were excluded because they had more than one missing item in one of the SOSG-OQ
2
domains or in the EQ-5D questionnaire. None of the patients were below 18 years of age.
3
Eighty-two patients remained for analysis, we only included a patients’ first survey to avoid a
4
learning curve on questionnaire completion. Secondary use of data –for the purpose of this
5
study– was approved by our institutional review board and a waiver of informed consent was
6
obtained.
7 8 9
Outcome measures
10
Baseline characteristics were obtained from patients’ medical records and consisted of: age at
11
time of survey completion, sex, primary tumor type, affected anatomical location, presence of
12
other bone metastases and visceral metastases, presence of pain (back, neck, or radiating pain),
13
symptoms of neurological compromise (weakness, sensory deficit, or bowel and bladder
14
dysfunction), Eastern Cooperative Oncology Group (ECOG) performance status, prior surgery,
15
radiotherapy, and systemic therapy for metastatic spine disease, myeloma, or lymphoma.
16 17
The SOSG-OQ consists of 27 questions divided into 6 domains: physical function (4 items),
18
neurological function (4 items), pain (4 items), mental health (4 items), social function (4 items),
19
and post therapy questions (7 items) (Table 1).6 All questions have 5 answer options.6 Post
20
therapy questions are only completed if the patient indicates being post treatment for the spine
21
disease. We scored individual items within the SOSG-OQ on a scale from 0 to 4. Items 12, 15,
22
19, and 20 were scored inversely due to their reversed nature (Table 1). A total score was
23
calculated per domain by summing the points of individual items within the domain. The score
6
Page 6 of 30
1
per domain ranges from 0 to 16, except for the post therapy domain score which ranges from 0 to
2
28. The overall score of the SOSG-OQ was calculated by summing the points of all items,
3
excluding the post therapy questions, resulting in a score ranging from 0 to 80 with a higher
4
score indicating worse quality of life. We excluded the post therapy questions from calculation
5
of the SOSG-OQ quality of life score as these items pertain to the provided treatment. Including
6
these in the overall SOSG-OQ score renders it difficult to compare pre- and post treatment
7
quality of life scores. This domain seems to be more useful in assessing success of the treatment.
8 9
The EQ-5D is developed by the EuroQol –an international collaboration of multidisciplinary
10
researchers developing instruments that describe and value health– and is a commonly used
11
instrument to assess quality of life in 5 dimensions (mobility, self-care, usual activities, pain-
12
discomfort, and anxiety-depression) through 5 questions.17 We used the English version for the
13
United States with 3 answer options for every question (EQ-5D 3L).18 An overall EQ-5D index
14
for those who completed all 5 questions (n = 77) was calculated using an algorithm described by
15
Shaw et al.19 The EQ-5D index score ranges from 0 to 1 with a higher score indicating better
16
quality of life.
17 18
Statistical analysis
19
Categorical variables are demonstrated as frequencies with percentages, and continuous variables
20
as median with interquartile range (IQR) as histograms suggested nonparametric distribution of
21
continuous variables.
22
7
Page 7 of 30
1
We assessed the association between the overall SOSG-OQ and EQ-5D index score using the
2
Spearman rank correlation to determine the shared variance. The Spearman rank correlation
3
coefficient ranges from -1 to 1; 1 indicating a perfect correlation, 0 indicating no correlation, and
4
-1 indicating a perfect inverse correlation.
5 6
Coverage, precision, and reliability were assessed for the SOSG-OQ, its domains, and the EQ-
7
5D questionnaire. Instrument coverage and precision were assessed by determining item
8
completion rate, median score with range, and floor and ceiling effect. Floor effect is a term used
9
to describe a situation in which subjects in a study have scores at the lowest possible limit
10
(floor); ceiling effect reflects scores at the upper limit (ceiling). These effects influence the
11
analyses because it can skew results and reduce variation in a questionnaire.20 Reliability was
12
assessed using Cronbach alpha; a measure of internal consistency ranging from 0 to 1, with a
13
higher score indicating higher internal consistency. A Cronbach alpha above 0.70 is
14
recommended; however, a high Cronbach alpha (above 0.90) might indicate question
15
redundancy.21,22 We used bootstrapping (number of resamples: 1000) to calculate the standard
16
error, z statistic, p value, and 95% confidence intervals around the Cronbach alpha values to
17
compare the internal consistency of the EQ-5D and SOSG-OQ.
18 19
Multitrait analysis was used to assess convergent and divergent validity of items within the
20
domains of the SOSG-OQ.23 These measures demonstrate the degree to which the domain
21
measures what it claims to be measuring. The convergent validity of each item was assessed
22
using the Spearman rank correlation coefficient of every item with its own domain score. The
23
convergent validity was considered satisfactory if the correlation coefficient exceeded 0.40.24 For
8
Page 8 of 30
1
the divergent validity, the correlation coefficient between each item and its own domain score
2
should be higher than the correlation with other domain scores. We assessed the item-total
3
Spearman rank correlation, indicating the association of the item with the overall SOSG-OQ
4
score. An item-total correlation below 0.40 was considered insufficient.25 Inter-item Spearman
5
rank correlations were assessed to determine if items were redundant to each other. We
6
considered items with an inter-item correlation above 0.80 to be redundant.
7 8
An exploratory factor analysis was performed to assess underlying traits of the SOSG-OQ.
9
Exploratory factor analysis is a method that looks for underlying traits (unobserved) being
10
measured by items (observed) within a questionnaire and subsequently analyzes the correlation
11
(i.e. factor loading) of these items with the identified traits. We allowed the exploratory factor
12
analysis to retain 5 factors and assessed if items were grouped in accordance with the 5
13
predefined domains. The factor structure was rotated using varimax rotation to optimize the
14
factor loadings among the 5 factors. A factor loading of 1 indicates perfect association of the
15
item with the underlying trait, 0 indicates no association, and -1 indicates perfect inverse
16
association of the item with the underlying trait.
17 18
A two-sided P value below 0.05 was considered to indicate statistical significance. Stata 12
19
(StataCorp LP, Texas, USA) was used for statistical analyses.
20 21
Sample size calculation
9
Page 9 of 30
1
A sample size calculation showed that a minimum sample size of 82 patients would provide 80%
2
statistical power (beta 0.20; alpha 0.05) to detect a correlation of 0.30 between the SOSG-OQ
3
and EQ-5D.
4 5
Baseline characteristics
6
There were 41 (50%) men, and 41 women, median age was 59 years (Table 2). The five most
7
common primary tumors were breast (18 of 82, 22%), lung (9 of 82, 11%), kidney (7 of 82,
8
8.5%), multiple myeloma (6 of 82, 7.3%), and colon/rectum (4 of 82, 4.9%). The thoracic spine
9
was most commonly affected (50 of 82, 61%). Twenty-three patients (28%) had multiple spine
10
regions affected. Forty patients (49%) had other –non-spinal– bone metastases, 37 (45%) had
11
visceral metastases (Table 2). Most patients had pain (66 of 82, 81%), 33 (40%) had neurological
12
compromise, and there was substantial variability in ECOG performance status (Table 2).
13 14
Funding
15
Departmental
16 17
Results
18
Comparing the SOSG-OQ and EQ-5D
19
The Spearman rank correlation between the SOSG-OQ and EQ-5D questionnaire was substantial
20
(r = -0.83, P < 0.001).
21
10
Page 10 of 30
1
Internal consistency of the overall SOSG-OQ (0.92, 95% CI: 0.89 – 0.94) was higher as
2
compared to the internal consistency of the EQ-5D (0.73, 95% CI: 0.63 – 0.84) (P < 0.001)
3
(Table 3).
4 5
The overall SOSG-OQ score had no floor or ceiling effect (range 3 to 64; possible range 0 to 80)
6
indicating good coverage, while the EQ-5D had a ceiling effect of 10% (8 of 82) (Table 2).
7 8
Assessing coverage, reliability and validity of the SOSG-OQ and its domains
9
Internal consistency as determined by Cronbach alpha per domain ranged from 0.54
10
(neurological function) to 0.86 (physical function) (Table 3).
11 12
Every domain within the SOSG-OQ had a floor effect, which was largest in the neurological
13
function domain (22 of 82 patients had a score of 0, 27%) (Table 3). Ceiling effect only occurred
14
in the Pain domain (3 of 82 patients had a score of 16, 3.7%).
15 16
Convergent validity was above the threshold of 0.40 for all items (Table 4). Item 7 (What
17
assistance do you need with your walking?) was the only item with a higher other domain (r =
18
0.71, physical function) than own domain (r = 0.65, neurological function) correlation (Table 4).
19 20
Item 5 (Do you have weakness in your legs?) and 8 (Do you have difficulty controlling your
21
bowel and bladder function beyond episodes of diarrhea / constipation?) did not have
22
satisfactory (r < 0.40) item-total correlations (Table 4). All inter-item correlations were below
23
the 0.80 threshold indicating no redundancy of items (Appendix 1). Item 3 (Does your spine limit
11
Page 11 of 30
1
your ability to care for yourself?) and item 4 (Do you require assistance from others to travel
2
outside of the home?) had the highest inter-item correlation (r = 0.74), item 2 (What is your
3
ability to work/study?) and item 8 (Do you have difficulty controlling your bowel and bladder
4
function beyond episodes of diarrhea/constipation?) had the lowest inter-item correlation (r =
5
0.037) (Appendix 1). A distribution of responses to individual items are demonstrated in
6
appendix 2.
7 8
Factor analysis demonstrated underlying traits corresponding with the physical function,
9
neurological function, and pain domains. Except for item 7 within the neurological function
10
domain (What assistance do you need with your walking?), which was grouped with the physical
11
function domain items. The items within the mental health and social function domains
12
demonstrated associations with a variety of underlying traits (Table 5).
13 14
Discussion
15
The questionnaire developed by the Spine Oncology Study Group to assess quality of life in
16
patients with metastatic spine disease has not been validated in a sample of patients.6 This study
17
tested the SOSG-OQ and compared it to the generic EQ-5D questionnaire in patients with
18
metastatic spine disease. We found a substantial correlation between the questionnaires
19
indicating that they measure the same construct: quality of life. The SOSG-OQ was superior to
20
the EQ-5D in terms of internal consistency and coverage.
21 22
The high inverse correlation between the SOSG-OQ and EQ-5D indicates that they have
23
substantial shared variance and that an increase in the SOSG-OQ (higher score indicates worse
12
Page 12 of 30
1
quality of life) corresponds to a decrease in EQ-5D (higher score indicates better quality of life)
2
and vice versa. Both questionnaires reached the recommended threshold for sufficient internal
3
consistency, but the internal consistency of the SOSG-OQ was higher as compared to the EQ-
4
5D.21,22 However, the internal consistency of the SOSG-OQ (0.92) might indicate item
5
redundancy.21,22 Additional inter-item correlation analysis within the SOSG-OQ refuted possible
6
item redundancy as all correlations were below the 0.80 threshold. The absence of floor and
7
ceiling effects in the overall SOSG-OQ score suggests good coverage, meaning that the full
8
spectrum of quality of life in patients with spine metastatic disease can be measured through this
9
questionnaire. The issue of the ceiling effect in the EQ-5D index score has been demonstrated in
10
previous studies and suggests that the questionnaire is not capable of distinguishing between
11
health states close to full health.26-28 Based on these findings, the SOSG-OQ seems to have better
12
coverage and higher reliability to assess quality of life in patients with metastatic spine disease.
13 14
Further analysis of the SOSG-OQ demonstrated sufficient internal consistency, good coverage,
15
and convergent and divergent validity of all domains except the neurological function domain.
16
The neurological function domain had a substantial floor effect, low internal consistency, two
17
items with low correlations with the overall SOSG-OQ score, and one item that belonged to a
18
different domain. The floor effect can be explained by absence of neurological compromise in
19
many patients with metastatic spine disease; 49 of the 82 (60%) patients in our cohort had no
20
symptoms of neurological compromise.10,29,30 Low internal consistency can be a result of poor
21
question phrasing and multidimensionality. Item 7 (What assistance do you need with your
22
walking?) seems to belong to the physical function domain as demonstrated by the multitrait and
23
exploratory factor analysis. Item 8 (Do you have difficulty controlling your bowel and bladder
13
Page 13 of 30
1
function beyond episodes of diarrhea / constipation?) assesses multiple issues but only allows
2
for one answer (also called a double-barreled question). Reconsidering the content of the
3
neurological function domain and rephrasing item 8 could further improve the reliability of the
4
SOSG-OQ.
5 6
Items in the mental health and social function domains demonstrate associations with a variety of
7
underlying traits. This might be explained by the complex interplay of psychological factors with
8
physical function and pain.31-34
9 10
This study has several limitations. First, patients only completed the questionnaire once and we
11
did not assess test-retest reliability. We see this as a minor limitation as the concept being
12
measured –quality of life– might change rapidly in patients with metastatic cancer, myeloma, or
13
lymphoma, rendering test-retest reliability analysis less valid.35 Second, patients completed the
14
questionnaire in the outpatient clinic; we did not include a subset of patients admitted to the
15
hospital, presenting to the emergency room, or to a hospice with severe illness. Furthermore, we
16
analyzed a convenience sample of 82 patients while 196 potentially eligible patients encountered
17
our outpatient clinic. This might have resulted in selection bias and our sample might therefore
18
not be representative for all patients with spine metastatic disease. This could have influenced
19
the coverage –floor and/or ceiling effect– of the SOSG-OQ and EQ-5D. However, we believe
20
that this did no compromise the correlation, internal consistency, exploratory factor analysis, or
21
multitrait analysis as our sample was heterogeneous with regards to baseline characteristics –
22
tumor type, systemic load, ECOG performance status, pain, and neurological compromise– and
23
outcome. The SOSG-OQ therefore seems to be useful for measuring quality of life in patients
14
Page 14 of 30
1
with metastatic spine disease, but its coverage needs to be assessed in future studies. Third, the
2
number of patients included did not allow us to use the more robust item response theory (IRT)
3
analysis. IRT analysis is considered superior to the classical test theory –Cronbach alpha,
4
multitrait analysis, and exploratory factor analysis– in assessing psychometric properties of a
5
questionnaire.36,37 Future studies with larger numbers should apply this methodology to further
6
analyze validity of the SOSG-OQ. The ultimate goal should be the development of a Computer
7
Adaptive Testing item-bank which offers a dynamic selection of items wherein the response to
8
each item guides the system's choice of the next item, resulting in an individually tailored series
9
of questions.38,39 This reduces questionnaire burden, improves coverage, and maintain reliability
10
over the full-spectrum of the construct being measured (quality of life).38,39 Fourth, we compared
11
the SOSG-OQ with the short (5-question) EQ-5D to assess its face validity; however, a more
12
comprehensive comparison with other quality of life questionnaires –such as the QLQ-C30 and
13
PROMIS Global Health questionnaires– helps establish which questionnaire is optimal in terms
14
of reliability, questionnaire burden, and coverage to assess quality of life in these patients.40,41
15 16
In conclusion, our study proposes a scoring methodology –after reversing 4 inversely scored
17
items– for the SOSG-OQ and demonstrates that the questionnaire is a valid and reliable tool for
18
the assessment of quality of life in patients with metastatic spine disease, myeloma, or
19
lymphoma. The SOSG-OQ is superior to the EQ-5D in terms of coverage and reliability, but
20
consists of more questions.
21 22
15
Page 15 of 30
1
References
2
1.
Wai EK, Finkelstein JA, Tangente RP, Holden L, Chow E, Ford M, Yee A. Quality of
3
life in surgical treatment of metastatic spine disease. Spine (Phila Pa 1976). Mar 1
4
2003;28(5):508-512.
5
2.
Valesin Filho ES, de Abreu LC, Lima GH, de Cubero DI, Ueno FH, Figueiredo GS,
6
Valenti VE, Monteiro CB, Wajnsztejn R, Fujiki EN, Neto MR, Rodrigues LM. Pain and
7
quality of life in patients undergoing radiotherapy for spinal metastatic disease treatment.
8
Int Arch Med. 2013;6(1):6.
9
3.
Dea N, Versteeg A, Fisher C, Kelly A, Hartig D, Boyd M, Paquette S, Kwon BK, Dvorak
10
M, Street J. Adverse events in emergency oncological spine surgery: a prospective
11
analysis. J Neurosurg Spine. Nov 2014;21(5):698-703.
12
4.
Sutcliffe P, Connock M, Shyangdan D, Court R, Kandala NB, Clarke A. A systematic
13
review of evidence on malignant spinal metastases: natural history and technologies for
14
identifying patients at high risk of vertebral fracture and spinal cord compression. Health
15
Technol Assess. Sep 2013;17(42):1-274.
16
5.
Choi D, Morris S, Crockard A, Albert T, Bunger C, Fehlings M, Harrop J, Kawahara N,
17
Martin JA, Massicotte EM, Mazel C, Oner FC, Peul W, Tomita K, Wang M. Assessment
18
of quality of life after surgery for spinal metastases: position statement of the Global
19
Spine Tumour Study Group. World Neurosurg. Dec 2013;80(6):e175-179.
20
6.
Street J, Lenehan B, Berven S, Fisher C. Introducing a new health-related quality of life
21
outcome tool for metastatic disease of the spine: content validation using the International
22
Classification of Functioning, Disability, and Health; on behalf of the Spine Oncology
23
Study Group. Spine (Phila Pa 1976). Jun 15 2010;35(14):1377-1386.
16
Page 16 of 30
1
7.
Street J, Berven S, Fisher C, Ryken T. Health related quality of life assessment in
2
metastatic disease of the spine: a systematic review. Spine (Phila Pa 1976). Oct 15
3
2009;34(22 Suppl):S128-134.
4
8.
de Vries M, Ouwendijk R Fau - Kessels AG, Kessels Ag Fau - de Haan MW, de Haan
5
Mw Fau - Flobbe K, Flobbe K Fau - Hunink MGM, Hunink Mg Fau - van Engelshoven
6
JMA, van Engelshoven Jm Fau - Nelemans PJ, Nelemans PJ. Comparison of generic and
7
disease-specific questionnaires for the assessment of quality of life in patients with
8
peripheral arterial disease. 20050315 DCOM- 20050506 (0741-5214 (Print)).
9
9.
Heniford BT, Walters Al Fau - Lincourt AE, Lincourt Ae Fau - Novitsky YW, Novitsky
10
Yw Fau - Hope WW, Hope Ww Fau - Kercher KW, Kercher KW. Comparison of generic
11
versus specific quality-of-life scales for mesh hernia repairs. 20080404 DCOM-
12
20080422 (1879-1190 (Electronic)).
13
10.
Patchell RA, Tibbs PA, Regine WF, Payne R, Saris S, Kryscio RJ, Mohiuddin M, Young
14
B. Direct decompressive surgical resection in the treatment of spinal cord compression
15
caused by metastatic cancer: a randomised trial. Lancet. Aug 20-26 2005;366(9486):643-
16
648.
17
11.
Miscusi M, Polli FM, Forcato S, Ricciardi L, Frati A, Cimatti M, De Martino L, Ramieri
18
A, Raco A. Comparison of minimally invasive surgery with standard open surgery for
19
vertebral thoracic metastases causing acute myelopathy in patients with short- or mid-
20
term life expectancy: surgical technique and early clinical results. J Neurosurg Spine.
21
May 2015;22(5):518-525.
17
Page 17 of 30
1
12.
Pflugmacher R, Taylor R, Agarwal A, Melcher I, Disch A, Haas NP, Klostermann C.
2
Balloon kyphoplasty in the treatment of metastatic disease of the spine: a 2-year
3
prospective evaluation. Eur Spine J. Aug 2008;17(8):1042-1048.
4
13.
5 6
Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. 2015;350:g7818.
14.
Nelson E, Conger B, Douglass R, Gephart D, Kirk J, Page R, Clark A, Johnson K, Stone
7
K, Wasson J, Zubkoff M. Functional health status levels of primary care patients. JAMA.
8
Jun 24 1983;249(24):3331-3338.
9
15.
10 11
Organization WH. International Classification of Functioning, Disability and Health (ICF)2001.
16.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic
12
data capture (REDCap)--a metadata-driven methodology and workflow process for
13
providing translational research informatics support. J Biomed Inform. Apr
14
2009;42(2):377-381.
15
17.
16 17
Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med. Jul 2001;33(5):337-343.
18.
Janssen MF, Pickard AS, Golicki D, Gudex C, Niewada M, Scalone L, Swinburn P,
18
Busschbach J. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L
19
across eight patient groups: a multi-country study. Qual Life Res. Sep 2013;22(7):1717-
20
1727.
21 22
19.
Shaw JW, Johnson JA, Coons SJ. US valuation of the EQ-5D health states: development and testing of the D1 valuation model. Med Care. Mar 2005;43(3):203-220.
18
Page 18 of 30
1
20.
Delucchi KL, Bostrom A. Methods for analysis of skewed data distributions in
2
psychiatric clinical studies: working with many zero values. Am J Psychiatry. Jul
3
2004;161(7):1159-1168.
4
21.
5
Panayides P. Coefficient Alpha Interpret With Caution. Europe's Journal of Psychology. 2013;9(4):687-696.
6
22.
Nunnally JC. Psychometric theory 2nd ed. ed. New York, NY: McGraw-Hill; 1978.
7
23.
Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-
8 9
multimethod matrix. Psychol Bull. Mar 1959;56(2):81-105. 24.
Panouilleres M, Anota A, Nguyen TV, Bredart A, Bosset JF, Monnier A, Mercier M,
10
Hardouin JB. Evaluation properties of the French version of the OUT-PATSAT35
11
satisfaction with care questionnaire according to classical and item response theory
12
analyses. Qual Life Res. Mar 7 2014.
13
25.
Gliem JA, Gliem RR. Calculating, Interpreting, and Reporting Cronbach’s Alpha
14
Reliability Coefficient for Likert-Type Scales. Paper Presented at the Midwest Research-
15
to-Practice Conference in Adult, Continuing, and Community Education. 2003.
16
26.
17 18
Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Econ. Sep 2004;13(9):873-884.
27.
Soer R, Reneman MF, Speijer BL, Coppes MH, Vroomen PC. Clinimetric properties of
19
the EuroQol-5D in patients with chronic low back pain. Spine J. Nov 2012;12(11):1035-
20
1039.
21
28.
Bharmal M, Thomas J, 3rd. Comparing the EQ-5D and the SF-6D descriptive systems to
22
assess their ceiling effects in the US general population. Value Health. Jul-Aug
23
2006;9(4):262-271.
19
Page 19 of 30
1
29.
Singh K, Samartzis D, Vaccaro AR, Andersson GB, An HS, Heller JG. Current concepts
2
in the management of metastatic spinal disease. The role of minimally-invasive
3
approaches. J Bone Joint Surg Br. Apr 2006;88(4):434-442.
4
30.
Saengnipanthkul S, Jirarattanaphochai K, Rojviroj S, Sirichativapee W, Mahakkanukrauh
5
C. Metastatic adenocarcinoma of the spine. Spine (Phila Pa 1976). Apr 1992;17(4):427-
6
430.
7
31.
8 9
He J, Xiong W, Li F, Luo W, Gao S. Depression influences pain and function after cervical disc arthroplasty. J Neurosurg Sci. Sep 12 2014.
32.
Hirakawa Y, Hara M, Fujiwara A, Hanada H, Morioka S. The relationship among
10
psychological factors, neglect-like symptoms and postoperative pain after total knee
11
arthroplasty. Pain Res Manag. Sep-Oct 2014;19(5):251-256.
12
33.
13 14
Nota SP, Bot AG, Ring D, Kloen P. Disability and depression after orthopaedic trauma. Injury. Jun 21 2014.
34.
Vranceanu AM, Bachoura A, Weening A, Vrahas M, Smith RM, Ring D. Psychological
15
factors predict disability and pain intensity after skeletal trauma. J Bone Joint Surg Am.
16
Feb 5 2014;96(3):e20.
17
35.
18 19
Davidshofer KR, Murphy CO. Psychological testing : principles and applications. 6 ed: Upper Saddle River, N.J.: Pearson/Prentice Hall; 2005.
36.
Petrillo J, Cano SJ, McLeod LD, Coon CD. Using classical test theory, item response
20
theory, and rasch measurement theory to evaluate patient-reported outcome measures: a
21
comparison of worked examples. Value Health. Jan 2015;18(1):25-34.
20
Page 20 of 30
1
37.
Cappelleri JC, Jason Lundy J, Hays RD. Overview of classical test theory and item
2
response theory for the quantitative assessment of items in developing patient-reported
3
outcomes measures. Clin Ther. May 2014;36(5):648-662.
4
38.
Hung M, Franklin JD, Hon SD, Cheng C, Conrad J, Saltzman CL. Time for a paradigm
5
shift with computerized adaptive testing of general physical function outcomes
6
measurements. Foot Ankle Int. Jan 2014;35(1):1-7.
7
39.
Hays RD, Liu H, Spritzer K, Cella D. Item response theory analyses of physical
8
functioning items in the medical outcomes study. Med Care. May 2007;45(5 Suppl
9
1):S32-38.
10
40.
Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B,
11
Rose M. The Patient-Reported Outcomes Measurement Information System (PROMIS):
12
progress of an NIH Roadmap cooperative group during its first two years. Med Care.
13
May 2007;45(5 Suppl 1):S3-S11.
14
41.
Scott N, Fayers P, Aaronson N, Bottomley A, de Graeff A, Groenvold M, Gundy C,
15
Koller M, Petersen M, Sprangers M, Group. obotEQoL. EORTC QLQ-C30 Reference
16
Values. Brussels: EORTC Quality of Life Group Publications; 2008.
17 18 Table 1: Domains, items, and answer direction of the Spine Oncology Study Group - Outcomes Questionnaire (SOSG-OQ)
Domains
Items*
Domain I: Physical Function
1. What is your current level of activity?
21
Page 21 of 30
2. What is your ability to work (including at home)/ study? 3. Does your spine limit your ability to care for yourself? 4. Do you require assistance from others to travel outside of the home? Domain II: Neurological Function
5. Do you have weakness in your legs? 6. Do you have weakness in your arms? 7. What assistance do you need with your walking?
8. Do you have difficulty controlling your bowel and bladder function beyond episodes o Domain III: Pain
9. Overall, on average, how much back/neck pain have you had in the past 4 weeks? 10. When you are in your most comfortable position, do you still experience back / neck 11. How much has your pain limited mobility (sitting, standing, your walking)? 12. How confident do you feel about your ability to manage your pain on your own?
Domain IV: Mental Health
13. Have you felt depressed over the past 4 weeks? 14. Do you feel anxiety about your Health related to your spine? 15. Do you have a lot of energy ? 16. When I feel pain, it is awful and I feel it overwhelms me
Domain V: Social Function
17. Does your spine influence your ability to concentrate on conversations, reading and t
18. Do you feel that your spine condition impairs/compromises your personal relationsh 19. Are you comfortable meeting new people?
22
Page 22 of 30
20. Do you leave the house for social functions? Domain VI: Post Therapy Questions
21. Are you satisfied with the results of your spine tumor management? 22. Would you choose the same management of your spine tumor again?
23. How has treatment of your spine changed your physical function and ability to pursu 24. How has treatment of your spine affected your spinal cord and/or nerve function? 25. How has your treatment affected your overall pain from your spine? 26. How has treatment of your spine changed your depression and anxiety? 27. How has treatment of your spine changed your ability to function socially?
* All items are scored on a 5-point scale,
† - = higher score corresponds to worse quality of life, + = higher score corresponds to better quality of life. The scores of items marke
1 2 Table 2: Baseline characteristics
n = 82
Median (Interquartile range) Age (in years)
59 (51 - 68)
n (%) Men
41 (50)
23
Page 23 of 30
Primary tumor Breast
18 (22)
Lung
9 (11)
Kidney
7 (8.5)
Multiple myeloma
6 (7.3)
Colon/rectum
4 (4.9)
Prostate
4 (4.9)
Liver
4 (4.9)
Parotid
3 (3.7)
Thyroid
3 (3.7)
Lymphoma
3 (3.7)
Neuroendocrine
3 (3.7)
Pancreas
2 (2.4)
Esophageal
2 (2.4)
Bladder
2 (2.4)
Other/Unknown*
12 (15)
Anatomical location†
24
Page 24 of 30
Cervical spine
9 (11)
Thoracic spine
50 (61)
Lumbar spine
38 (46)
Sacral spine
11 (13)
ECOG §
0
25 (32)
1
30 (39)
2
15 (19)
3
6 (7.8)
4
1 (1.3)
Pain
66 (81)
Symptoms of neurological compromise
33 (40)
Other bone metastatic lesions
40 (49)
Visceral metastases
37 (45)
Prior surgery for metastatic spine disease
28 (34)
25
Page 25 of 30
Prior radiotherapy for metastatic spine disease
34 (41)
Prior systemic therapy
53 (65)
* Chondroblastoma (1), Chordoma (1), Germ cell tumor (1), Myoepithelial tumor (1), Giant cell tumor (1), Fibrosarcoma (1), Mesothelioma (1), Squamous cell carcinoma of unknown origin (2), Adenocarcinoma of unknown origin (2), Melanoma (1) † Twenty-three patients had spine lesions in multiple regions § ECOG = Eastern Cooperative Oncology Group performance status was only available for 77 (94%) of 82 patients.
1 2
Table 3: Item completion rate, internal consistency, floor and ceiling effect, and score distribution per domain of the Spine Oncolo Questionnaire (SOSG-OQ) and for the EQ-5D 3L.
Number of items per
Item completion
Median
scale
rate n (%)
(Interquartile range)
SOSG-OQ†
20 items
1632/1640 (99.5)
30 (18 - 38)
3 - 64
0.92 (0.89 – 0.
EQ-5D 3L score‡
5 items
400/410 (97.6)
0.71 (0.45 - 0.82)
0.05 - 1
0.73 (0.63 – 0.
Physical Function
4 items
328 /328 (100)
5 (3 - 9)
0 - 15
0.86
Neurological Function
4 items
327/328 (99.7)
2 (0 - 4)
0 - 10
0.54
Overall
Range
Cronbach alph (95% CI)
Domains within SOSG-OQ‡
26
Page 26 of 30
Pain
4 items
325/328 (99.1)
9 (5 - 12)
0 - 16
0.85
Mental Health
4 items
328/328 (100)
6 (4 - 9)
0 - 15
0.76
Social Function
4 items
325/328 (99.1)
5 (2 - 8)
0 - 14
0.69
7 items
367/371 (98.9)
9 (5 - 13)
0 - 19
0.83
Post therapy questions (n = 53)
* Cronbach's alpha represents the internal consistency, (95% CI = 95% confidence interval calculated through bootstrapping [1000 res † The post therapy questions are excluded from the overall scoring and analysis. ‡ The EQ-5D euroqol quality of life 5-item questionnaire
1 2
Table 4: Multitrait analysis: assessing convergent and divergent validity of individual items with the general 5 domains of the Spin Oncology Study Group - Outcomes Questionnaire (SOSG-OQ) and the overall SOSG-OQ
Item
Physical
Neurological
Domains
number
Function
Function
Physical Function
Item 1
0.84
0.47
Item 2
0.82
Item 3
Pain
Mental
Social
Health
Function
0.60
0.44
0.56
0.73
0.50
0.48
0.36
0.51
0.64
0.82
0.51
0.63
0.44
0.63
0.78
Item 4
0.90
0.64
0.52
0.29
0.57
0.72
Neurological
Item 5
0.38
0.77
0.28
0.09
0.21
0.38†
Function
Item 6
0.39
0.58
0.38
0.29
0.37
0.46
27
Overal
SOSG-OQ s
Page 27 of 30
Pain
Mental Health
Social Function
Item 7*
0.71
0.65
0.38
0.17
0.41
0.57
Item 8
0.11
0.52
0.11
0.15
0.14
0.21†
Item 9
0.57
0.37
0.83
0.55
0.56
0.74
Item 10
0.42
0.35
0.83
0.60
0.55
0.69
Item 11
0.68
0.52
0.85
0.53
0.66
0.82
Item 12
0.43
0.29
0.78
0.53
0.53
0.66
Item 13
0.24
0.17
0.40
0.80
0.39
0.46
Item 14
0.24
0.09
0.43
0.83
0.50
0.53
Item 15
0.51
0.39
0.51
0.67
0.62
0.64
Item 16
0.44
0.28
0.72
0.76
0.63
0.71
Item 17
0.50
0.37
0.68
0.68
0.71
0.72
Item 18
0.51
0.38
0.58
0.58
0.80
0.73
Item 19
0.25
0.14
0.38
0.38
0.67
0.40
Item 20
0.68
0.35
0.45
0.45
0.73
0.67
Bold font indicates the correlation of the item with the own domain (convergent validity); Normal font indicates the correlation of the with other domains (divergent validity). * Indicates higher divergent than convergent validity † Indicates insufficient item-total correlation (< 0.40)
1 2
28
Page 28 of 30
Table 5: factor loadings from exploratory factor analysis after varimax rotation
Domains
Variable
Factor1
Factor2
Factor 3
Factor 4
Factor 5
Physical Function
Item 1
0.38
0.76
0.12
-0.09
-0.05
Item 2
0.17
0.68
0.15
0.16
0.07
Item 3
0.43
0.67
0.09
0.13
0.08
Item 4
0.22
0.84
-0.02
0.13
0.12
Neurological
Item 5
0.24
0.34
0.04
-0.19
0.55
Function
Item 6
0.32
0.21
-0.03
0.17
0.32
Item 7
0.07
0.81
0.01
0.01
0.16
Item 8
0.17
0.04
0.00
0.03
0.37
Item 9
0.67
0.39
0.15
-0.04
-0.09
Item 10
0.78
0.17
0.16
0.04
0.12
Item 11
0.69
0.47
0.05
0.06
0.09
Item 12
0.55
0.27
0.22
0.15
-0.02
Item 13
0.25
0.07
0.72
0.07
0.02
Item 14
0.34
0.07
0.72
0.09
-0.02
Item 15
0.34
0.34
0.26
0.46
0.14
Item 16
0.72
0.15
0.31
0.11
-0.03
Pain
Mental Health
29
Page 29 of 30
Social Function
Item 17
0.65
0.27
0.38
0.03
0.22
Item 18
0.69
0.25
0.18
0.24
0.17
Item 19
0.09
0.15
0.10
0.58
-0.12
Item 20
0.22
0.64
0.16
0.44
-0.22
Bold indicates the highest factor loading per item
1 2
30
Page 30 of 30
S1529-9430(15)01197-3 http://dx.doi.org/doi: 10.1016/j.spinee.2015.07.456 SPINEE 56497
To appear in:
The Spine Journal
Received date: Revised date: Accepted date:
9-2-2015 22-5-2015 23-7-2015
Please cite this article as: Stein J. Janssen, Teun Teunis, Eva van Dijk, Marco L. Ferrone, John H. Shin, Francis Hornicek, Joseph H. Schwab, Validation of the Spine Oncology Study Group Outcomes Questionnaire to assess quality of life in patients with metastatic spine disease, The Spine Journal (2015), http://dx.doi.org/doi: 10.1016/j.spinee.2015.07.456. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1
Validation of the Spine Oncology Study Group Outcomes Questionnaire to assess quality of
2
life in patients with metastatic spine disease
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
Stein J. Janssen, MD1 Research Fellow [email protected] Teun Teunis, MD1 Research Fellow [email protected] Eva van Dijk, BSc1 Graduate research assistant [email protected] Marco L. Ferrone, MD3 Spine surgeon [email protected] John H. Shin, MD2 Spine surgeon [email protected] Francis Hornicek, MD, MS, PhD1 Chief Orthopaedic Oncology Service [email protected] Joseph H. Schwab, MD, MS1 Instructor in Orthopaedic Surgery [email protected] Affiliation address: 1) Department of Orthopaedic Surgery, Orthopaedic Oncology Service, Massachusetts General Hospital – Harvard Medical School, 55 Fruit Street, Boston, Massachusetts, 02114, United States of America 2) Department of Neurosurgery, Massachusetts General Hospital – Harvard Medical School, 55 Fruit Street, Boston, Massachusetts, 02114, United States of America 3) Department of Orthopaedic Surgery, Spine Service, Brigham and Women’s Hospital – Harvard Medical School, 75 Francis Street, Boston, MA 02115, United States of America Corresponding author & request for reprints: Stein J. Janssen, MD
1
Page 1 of 30
1 2 3 4 5 6
Address: Room 3.946, Yawkey building, Massachusetts General Hospital 55 Fruit Street, Boston, MA 02114 Telephone: +1-617-726-1569 Fax: +1-617-643-1274 E-Mail: [email protected] Abstract:
7
Background context: General questionnaires are often used to assess quality of life in patients
8
with spine metastases, while a disease-specific survey did not exist until recently. The Spine
9
Oncology Study Group has developed a questionnaire (SOSG-OQ) to measure quality of life in
10
these patients. However, a scoring system was not developed and the questionnaire was not
11
validated in a group of patients, nor was it compared to other general quality of life
12
questionnaires such as the EuroQol 5 Dimensions (EQ-5D) questionnaire.
13
Purpose: Our primary null hypothesis is that there is no association between the SOSG-OQ and
14
EQ-5D. Our secondary null hypothesis is that there is no difference in coverage and internal
15
consistency between the SOSG-OQ and EQ-5D. We also assess coverage, consistency, and
16
validity of the domains within the SOSG-OQ.
17
Study Design/Setting: Survey study from a tertiary care spine referral center.
18
Patient Sample: 82 patients with spine metastases, myeloma, or lymphoma.
19
Outcome Measures: The SOSG-OQ (27 questions, 6 domains) score ranges from 0 to 80 with a
20
higher score indicating worse quality of life. The EQ-5D (5 questions, 5 domains) index score
21
ranges from 0 to 1 with a higher score indicating better quality of life.
22
Methods: The association between the SOSG-OQ and EQ-5D index score was assessed using
23
the Spearman rank correlation. Instrument coverage and precision were assessed by determining
24
item completion rate, median score with range, and floor and ceiling effect. Internal consistency
25
was assessed using Cronbach alpha. Multitrait analysis and exploratory factor analysis were used
2
Page 2 of 30
1
to analyze properties of the individual domains in the SOSG-OQ. No grants or funds were
2
received for this study.
3
Results: The Spearman rank correlation between the SOSG-OQ and EQ-5D questionnaire was
4
high (r = -0.83, P < 0.001). Internal consistency of the SOSG-OQ (0.92, 95% CI: 0.89 – 0.94)
5
was higher as compared to the internal consistency of the EQ-5D (0.73, 95% CI: 0.63 – 0.84; P <
6
0.001). The SOSG-OQ score had no floor or ceiling effect indicating good coverage (median 30,
7
range 3 – 64), while the EQ-5D had a ceiling effect of 10% (median 0.71, range 0.05 – 1).
8
Conclusions: In conclusion, our study proposes a scoring methodology –after reversing 4
9
inversely scored items– for the SOSG-OQ and demonstrates that the questionnaire is a valid tool
10
for the assessment of quality of life in patients with metastatic spine disease. The SOSG-OQ is
11
superior to the EQ-5D in terms of coverage and internal consistency, but consists of more
12
questions.
13
Keywords: spine oncology study group; quality of life; metastases; spine; validity; eq-5d;
14
questionnaire; survey
15
Abbreviations: SOSG-OQ: Spine Oncology Study Group – Outcomes Questionnaire; EQ-5D:
16
EuroQol 5 Dimensions questionnaire.
17 18 19
Introduction
20
Quality of life is an important outcome measure for patients with metastatic spine disease.1-5
21
Some advocate the use of general quality of life questionnaires to allow for comparison across
22
diseases and calculation of cost effectiveness.5 Others prefer disease-specific tools as these
23
discriminate better between patients, are more sensitive to change within a patient, are easier to
3
Page 3 of 30
1
understand for a patient, and result in better coverage of the construct being measured.6-9 A
2
systematic review by Street et al. demonstrated that many different tools are used to assess
3
outcome in patients with metastatic spine disease.6 However, most tools are clinician reported
4
outcomes and measure only one aspect of quality of life (e.g. ECOG, Frankel scale scores, or
5
ambulatory status).6,10 Only a few studies used patient reported quality of life outcome measures
6
(e.g. EQ-5D, SF-36, QLQ-C30).5,6,11,12 Patient reported outcome measures should be the standard
7
as clinicians’ understanding of the effect of disease on patients’ daily lives is poor.13,14 While
8
general questionnaires have been used to assess quality of life metrics in patients with metastatic
9
spine disease, a disease-specific survey did not exist until recently.5-7
10 11
The Spine Oncology Study Group (SOSG) –an international group of experts– has developed an
12
Outcome Questionnaire to measure quality of life in patients with metastatic spine disease.6 The
13
questionnaire was developed through consensus discussion of the SOSG and followed by a
14
systematic evaluation to examine if all quality of life domains were adequately represented.6
15
Independent health professionals rated the individual questions of the Spine Oncology Study
16
Group Outcomes Questionnaire (SOSG-OQ; Table 1) using the International Classification of
17
Functioning, Disability, and Health (ICF) framework.6,15 The ICF belongs to the World Health
18
Organization’s family of international health classifications and is a framework for measuring
19
health and disability, it is not a measuring instrument itself.15 Analysis of the SOSG-OQ using
20
the ICF framework demonstrated good representation of all 4 domains specified by the ICF to be
21
relevant for measurement of function and disability (body structures, body function, activity and
22
participation, and environmental factors).6 However, a scoring system was not developed and the
4
Page 4 of 30
1
questionnaire was not validated in a group of patients, nor was it compared to other general
2
quality of life questionnaires.
3 4
In this study, we compare the disease-specific SOSG-OQ to the generic EuroQol 5 Dimensions
5
(EQ-5D) quality of life questionnaire in patients with metastatic spine disease. Our primary null
6
hypothesis is that there is no association between the SOSG-OQ and EQ-5D. Our secondary null
7
hypothesis is that there is no difference in coverage –floor and ceiling effect– and internal
8
consistency between the SOSG-OQ and EQ-5D. We also assess coverage, internal consistency,
9
and validity of the domains within the SOSG-OQ.
10 11
Methods
12
Study Design
13
We analyzed a convenience sample of 82 patients with spine metastases, myeloma, or
14
lymphoma, who completed the SOSG-OQ and EQ-5D questionnaires for a quality improvement
15
program at our Orthopaedic Spine Oncology unit. All patients presenting to our outpatient clinic
16
are asked to complete these questionnaires independently before visiting with the surgeon. Data
17
is collected through REDCap (Vanderbilt University, Nashville, Tennessee, United States of
18
America) on a tablet computer.16 REDCap is an online data collection tool that enables
19
researchers to create study-specific surveys to capture participant data securely online.16 One
20
hundred ninety-six patients with metastases, myeloma, or lymphoma of the spine encountered
21
our Orthopaedic Spine Oncology unit between September 2011 and November 2014 and were
22
eligible for this study. Patients were included regardless of previous treatment and disease stage.
23
Eighty-five (43%) patients completed the SOSG-OQ and EQ-5D questionnaire. Three (3.5%)
5
Page 5 of 30
1
patients were excluded because they had more than one missing item in one of the SOSG-OQ
2
domains or in the EQ-5D questionnaire. None of the patients were below 18 years of age.
3
Eighty-two patients remained for analysis, we only included a patients’ first survey to avoid a
4
learning curve on questionnaire completion. Secondary use of data –for the purpose of this
5
study– was approved by our institutional review board and a waiver of informed consent was
6
obtained.
7 8 9
Outcome measures
10
Baseline characteristics were obtained from patients’ medical records and consisted of: age at
11
time of survey completion, sex, primary tumor type, affected anatomical location, presence of
12
other bone metastases and visceral metastases, presence of pain (back, neck, or radiating pain),
13
symptoms of neurological compromise (weakness, sensory deficit, or bowel and bladder
14
dysfunction), Eastern Cooperative Oncology Group (ECOG) performance status, prior surgery,
15
radiotherapy, and systemic therapy for metastatic spine disease, myeloma, or lymphoma.
16 17
The SOSG-OQ consists of 27 questions divided into 6 domains: physical function (4 items),
18
neurological function (4 items), pain (4 items), mental health (4 items), social function (4 items),
19
and post therapy questions (7 items) (Table 1).6 All questions have 5 answer options.6 Post
20
therapy questions are only completed if the patient indicates being post treatment for the spine
21
disease. We scored individual items within the SOSG-OQ on a scale from 0 to 4. Items 12, 15,
22
19, and 20 were scored inversely due to their reversed nature (Table 1). A total score was
23
calculated per domain by summing the points of individual items within the domain. The score
6
Page 6 of 30
1
per domain ranges from 0 to 16, except for the post therapy domain score which ranges from 0 to
2
28. The overall score of the SOSG-OQ was calculated by summing the points of all items,
3
excluding the post therapy questions, resulting in a score ranging from 0 to 80 with a higher
4
score indicating worse quality of life. We excluded the post therapy questions from calculation
5
of the SOSG-OQ quality of life score as these items pertain to the provided treatment. Including
6
these in the overall SOSG-OQ score renders it difficult to compare pre- and post treatment
7
quality of life scores. This domain seems to be more useful in assessing success of the treatment.
8 9
The EQ-5D is developed by the EuroQol –an international collaboration of multidisciplinary
10
researchers developing instruments that describe and value health– and is a commonly used
11
instrument to assess quality of life in 5 dimensions (mobility, self-care, usual activities, pain-
12
discomfort, and anxiety-depression) through 5 questions.17 We used the English version for the
13
United States with 3 answer options for every question (EQ-5D 3L).18 An overall EQ-5D index
14
for those who completed all 5 questions (n = 77) was calculated using an algorithm described by
15
Shaw et al.19 The EQ-5D index score ranges from 0 to 1 with a higher score indicating better
16
quality of life.
17 18
Statistical analysis
19
Categorical variables are demonstrated as frequencies with percentages, and continuous variables
20
as median with interquartile range (IQR) as histograms suggested nonparametric distribution of
21
continuous variables.
22
7
Page 7 of 30
1
We assessed the association between the overall SOSG-OQ and EQ-5D index score using the
2
Spearman rank correlation to determine the shared variance. The Spearman rank correlation
3
coefficient ranges from -1 to 1; 1 indicating a perfect correlation, 0 indicating no correlation, and
4
-1 indicating a perfect inverse correlation.
5 6
Coverage, precision, and reliability were assessed for the SOSG-OQ, its domains, and the EQ-
7
5D questionnaire. Instrument coverage and precision were assessed by determining item
8
completion rate, median score with range, and floor and ceiling effect. Floor effect is a term used
9
to describe a situation in which subjects in a study have scores at the lowest possible limit
10
(floor); ceiling effect reflects scores at the upper limit (ceiling). These effects influence the
11
analyses because it can skew results and reduce variation in a questionnaire.20 Reliability was
12
assessed using Cronbach alpha; a measure of internal consistency ranging from 0 to 1, with a
13
higher score indicating higher internal consistency. A Cronbach alpha above 0.70 is
14
recommended; however, a high Cronbach alpha (above 0.90) might indicate question
15
redundancy.21,22 We used bootstrapping (number of resamples: 1000) to calculate the standard
16
error, z statistic, p value, and 95% confidence intervals around the Cronbach alpha values to
17
compare the internal consistency of the EQ-5D and SOSG-OQ.
18 19
Multitrait analysis was used to assess convergent and divergent validity of items within the
20
domains of the SOSG-OQ.23 These measures demonstrate the degree to which the domain
21
measures what it claims to be measuring. The convergent validity of each item was assessed
22
using the Spearman rank correlation coefficient of every item with its own domain score. The
23
convergent validity was considered satisfactory if the correlation coefficient exceeded 0.40.24 For
8
Page 8 of 30
1
the divergent validity, the correlation coefficient between each item and its own domain score
2
should be higher than the correlation with other domain scores. We assessed the item-total
3
Spearman rank correlation, indicating the association of the item with the overall SOSG-OQ
4
score. An item-total correlation below 0.40 was considered insufficient.25 Inter-item Spearman
5
rank correlations were assessed to determine if items were redundant to each other. We
6
considered items with an inter-item correlation above 0.80 to be redundant.
7 8
An exploratory factor analysis was performed to assess underlying traits of the SOSG-OQ.
9
Exploratory factor analysis is a method that looks for underlying traits (unobserved) being
10
measured by items (observed) within a questionnaire and subsequently analyzes the correlation
11
(i.e. factor loading) of these items with the identified traits. We allowed the exploratory factor
12
analysis to retain 5 factors and assessed if items were grouped in accordance with the 5
13
predefined domains. The factor structure was rotated using varimax rotation to optimize the
14
factor loadings among the 5 factors. A factor loading of 1 indicates perfect association of the
15
item with the underlying trait, 0 indicates no association, and -1 indicates perfect inverse
16
association of the item with the underlying trait.
17 18
A two-sided P value below 0.05 was considered to indicate statistical significance. Stata 12
19
(StataCorp LP, Texas, USA) was used for statistical analyses.
20 21
Sample size calculation
9
Page 9 of 30
1
A sample size calculation showed that a minimum sample size of 82 patients would provide 80%
2
statistical power (beta 0.20; alpha 0.05) to detect a correlation of 0.30 between the SOSG-OQ
3
and EQ-5D.
4 5
Baseline characteristics
6
There were 41 (50%) men, and 41 women, median age was 59 years (Table 2). The five most
7
common primary tumors were breast (18 of 82, 22%), lung (9 of 82, 11%), kidney (7 of 82,
8
8.5%), multiple myeloma (6 of 82, 7.3%), and colon/rectum (4 of 82, 4.9%). The thoracic spine
9
was most commonly affected (50 of 82, 61%). Twenty-three patients (28%) had multiple spine
10
regions affected. Forty patients (49%) had other –non-spinal– bone metastases, 37 (45%) had
11
visceral metastases (Table 2). Most patients had pain (66 of 82, 81%), 33 (40%) had neurological
12
compromise, and there was substantial variability in ECOG performance status (Table 2).
13 14
Funding
15
Departmental
16 17
Results
18
Comparing the SOSG-OQ and EQ-5D
19
The Spearman rank correlation between the SOSG-OQ and EQ-5D questionnaire was substantial
20
(r = -0.83, P < 0.001).
21
10
Page 10 of 30
1
Internal consistency of the overall SOSG-OQ (0.92, 95% CI: 0.89 – 0.94) was higher as
2
compared to the internal consistency of the EQ-5D (0.73, 95% CI: 0.63 – 0.84) (P < 0.001)
3
(Table 3).
4 5
The overall SOSG-OQ score had no floor or ceiling effect (range 3 to 64; possible range 0 to 80)
6
indicating good coverage, while the EQ-5D had a ceiling effect of 10% (8 of 82) (Table 2).
7 8
Assessing coverage, reliability and validity of the SOSG-OQ and its domains
9
Internal consistency as determined by Cronbach alpha per domain ranged from 0.54
10
(neurological function) to 0.86 (physical function) (Table 3).
11 12
Every domain within the SOSG-OQ had a floor effect, which was largest in the neurological
13
function domain (22 of 82 patients had a score of 0, 27%) (Table 3). Ceiling effect only occurred
14
in the Pain domain (3 of 82 patients had a score of 16, 3.7%).
15 16
Convergent validity was above the threshold of 0.40 for all items (Table 4). Item 7 (What
17
assistance do you need with your walking?) was the only item with a higher other domain (r =
18
0.71, physical function) than own domain (r = 0.65, neurological function) correlation (Table 4).
19 20
Item 5 (Do you have weakness in your legs?) and 8 (Do you have difficulty controlling your
21
bowel and bladder function beyond episodes of diarrhea / constipation?) did not have
22
satisfactory (r < 0.40) item-total correlations (Table 4). All inter-item correlations were below
23
the 0.80 threshold indicating no redundancy of items (Appendix 1). Item 3 (Does your spine limit
11
Page 11 of 30
1
your ability to care for yourself?) and item 4 (Do you require assistance from others to travel
2
outside of the home?) had the highest inter-item correlation (r = 0.74), item 2 (What is your
3
ability to work/study?) and item 8 (Do you have difficulty controlling your bowel and bladder
4
function beyond episodes of diarrhea/constipation?) had the lowest inter-item correlation (r =
5
0.037) (Appendix 1). A distribution of responses to individual items are demonstrated in
6
appendix 2.
7 8
Factor analysis demonstrated underlying traits corresponding with the physical function,
9
neurological function, and pain domains. Except for item 7 within the neurological function
10
domain (What assistance do you need with your walking?), which was grouped with the physical
11
function domain items. The items within the mental health and social function domains
12
demonstrated associations with a variety of underlying traits (Table 5).
13 14
Discussion
15
The questionnaire developed by the Spine Oncology Study Group to assess quality of life in
16
patients with metastatic spine disease has not been validated in a sample of patients.6 This study
17
tested the SOSG-OQ and compared it to the generic EQ-5D questionnaire in patients with
18
metastatic spine disease. We found a substantial correlation between the questionnaires
19
indicating that they measure the same construct: quality of life. The SOSG-OQ was superior to
20
the EQ-5D in terms of internal consistency and coverage.
21 22
The high inverse correlation between the SOSG-OQ and EQ-5D indicates that they have
23
substantial shared variance and that an increase in the SOSG-OQ (higher score indicates worse
12
Page 12 of 30
1
quality of life) corresponds to a decrease in EQ-5D (higher score indicates better quality of life)
2
and vice versa. Both questionnaires reached the recommended threshold for sufficient internal
3
consistency, but the internal consistency of the SOSG-OQ was higher as compared to the EQ-
4
5D.21,22 However, the internal consistency of the SOSG-OQ (0.92) might indicate item
5
redundancy.21,22 Additional inter-item correlation analysis within the SOSG-OQ refuted possible
6
item redundancy as all correlations were below the 0.80 threshold. The absence of floor and
7
ceiling effects in the overall SOSG-OQ score suggests good coverage, meaning that the full
8
spectrum of quality of life in patients with spine metastatic disease can be measured through this
9
questionnaire. The issue of the ceiling effect in the EQ-5D index score has been demonstrated in
10
previous studies and suggests that the questionnaire is not capable of distinguishing between
11
health states close to full health.26-28 Based on these findings, the SOSG-OQ seems to have better
12
coverage and higher reliability to assess quality of life in patients with metastatic spine disease.
13 14
Further analysis of the SOSG-OQ demonstrated sufficient internal consistency, good coverage,
15
and convergent and divergent validity of all domains except the neurological function domain.
16
The neurological function domain had a substantial floor effect, low internal consistency, two
17
items with low correlations with the overall SOSG-OQ score, and one item that belonged to a
18
different domain. The floor effect can be explained by absence of neurological compromise in
19
many patients with metastatic spine disease; 49 of the 82 (60%) patients in our cohort had no
20
symptoms of neurological compromise.10,29,30 Low internal consistency can be a result of poor
21
question phrasing and multidimensionality. Item 7 (What assistance do you need with your
22
walking?) seems to belong to the physical function domain as demonstrated by the multitrait and
23
exploratory factor analysis. Item 8 (Do you have difficulty controlling your bowel and bladder
13
Page 13 of 30
1
function beyond episodes of diarrhea / constipation?) assesses multiple issues but only allows
2
for one answer (also called a double-barreled question). Reconsidering the content of the
3
neurological function domain and rephrasing item 8 could further improve the reliability of the
4
SOSG-OQ.
5 6
Items in the mental health and social function domains demonstrate associations with a variety of
7
underlying traits. This might be explained by the complex interplay of psychological factors with
8
physical function and pain.31-34
9 10
This study has several limitations. First, patients only completed the questionnaire once and we
11
did not assess test-retest reliability. We see this as a minor limitation as the concept being
12
measured –quality of life– might change rapidly in patients with metastatic cancer, myeloma, or
13
lymphoma, rendering test-retest reliability analysis less valid.35 Second, patients completed the
14
questionnaire in the outpatient clinic; we did not include a subset of patients admitted to the
15
hospital, presenting to the emergency room, or to a hospice with severe illness. Furthermore, we
16
analyzed a convenience sample of 82 patients while 196 potentially eligible patients encountered
17
our outpatient clinic. This might have resulted in selection bias and our sample might therefore
18
not be representative for all patients with spine metastatic disease. This could have influenced
19
the coverage –floor and/or ceiling effect– of the SOSG-OQ and EQ-5D. However, we believe
20
that this did no compromise the correlation, internal consistency, exploratory factor analysis, or
21
multitrait analysis as our sample was heterogeneous with regards to baseline characteristics –
22
tumor type, systemic load, ECOG performance status, pain, and neurological compromise– and
23
outcome. The SOSG-OQ therefore seems to be useful for measuring quality of life in patients
14
Page 14 of 30
1
with metastatic spine disease, but its coverage needs to be assessed in future studies. Third, the
2
number of patients included did not allow us to use the more robust item response theory (IRT)
3
analysis. IRT analysis is considered superior to the classical test theory –Cronbach alpha,
4
multitrait analysis, and exploratory factor analysis– in assessing psychometric properties of a
5
questionnaire.36,37 Future studies with larger numbers should apply this methodology to further
6
analyze validity of the SOSG-OQ. The ultimate goal should be the development of a Computer
7
Adaptive Testing item-bank which offers a dynamic selection of items wherein the response to
8
each item guides the system's choice of the next item, resulting in an individually tailored series
9
of questions.38,39 This reduces questionnaire burden, improves coverage, and maintain reliability
10
over the full-spectrum of the construct being measured (quality of life).38,39 Fourth, we compared
11
the SOSG-OQ with the short (5-question) EQ-5D to assess its face validity; however, a more
12
comprehensive comparison with other quality of life questionnaires –such as the QLQ-C30 and
13
PROMIS Global Health questionnaires– helps establish which questionnaire is optimal in terms
14
of reliability, questionnaire burden, and coverage to assess quality of life in these patients.40,41
15 16
In conclusion, our study proposes a scoring methodology –after reversing 4 inversely scored
17
items– for the SOSG-OQ and demonstrates that the questionnaire is a valid and reliable tool for
18
the assessment of quality of life in patients with metastatic spine disease, myeloma, or
19
lymphoma. The SOSG-OQ is superior to the EQ-5D in terms of coverage and reliability, but
20
consists of more questions.
21 22
15
Page 15 of 30
1
References
2
1.
Wai EK, Finkelstein JA, Tangente RP, Holden L, Chow E, Ford M, Yee A. Quality of
3
life in surgical treatment of metastatic spine disease. Spine (Phila Pa 1976). Mar 1
4
2003;28(5):508-512.
5
2.
Valesin Filho ES, de Abreu LC, Lima GH, de Cubero DI, Ueno FH, Figueiredo GS,
6
Valenti VE, Monteiro CB, Wajnsztejn R, Fujiki EN, Neto MR, Rodrigues LM. Pain and
7
quality of life in patients undergoing radiotherapy for spinal metastatic disease treatment.
8
Int Arch Med. 2013;6(1):6.
9
3.
Dea N, Versteeg A, Fisher C, Kelly A, Hartig D, Boyd M, Paquette S, Kwon BK, Dvorak
10
M, Street J. Adverse events in emergency oncological spine surgery: a prospective
11
analysis. J Neurosurg Spine. Nov 2014;21(5):698-703.
12
4.
Sutcliffe P, Connock M, Shyangdan D, Court R, Kandala NB, Clarke A. A systematic
13
review of evidence on malignant spinal metastases: natural history and technologies for
14
identifying patients at high risk of vertebral fracture and spinal cord compression. Health
15
Technol Assess. Sep 2013;17(42):1-274.
16
5.
Choi D, Morris S, Crockard A, Albert T, Bunger C, Fehlings M, Harrop J, Kawahara N,
17
Martin JA, Massicotte EM, Mazel C, Oner FC, Peul W, Tomita K, Wang M. Assessment
18
of quality of life after surgery for spinal metastases: position statement of the Global
19
Spine Tumour Study Group. World Neurosurg. Dec 2013;80(6):e175-179.
20
6.
Street J, Lenehan B, Berven S, Fisher C. Introducing a new health-related quality of life
21
outcome tool for metastatic disease of the spine: content validation using the International
22
Classification of Functioning, Disability, and Health; on behalf of the Spine Oncology
23
Study Group. Spine (Phila Pa 1976). Jun 15 2010;35(14):1377-1386.
16
Page 16 of 30
1
7.
Street J, Berven S, Fisher C, Ryken T. Health related quality of life assessment in
2
metastatic disease of the spine: a systematic review. Spine (Phila Pa 1976). Oct 15
3
2009;34(22 Suppl):S128-134.
4
8.
de Vries M, Ouwendijk R Fau - Kessels AG, Kessels Ag Fau - de Haan MW, de Haan
5
Mw Fau - Flobbe K, Flobbe K Fau - Hunink MGM, Hunink Mg Fau - van Engelshoven
6
JMA, van Engelshoven Jm Fau - Nelemans PJ, Nelemans PJ. Comparison of generic and
7
disease-specific questionnaires for the assessment of quality of life in patients with
8
peripheral arterial disease. 20050315 DCOM- 20050506 (0741-5214 (Print)).
9
9.
Heniford BT, Walters Al Fau - Lincourt AE, Lincourt Ae Fau - Novitsky YW, Novitsky
10
Yw Fau - Hope WW, Hope Ww Fau - Kercher KW, Kercher KW. Comparison of generic
11
versus specific quality-of-life scales for mesh hernia repairs. 20080404 DCOM-
12
20080422 (1879-1190 (Electronic)).
13
10.
Patchell RA, Tibbs PA, Regine WF, Payne R, Saris S, Kryscio RJ, Mohiuddin M, Young
14
B. Direct decompressive surgical resection in the treatment of spinal cord compression
15
caused by metastatic cancer: a randomised trial. Lancet. Aug 20-26 2005;366(9486):643-
16
648.
17
11.
Miscusi M, Polli FM, Forcato S, Ricciardi L, Frati A, Cimatti M, De Martino L, Ramieri
18
A, Raco A. Comparison of minimally invasive surgery with standard open surgery for
19
vertebral thoracic metastases causing acute myelopathy in patients with short- or mid-
20
term life expectancy: surgical technique and early clinical results. J Neurosurg Spine.
21
May 2015;22(5):518-525.
17
Page 17 of 30
1
12.
Pflugmacher R, Taylor R, Agarwal A, Melcher I, Disch A, Haas NP, Klostermann C.
2
Balloon kyphoplasty in the treatment of metastatic disease of the spine: a 2-year
3
prospective evaluation. Eur Spine J. Aug 2008;17(8):1042-1048.
4
13.
5 6
Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. 2015;350:g7818.
14.
Nelson E, Conger B, Douglass R, Gephart D, Kirk J, Page R, Clark A, Johnson K, Stone
7
K, Wasson J, Zubkoff M. Functional health status levels of primary care patients. JAMA.
8
Jun 24 1983;249(24):3331-3338.
9
15.
10 11
Organization WH. International Classification of Functioning, Disability and Health (ICF)2001.
16.
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic
12
data capture (REDCap)--a metadata-driven methodology and workflow process for
13
providing translational research informatics support. J Biomed Inform. Apr
14
2009;42(2):377-381.
15
17.
16 17
Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med. Jul 2001;33(5):337-343.
18.
Janssen MF, Pickard AS, Golicki D, Gudex C, Niewada M, Scalone L, Swinburn P,
18
Busschbach J. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L
19
across eight patient groups: a multi-country study. Qual Life Res. Sep 2013;22(7):1717-
20
1727.
21 22
19.
Shaw JW, Johnson JA, Coons SJ. US valuation of the EQ-5D health states: development and testing of the D1 valuation model. Med Care. Mar 2005;43(3):203-220.
18
Page 18 of 30
1
20.
Delucchi KL, Bostrom A. Methods for analysis of skewed data distributions in
2
psychiatric clinical studies: working with many zero values. Am J Psychiatry. Jul
3
2004;161(7):1159-1168.
4
21.
5
Panayides P. Coefficient Alpha Interpret With Caution. Europe's Journal of Psychology. 2013;9(4):687-696.
6
22.
Nunnally JC. Psychometric theory 2nd ed. ed. New York, NY: McGraw-Hill; 1978.
7
23.
Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-
8 9
multimethod matrix. Psychol Bull. Mar 1959;56(2):81-105. 24.
Panouilleres M, Anota A, Nguyen TV, Bredart A, Bosset JF, Monnier A, Mercier M,
10
Hardouin JB. Evaluation properties of the French version of the OUT-PATSAT35
11
satisfaction with care questionnaire according to classical and item response theory
12
analyses. Qual Life Res. Mar 7 2014.
13
25.
Gliem JA, Gliem RR. Calculating, Interpreting, and Reporting Cronbach’s Alpha
14
Reliability Coefficient for Likert-Type Scales. Paper Presented at the Midwest Research-
15
to-Practice Conference in Adult, Continuing, and Community Education. 2003.
16
26.
17 18
Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Econ. Sep 2004;13(9):873-884.
27.
Soer R, Reneman MF, Speijer BL, Coppes MH, Vroomen PC. Clinimetric properties of
19
the EuroQol-5D in patients with chronic low back pain. Spine J. Nov 2012;12(11):1035-
20
1039.
21
28.
Bharmal M, Thomas J, 3rd. Comparing the EQ-5D and the SF-6D descriptive systems to
22
assess their ceiling effects in the US general population. Value Health. Jul-Aug
23
2006;9(4):262-271.
19
Page 19 of 30
1
29.
Singh K, Samartzis D, Vaccaro AR, Andersson GB, An HS, Heller JG. Current concepts
2
in the management of metastatic spinal disease. The role of minimally-invasive
3
approaches. J Bone Joint Surg Br. Apr 2006;88(4):434-442.
4
30.
Saengnipanthkul S, Jirarattanaphochai K, Rojviroj S, Sirichativapee W, Mahakkanukrauh
5
C. Metastatic adenocarcinoma of the spine. Spine (Phila Pa 1976). Apr 1992;17(4):427-
6
430.
7
31.
8 9
He J, Xiong W, Li F, Luo W, Gao S. Depression influences pain and function after cervical disc arthroplasty. J Neurosurg Sci. Sep 12 2014.
32.
Hirakawa Y, Hara M, Fujiwara A, Hanada H, Morioka S. The relationship among
10
psychological factors, neglect-like symptoms and postoperative pain after total knee
11
arthroplasty. Pain Res Manag. Sep-Oct 2014;19(5):251-256.
12
33.
13 14
Nota SP, Bot AG, Ring D, Kloen P. Disability and depression after orthopaedic trauma. Injury. Jun 21 2014.
34.
Vranceanu AM, Bachoura A, Weening A, Vrahas M, Smith RM, Ring D. Psychological
15
factors predict disability and pain intensity after skeletal trauma. J Bone Joint Surg Am.
16
Feb 5 2014;96(3):e20.
17
35.
18 19
Davidshofer KR, Murphy CO. Psychological testing : principles and applications. 6 ed: Upper Saddle River, N.J.: Pearson/Prentice Hall; 2005.
36.
Petrillo J, Cano SJ, McLeod LD, Coon CD. Using classical test theory, item response
20
theory, and rasch measurement theory to evaluate patient-reported outcome measures: a
21
comparison of worked examples. Value Health. Jan 2015;18(1):25-34.
20
Page 20 of 30
1
37.
Cappelleri JC, Jason Lundy J, Hays RD. Overview of classical test theory and item
2
response theory for the quantitative assessment of items in developing patient-reported
3
outcomes measures. Clin Ther. May 2014;36(5):648-662.
4
38.
Hung M, Franklin JD, Hon SD, Cheng C, Conrad J, Saltzman CL. Time for a paradigm
5
shift with computerized adaptive testing of general physical function outcomes
6
measurements. Foot Ankle Int. Jan 2014;35(1):1-7.
7
39.
Hays RD, Liu H, Spritzer K, Cella D. Item response theory analyses of physical
8
functioning items in the medical outcomes study. Med Care. May 2007;45(5 Suppl
9
1):S32-38.
10
40.
Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B,
11
Rose M. The Patient-Reported Outcomes Measurement Information System (PROMIS):
12
progress of an NIH Roadmap cooperative group during its first two years. Med Care.
13
May 2007;45(5 Suppl 1):S3-S11.
14
41.
Scott N, Fayers P, Aaronson N, Bottomley A, de Graeff A, Groenvold M, Gundy C,
15
Koller M, Petersen M, Sprangers M, Group. obotEQoL. EORTC QLQ-C30 Reference
16
Values. Brussels: EORTC Quality of Life Group Publications; 2008.
17 18 Table 1: Domains, items, and answer direction of the Spine Oncology Study Group - Outcomes Questionnaire (SOSG-OQ)
Domains
Items*
Domain I: Physical Function
1. What is your current level of activity?
21
Page 21 of 30
2. What is your ability to work (including at home)/ study? 3. Does your spine limit your ability to care for yourself? 4. Do you require assistance from others to travel outside of the home? Domain II: Neurological Function
5. Do you have weakness in your legs? 6. Do you have weakness in your arms? 7. What assistance do you need with your walking?
8. Do you have difficulty controlling your bowel and bladder function beyond episodes o Domain III: Pain
9. Overall, on average, how much back/neck pain have you had in the past 4 weeks? 10. When you are in your most comfortable position, do you still experience back / neck 11. How much has your pain limited mobility (sitting, standing, your walking)? 12. How confident do you feel about your ability to manage your pain on your own?
Domain IV: Mental Health
13. Have you felt depressed over the past 4 weeks? 14. Do you feel anxiety about your Health related to your spine? 15. Do you have a lot of energy ? 16. When I feel pain, it is awful and I feel it overwhelms me
Domain V: Social Function
17. Does your spine influence your ability to concentrate on conversations, reading and t
18. Do you feel that your spine condition impairs/compromises your personal relationsh 19. Are you comfortable meeting new people?
22
Page 22 of 30
20. Do you leave the house for social functions? Domain VI: Post Therapy Questions
21. Are you satisfied with the results of your spine tumor management? 22. Would you choose the same management of your spine tumor again?
23. How has treatment of your spine changed your physical function and ability to pursu 24. How has treatment of your spine affected your spinal cord and/or nerve function? 25. How has your treatment affected your overall pain from your spine? 26. How has treatment of your spine changed your depression and anxiety? 27. How has treatment of your spine changed your ability to function socially?
* All items are scored on a 5-point scale,
† - = higher score corresponds to worse quality of life, + = higher score corresponds to better quality of life. The scores of items marke
1 2 Table 2: Baseline characteristics
n = 82
Median (Interquartile range) Age (in years)
59 (51 - 68)
n (%) Men
41 (50)
23
Page 23 of 30
Primary tumor Breast
18 (22)
Lung
9 (11)
Kidney
7 (8.5)
Multiple myeloma
6 (7.3)
Colon/rectum
4 (4.9)
Prostate
4 (4.9)
Liver
4 (4.9)
Parotid
3 (3.7)
Thyroid
3 (3.7)
Lymphoma
3 (3.7)
Neuroendocrine
3 (3.7)
Pancreas
2 (2.4)
Esophageal
2 (2.4)
Bladder
2 (2.4)
Other/Unknown*
12 (15)
Anatomical location†
24
Page 24 of 30
Cervical spine
9 (11)
Thoracic spine
50 (61)
Lumbar spine
38 (46)
Sacral spine
11 (13)
ECOG §
0
25 (32)
1
30 (39)
2
15 (19)
3
6 (7.8)
4
1 (1.3)
Pain
66 (81)
Symptoms of neurological compromise
33 (40)
Other bone metastatic lesions
40 (49)
Visceral metastases
37 (45)
Prior surgery for metastatic spine disease
28 (34)
25
Page 25 of 30
Prior radiotherapy for metastatic spine disease
34 (41)
Prior systemic therapy
53 (65)
* Chondroblastoma (1), Chordoma (1), Germ cell tumor (1), Myoepithelial tumor (1), Giant cell tumor (1), Fibrosarcoma (1), Mesothelioma (1), Squamous cell carcinoma of unknown origin (2), Adenocarcinoma of unknown origin (2), Melanoma (1) † Twenty-three patients had spine lesions in multiple regions § ECOG = Eastern Cooperative Oncology Group performance status was only available for 77 (94%) of 82 patients.
1 2
Table 3: Item completion rate, internal consistency, floor and ceiling effect, and score distribution per domain of the Spine Oncolo Questionnaire (SOSG-OQ) and for the EQ-5D 3L.
Number of items per
Item completion
Median
scale
rate n (%)
(Interquartile range)
SOSG-OQ†
20 items
1632/1640 (99.5)
30 (18 - 38)
3 - 64
0.92 (0.89 – 0.
EQ-5D 3L score‡
5 items
400/410 (97.6)
0.71 (0.45 - 0.82)
0.05 - 1
0.73 (0.63 – 0.
Physical Function
4 items
328 /328 (100)
5 (3 - 9)
0 - 15
0.86
Neurological Function
4 items
327/328 (99.7)
2 (0 - 4)
0 - 10
0.54
Overall
Range
Cronbach alph (95% CI)
Domains within SOSG-OQ‡
26
Page 26 of 30
Pain
4 items
325/328 (99.1)
9 (5 - 12)
0 - 16
0.85
Mental Health
4 items
328/328 (100)
6 (4 - 9)
0 - 15
0.76
Social Function
4 items
325/328 (99.1)
5 (2 - 8)
0 - 14
0.69
7 items
367/371 (98.9)
9 (5 - 13)
0 - 19
0.83
Post therapy questions (n = 53)
* Cronbach's alpha represents the internal consistency, (95% CI = 95% confidence interval calculated through bootstrapping [1000 res † The post therapy questions are excluded from the overall scoring and analysis. ‡ The EQ-5D euroqol quality of life 5-item questionnaire
1 2
Table 4: Multitrait analysis: assessing convergent and divergent validity of individual items with the general 5 domains of the Spin Oncology Study Group - Outcomes Questionnaire (SOSG-OQ) and the overall SOSG-OQ
Item
Physical
Neurological
Domains
number
Function
Function
Physical Function
Item 1
0.84
0.47
Item 2
0.82
Item 3
Pain
Mental
Social
Health
Function
0.60
0.44
0.56
0.73
0.50
0.48
0.36
0.51
0.64
0.82
0.51
0.63
0.44
0.63
0.78
Item 4
0.90
0.64
0.52
0.29
0.57
0.72
Neurological
Item 5
0.38
0.77
0.28
0.09
0.21
0.38†
Function
Item 6
0.39
0.58
0.38
0.29
0.37
0.46
27
Overal
SOSG-OQ s
Page 27 of 30
Pain
Mental Health
Social Function
Item 7*
0.71
0.65
0.38
0.17
0.41
0.57
Item 8
0.11
0.52
0.11
0.15
0.14
0.21†
Item 9
0.57
0.37
0.83
0.55
0.56
0.74
Item 10
0.42
0.35
0.83
0.60
0.55
0.69
Item 11
0.68
0.52
0.85
0.53
0.66
0.82
Item 12
0.43
0.29
0.78
0.53
0.53
0.66
Item 13
0.24
0.17
0.40
0.80
0.39
0.46
Item 14
0.24
0.09
0.43
0.83
0.50
0.53
Item 15
0.51
0.39
0.51
0.67
0.62
0.64
Item 16
0.44
0.28
0.72
0.76
0.63
0.71
Item 17
0.50
0.37
0.68
0.68
0.71
0.72
Item 18
0.51
0.38
0.58
0.58
0.80
0.73
Item 19
0.25
0.14
0.38
0.38
0.67
0.40
Item 20
0.68
0.35
0.45
0.45
0.73
0.67
Bold font indicates the correlation of the item with the own domain (convergent validity); Normal font indicates the correlation of the with other domains (divergent validity). * Indicates higher divergent than convergent validity † Indicates insufficient item-total correlation (< 0.40)
1 2
28
Page 28 of 30
Table 5: factor loadings from exploratory factor analysis after varimax rotation
Domains
Variable
Factor1
Factor2
Factor 3
Factor 4
Factor 5
Physical Function
Item 1
0.38
0.76
0.12
-0.09
-0.05
Item 2
0.17
0.68
0.15
0.16
0.07
Item 3
0.43
0.67
0.09
0.13
0.08
Item 4
0.22
0.84
-0.02
0.13
0.12
Neurological
Item 5
0.24
0.34
0.04
-0.19
0.55
Function
Item 6
0.32
0.21
-0.03
0.17
0.32
Item 7
0.07
0.81
0.01
0.01
0.16
Item 8
0.17
0.04
0.00
0.03
0.37
Item 9
0.67
0.39
0.15
-0.04
-0.09
Item 10
0.78
0.17
0.16
0.04
0.12
Item 11
0.69
0.47
0.05
0.06
0.09
Item 12
0.55
0.27
0.22
0.15
-0.02
Item 13
0.25
0.07
0.72
0.07
0.02
Item 14
0.34
0.07
0.72
0.09
-0.02
Item 15
0.34
0.34
0.26
0.46
0.14
Item 16
0.72
0.15
0.31
0.11
-0.03
Pain
Mental Health
29
Page 29 of 30
Social Function
Item 17
0.65
0.27
0.38
0.03
0.22
Item 18
0.69
0.25
0.18
0.24
0.17
Item 19
0.09
0.15
0.10
0.58
-0.12
Item 20
0.22
0.64
0.16
0.44
-0.22
Bold indicates the highest factor loading per item
1 2
30
Page 30 of 30