- Email: [email protected]
Developing Item Response Theory–Based Short Forms to Measure the Social Impact of Burn Injuries
Accepted Manuscript Developing Item Response Theory-Based Short Forms to Measure the Social Impact of Burn Injuries Molly Marino, PhD, MPH, Emily C. Dore, MSW, Pengsheng Ni, MD, MPH, Colleen M. Ryan, MD, Jeff C. Schneider, MD, Amy Acton, RN, BSN, Alan M. Jette, PT PhD, Lewis E. Kazis, ScD PII:
S0003-9993(17)31072-9
DOI:
10.1016/j.apmr.2017.06.037
Reference:
YAPMR 57001
To appear in:
ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION
Received Date: 7 June 2017 Accepted Date: 28 June 2017
Please cite this article as: Marino M, Dore EC, Ni P, Ryan CM, Schneider JC, Acton A, Jette AM, Kazis LE, Developing Item Response Theory-Based Short Forms to Measure the Social Impact of Burn Injuries, ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION (2017), doi: 10.1016/ j.apmr.2017.06.037. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Title: Developing Item Response Theory-Based Short Forms to Measure the Social Impact of
Running head: Short Forms for the Social Impact of Burns
RI PT
Burn Injuries
SC
Authors: Molly Marino, PhD, MPH1; Emily C. Dore MSW2; Pengsheng Ni, MD, MPH1; Colleen M. Ryan, MD3,4,5; Jeff C. Schneider, MD4,6 Amy Acton RN, BSN7; and Alan M. Jette PT PhD2 and Lewis E. Kazis, ScD1;
M AN U
Affiliations: (1) Boston University School of Public Health, Health & Disabilities Research Institute (2) Boston University School of Public Health, Center for the Assessment of Pharmaceutical Practices (CAPP) Department of Health Law, Policy, and Management (3) Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts. (4) Harvard Medical School, Boston, Massachusetts (5) Shriners Hospitals for Children –Boston®, Boston, Massachusetts (6) Department of Physical Medicine and Rehabilitation, Spaulding Rehabilitation Hospital, Boston, Massachusetts (7) The Phoenix Society for Burn Survivors.
Conflicts of Interest: None
TE D
Acknowledgments: This work is supported by National Institute on Disability, Independent Living, and Rehabilitation Research Award Number 90DP0055.
AC C
EP
Corresponding Author: Molly Marino, PhD, MPH Health and Disability Research Institute Boston University School of Public Health 715 Albany Street Talbot T5W Boston, MA 02118 Telephone: 617-638-1990 Fax: 617-638-1999 Email: [email protected]
ACCEPTED MANUSCRIPT
Title: Developing Item Response Theory-Based Short Forms to Measure the Social Impact of Burn
2
Injuries
3
Abstract:
4
Objective: To develop self-reported short forms for the LIBRE Profile.
5
Design: Participants were selected for inclusion in the short forms based upon the item parameters of
6
discrimination and average difficulty. Test information function, as well as floor and ceiling effects, were
7
evaluated for both the full item bank and the short forms.
8
Setting: Participants were recruited were from the Phoenix Society (a support network for burn
9
survivors), peer support networks, social media, and mailings.
SC
RI PT
1
Participants: 601 burn survivors over 18 years of age in North America.
11
Interventions: none
12
Main Outcome Measures: The LIBRE-Profile
13
Results: Ten-item short forms were developed to cover the six LIBRE Profile scales: Relationships with
14
Family & Friends, Social Interactions, Social Activities, Work & Employment, Romantic Relationships,
15
and Sexual Relationships. Ceiling effects were ≤15% for all scales; floor effects were all <1%. The
16
marginal reliability of the short forms ranged from 0.85-0.89.
17
Conclusions: The LIBRE Profile-Short Forms demonstrated credible psychometric properties. The short
18
form version provides a viable alternative to administering the LIBRE Profile when resources do not
19
allow for computer/internet access. The full item bank, Computerized Adaptive Test, and short forms are
20
all scored along the same metric and therefore scores are comparable regardless of mode of
21
administration.
22
Key Words: Item Response Theory, Burn Injury, Outcome Measurement, Questionnaires, Psychometrics
23
Abbreviations: Total Body Surface Area Burned (TBSA), Item Response Theory (IRT), Graded
24
Response Model (GRM) Computerized Adaptive Testing (CAT), Life Impact Burn Recovery Evaluation
25
(LIBRE), Short Form (SF), Item Characteristic Curves (ICCs), Differential Item Functioning (DIF), Test
26
Information Function (TIF).
AC C
EP
TE D
M AN U
10
ACCEPTED MANUSCRIPT
Burn survivors face a wide-range of burn-related challenges, including physical and
28
psychological issues that potentially impact their ability to reintegrate into society. Physical issues such as
29
joint stiffness, chronic pain, chronic itch, temperature intolerance, scarring, inability to sweat, and fatigue
30
may limit a person’s ability to be active and to join in community events (1,2). Psychological issues may
31
result from the burn survivor’s altered appearance or from an abrupt and unexpected decrease in
32
independence. Anxiety, depression, and post-traumatic stress are common in almost 50% of burn
33
survivors, with 20%-30% of adult burn survivors reporting moderate-to-severe psychological and/or
34
social difficulties (3,4). Any of these conditions alone could isolate the burn survivor, but the combination
35
can make recovery especially difficult. Body image, return to work, and sexual function are some of the
36
psychosocial areas that have so far been covered in burn research (2,3,5–8).
M AN U
SC
RI PT
27
Individuals with facial disfigurements, such as a face burn, avoid social situations as a way to
38
deal with difficult emotions (5). Dissatisfaction with body image is a common issue for burn survivors,
39
and is linked to depression symptoms and social issues (5). Burn survivors report attracting unwanted
40
stares and comments from strangers due to their burn scars. This stigmatization is thought to result in
41
negative body image, anxiety, and depression (2). While some survivors return to work eventually, about
42
50%-60% need a change in employment and 28% of all burn survivors never return to work at all (3,8).
43
Barriers include being self-conscious about appearance, fear of leaving home, and fear of the workplace
44
(6). Burn survivors of both genders also experience decreased sexual satisfaction, which may relate to the
45
emotional and physical ramifications of the burn injury (3,9).
EP
AC C
46
TE D
37
Although some of these issues have previously been identified, burn survivors continue to report
47
that there is a lack of social support and treatment options. Until now, there has not been a measure
48
focused exclusively on the social impact of a burn injury. Studies, clinical observations, and burn survivor
49
feedback have all identified the social and psychological difficulties encountered during the rehabilitation
50
process (3,5,10). The ability to measure these difficulties would allow the clinician and patient to have a
51
directed discussion about treatment, optimizing and personalizing a burn survivor’s recovery (10–13).
ACCEPTED MANUSCRIPT
The Burn Specific Health Scale (BSHS) has a limited domain structure that does not examine
53
psychosocial aspects in depth (14). The Burn Outcomes Questionnaires (BOQ) developed by Shriners
54
Burn Hospital and the American Burn Association measure outcomes based on specific age ranges. These
55
measures include the BOQ0-4, which focuses on children younger than 5 years of age (15), the BOQ5-18 for
56
children 5 to 18 years of age (16), the BOQ11-18 for adolescents 11 to 18 years (17), and the Young Adult
57
Burn Outcomes Questionnaire (YABOQ) for ages 19 to 30 years (18). In addition, generic function and
58
quality of life assessments are limited in their applicability to burn survivors since patients with burns
59
face a complex and unique constellation of impairments that involve multiple domains. These domains
60
deserve acknowledgement and recognition by condition-specific instruments focusing on burn
61
survivors (14).
62
LIBRE PROFILE:
M AN U
SC
RI PT
52
The LIBRE Profile is a new self-reported measure of social participation for burn survivors.
64
Items were initially developed based upon a comprehensive literature review and focus groups with
65
clinical experts and burn survivors. We administered the initial group of 192 items to a convenience
66
sample of 601 burn survivors who were age 18 or older, having survived a burn that was at least 5%
67
TBSA or burns to one of four critical areas (face, hands, feet, and genitals), living in the United States or
68
Canada, and were able to read and understand English. Details on the sample are available elsewhere (19–
69
21).
EP
TE D
63
Initial analyses included an Exploratory Factor Analysis to examine the factor loading patterns
71
among all 192 items, Confirmatory Factor Analysis to confirm unidimensionality of each domain, and
72
calibration of the items using the two parameter Graded Response Model (GRM) Item Response Theory
73
(IRT) Model (22–24). We also examined differential item functioning (DIF), to identify whether people
74
at the same estimated ability level in a content domain respond differently to the same item based on
75
another variable. DIF was conducted based on age, gender, race, and time since burn injury using a 2-step
76
IRT based DIF method, and examined the difference in the Item Characteristic Curves (ICCs) using the
AC C
70
ACCEPTED MANUSCRIPT
77
weighted Area Between the expected score Curves (wABC) to examine DIF impact. Items that
78
demonstrated DIF were retained with calibrations specific to the relevant characteristic (25–27). The six scales of the LIBRE Profile are Relationships with Family & Friends, Social Interactions,
80
Social Activities, Work & Employment, Romantic Relationships, and Sexual Relationships. The scales
81
contain questions and statements with three different response categories. The first were agreement
82
responses: Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree, N/A. The
83
second was a frequency scale: Never, Almost Never, Sometimes, Often, Always, N/A. And the third type
84
of category was related to degree: Not at All, A Little Bit, Somewhat, Quite a Bit, Very Much, N/A.
85
Person IRT scores are transformed to a T-score distribution where the mean=50, SD=10 based on the
86
average of the overall sample of burn subjects used for the calibration phase of the study (T score=person
87
score*10+50). Lower scores correspond to poorer performance on the scale.
M AN U
SC
RI PT
79
The LIBRE Profile was originally developed for administration through Computerized Adaptive
89
Testing (CAT). The CAT draws upon items from the entire instrument, applying an algorithm to choose
90
which items to pose to the participant based on previous responses. The advantage of this administration
91
is that it achieves accurate and precise scores while administering fewer questionnaire items than a
92
traditional full-length measure (13,28). However, not every clinical setting is able to use the CAT because
93
of limited computer resources or expertise (12,29). For this reason, we set out to develop fixed short
94
forms—a subset of questions from the larger instrument—which can be administered and scored on
95
paper.
EP
AC C
96
TE D
88
This paper discusses the development and assessment of the psychometric properties of the IRT-
97
based LIBRE Profile-Short Forms (SF). The LIBRE Profile-SF will allow more people to have access to
98
the tool who do not have the ability to administer the CAT. This study was approved by the Boston
99
University Institutional Review Board.
100 101
Methods
102
Sample:
ACCEPTED MANUSCRIPT
103 104
The LIBRE Profile-SFs were developed using the data from the same sample of 601 burn survivors who provided the information for the development of the initial LIBRE Profile.
105 Strategy for SF Construction:
RI PT
106
Among the candidate items in each LIBRE Profile item pool, we used IRT methods to select
108
questions for SF inclusion based upon high item discrimination, large score range coverage, matching
109
between the item score range and the sample distribution, and a distribution of average item difficulty
110
across the continuum of a scale. Item characteristic curves reflect the relationship between each item and
111
the latent trait that is being assessed. The item discrimination parameter reflects the steepness (slope) of
112
the item characteristic curve. The higher the discrimination parameter for an item, the more that item is
113
able to distinguish among subjects at a similar latent trait level (30,31). Items that displayed DIF were not
114
candidates for LIBRE Profile-SF.
M AN U
SC
107
We calculated the target test information (TIF) value for the whole SF that would be needed to
116
achieve the acceptable reliability of ≥0.80. TIF was weighted by the outpatient sample distribution (using
117
the formula: I = ʃinfo(Θ)g(Θ)d Θ, I) in order to select items with information matched to the score range
118
of the sample. The information value (I) was calculated for each item, then for all possible two item
119
combinations, using the item with the highest I, and each of the other items. The two items with the
120
highest I were then selected, and the process was repeated for 3 items. This process was repeated until all
121
possible items were exhausted.
122
Statistical Analysis
EP
AC C
123
TE D
115
After we arrived at the final items for each SF, we examined their content compared to all the
124
items of each of the six scales. We calculated the internal consistency reliability of all SFs to determine
125
the number of items in the final accepted SF.
126
Using the item parameters obtained from the full item bank analysis, we scored the SF for each
127
respondent and calculated the proportion of the sample at the ceiling and floor of the two administration
128
methods for each scale. Last, we compared the marginal reliability (calculated by score variance-average
ACCEPTED MANUSCRIPT
or squared standard error/score variance) and information provided (called the test information function)
130
for each SF and the full item bank across the entire score distribution for each content domain. To
131
facilitate scoring the SFs, we generated conversion tables between the summed raw score and the scale T-
132
score. For each raw score, we collected all likelihoods of the response patterns yielding the same raw
133
score and formed the joint likelihood, then applied Expected A Posteriori method to calculate the scale
134
score based on the joint likelihood (32).
RI PT
129
Finally, to provide an example of one way LIBRE Profile scores can be displayed in the future,
136
we generated score a sample item map, illustrating where a person’s score lies relative to the item content
137
across the Family & Friends short form scale.
138
Results:
M AN U
SC
135
139
The sample includes 601 burn survivors (Table 1); 54.74% were female, the mean age at survey
140
administration was 44.55 (SD 15.98) years, mean TBSA was 40.45%, and the average time since burn
141
injury was 15.36 years.
There are six LIBRE Profile short forms with 10 items each. Table 2 presents the item content of
143
all SFs, the item difficulty, and slope for each item. Average item difficulty was the most narrow for the
144
Romantic Relationships scale (36.7-41.62), and widest for the Social Interactions scale (29.91-42.88).
145
Score ranges (Table 3) indicate coverage of almost three standard deviations below the mean on all
146
scales, and between one and two standard deviations above the mean on all scales.
EP
TE D
142
Table 3 presents the comparisons of several key psychometric properties of the two measures—
148
the full item bank and SF subset of items. The marginal reliability of the full item bank ranged from 0.87
149
to 0.93. For the SFs, marginal reliability ranged from 0.85-0.89. Ceiling effects were ≤15% for all scales;
150
floor effects were all <1%.
AC C
147
151
Figures 1a-1f illustrate the Test Information Function (TIF) of the full item bank compared to the
152
SFs. There is some sacrifice in score precision for each SF compared to the full item bank, but the SFs
153
generate reliability above 0.9 for a wide range of scale scores. The peaks of the TIF curves occur at the
154
same score ranges, which is also where most of the scores are located. At a maximum, the Sexual
ACCEPTED MANUSCRIPT
155
Relationships SF TIF curve covers 76.9% of the full item bank; the least amount of coverage is the Social
156
Interactions SF, which covers only 42.9% of the full item bank TIF curve. Figure 2 displays one possible way scores can currently be interpreted, using the Family &
158
Friends scale as an example. The items are ordered from hardest at the top (most difficult) to easiest (least
159
difficulty) along the vertical axis. The responses are ordered along the horizontal axis indicating the T-
160
score of that scale. Each shadow bar represents the expected category score range. For example, the first
161
item (“I have many friends in the city where I live.”), the 5 shadow bars from left to right represent 5
162
response categories. For the “Strongly disagree” category, the corresponding T score range is from 0 to
163
24.7; for “Disagree,” the corresponding T score ranges from 24.7 to 38; for “Neither Agree nor Disagree,”
164
the corresponding T score ranges from 38 to 48.6; for “Agree,” the corresponding T score ranges from
165
48.6 to 62.7; for “Strongly Agree,” the corresponding T score ranges from 62.7 to 100. A figure such as
166
this facilitates score interpretation by presenting a person’s score relative to all of the item content.
M AN U
SC
RI PT
157
167 Discussion
TE D
168
The LIBRE Profile-SF is made up of six ten-item short forms, one for each of the LIBRE Profile
170
scales. Results suggest the LIBRE Profile-SF can be an accurate and reliable way to measure social
171
participation in burn survivors.
EP
169
There is some loss of precision with the SF compared with the entire item bank. This is expected,
173
since the SFs contain fewer items. However, the SFs all have ceiling effects less than 15%, and the
174
marginal reliability ranged from 0.85 to 0.89, which is considered credible (30). These findings show that
175
the LIBRE Profile-SF demonstrates appropriate psychometric properties. The differences in scores occur
176
at the extremes, with lower reliability for people who have either very low or very high scores. Therefore,
177
although the LIBRE Profile-SF has fewer items compared to the entire item bank, it also can provide an
178
accurate level score as a substitute for the full item bank.
AC C
172
179
It is important to keep measures brief so that they can be used in a fast-paced clinical setting. We
180
developed ten-item short forms as an appropriate balance between precision and feasibility. Participants
ACCEPTED MANUSCRIPT
would likely prefer a precise score without spending a lengthy amount of time responding to items. More
182
items in a SF increase the precision of the score, but would also increase the respondent burden. A ten-
183
item SF provides an accurate score and is still feasible to administer. Other SFs could be developed using
184
different items or a different number of items (more or fewer).
RI PT
181
The Relationships with Family & Friends SF contains eight items about family and two items
186
about friends, compared to the entire item pool which contains 14 family items and 10 friend items,
187
suggesting scores in that domain are driven by items related to family. Four of the Social Interactions
188
items focus on strangers or meeting new people, reflecting the presence of that aspect of the domain in the
189
full scale. All three of the items in the full Social Activities scale that mention burns are included in the
190
SF, which is beneficial to keeping the items relevant to the burn survivor experience. The Work &
191
Employment SF focuses primarily on global satisfaction with work performance (8 of the 10 items),
192
whereas the full item bank has more of a balance between those items (11) and others involving social
193
dynamics in the workplace (8). The full item bank of the Romantic Relationships scale is the largest in the
194
LIBRE Profile (28 items), whereas the Sexual Relationships is one of the smallest (15 items); item
195
content for both SFs are generally representative of the full scales.
TE D
M AN U
SC
185
The short forms are meant to be an alternative to the CAT for situations where it cannot be
197
administered due to lack of access to a computer or the internet. The SFs are scored on the same metric as
198
the CAT. This means the scores of the SFs are comparable to those of the CAT on the similar metric. This
199
is important so that results from the two instruments can be analyzed within the same context as all other
200
LIBRE Profile scores. Scores will inform patients as to how they are faring within a context of social
201
participation following a burn injury. Currently, scores can be compared to those of other burn survivors
202
from the survey of 601 subjects as part of the original LIBRE Profile calibration study.
203
Study Limitations
AC C
EP
196
204
Limitations to this study include the sampling method. A convenience sample of 601 burn
205
survivors participated in the calibration phase of the study. Although they represent a wide range of
206
various demographic and clinical characteristics, the participants may represent a population that is more
ACCEPTED MANUSCRIPT
likely to be engaged in social support programs. This connection to resources may help to explain some of
208
the ceiling effects observed in the data set. There are also limitations inherent to the Graded Response
209
IRT model (GRM) chosen for the LIBRE Profile that are worth noting. In the GRM 2 parameter model,
210
the IRT scores from two different samples are not directly comparable; in order to make scores
211
comparable across samples we had to link the scores onto a common metric (33). However, we feel that
212
this limitation is worth the trade off because the 2-parameter approach provided us with the better model
213
fit (34). This approach has also been used throughout the literature, including the PROMIS initiative
214
(35,36).
SC
RI PT
207
Finally, because the LIBRE Profile is still in its early stages of development there is currently no
216
clinically meaningful guide to facilitate score interpretation beyond the item map example presented here.
217
Although future work will hopefully further this aim, the main guidance for score interpretation is based
218
upon the scale and score distribution (with a mean of 50 and standard deviation of 10).
M AN U
215
219 Conclusions
TE D
220
As a patient-reported outcome measure, the LIBRE Profile-SF can be used to better engage the
222
patient and inform treatment, contributing to a more complete rehabilitation process. The short form
223
version of the LIBRE Profile ensures accessibility when resources are scarce, either on an individual or
224
health care facility level. The LIBRE Profile-SF will allow more people access to the LIBRE Profile who
225
do not have the ability to administer the CAT. To obtain the LIBRE Profile SF, complete with scoring
226
instructions and conversion tables, email [email protected].
228
AC C
227
EP
221
ACCEPTED MANUSCRIPT
References
230 231
1.
Schneider JC, Qu HD. Neurologic and musculoskeletal complications of burn injuries. Phys Med Rehabil Clin N Am. 2011 May;22(2):261–275, vi.
232 233
2.
Lawrence JW, Mason ST, Schomer K, Klein MB. Epidemiology and impact of scarring after burn injury: a systematic review of the literature. J Burn Care Res. 2012 Feb;33(1):136–46.
234 235
3.
Blakeney PE, Rosenberg L, Rosenberg M, Faber AW. Psychosocial care of persons with severe burns. Burns. 2008 Jun;34(4):433–40.
236 237
4.
Madianos MG, Papaghelis M, Ioannovich J, Dafni R. Psychiatric Disorders in Burn Patients: A Follow-Up Study. Psychother Psychosom. 2001 Jan 3;70(1):30–7.
238 239 240
5.
Thombs BD, Notes LD, Lawrence JW, Magyar-Russell G, Bresnick MG, Fauerbach JA. From survival to socialization: a longitudinal study of body image in survivors of severe burn injury. J Psychosom Res. 2008 Feb;64(2):205–12.
241 242
6.
Esselman PC. Community integration outcome after burn injury. Phys Med Rehabil Clin N Am. 2011 May;22(2):351–356, vii.
243 244
7.
Mackey SP, Diba R, McKeown D, Wallace C, Booth S, Gilbert PM, et al. Return to work after burns: A qualitative research study. Burns. 2009 May;35(3):338–42.
245 246 247 248 249
8.
Mason ST, Esselman P, Fraser R, Schomer K, Truitt A, Johnson K. Return to Work After Burn Injury: A Systematic Review : Journal of Burn Care & Research [Internet]. LWW. [cited 2016 Nov 10]. Available from: http://journals.lww.com/burncareresearch/Fulltext/2012/01000/Return_to_Work_After_Burn_Injury ___A_Systematic.13.aspx
250 251
9.
Bianchi TL. Aspects of sexuality after burn injury: outcomes in men. J Burn Care Rehabil. 1997 Apr;18(2):183–186; discussion 182.
252 253 254
10.
Kvannli L, Finlay V, Edgar DW, Wu A, Wood FM. Using the Burn Specific Health Scale-Brief as a measure of quality of life after a burn—What score should clinicians expect? Burns. 2011 Feb;37(1):54–60.
255 256 257 258
11.
Ryan CM, Lee AF, Kazis LE, Shapiro GD, Schneider JC, Goverman J, et al. Is Real-Time Feedback of Burn-Specific Patient-Reported Outcome Measures in Clinical Settings Practical and Useful? A Pilot Study Implementing the Young Adult Burn Outcome Questionnaire. J Burn Care Res. 2016 Feb;37(1):64–74.
259 260 261
12.
McDonough CM, Ni P, Coster WJ, Haley SM, Jette AM. Development of an IRT-Based Short Form to Assess Applied Cognitive Function in Outpatient Rehabilitation. Am J Phys Med Rehabil. 2016 Jan;95(1):62–71.
262 263
13.
Cella D, Gershon R, Lai J-S, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16 Suppl 1:133–41.
264 265
14.
Willebrand M, Kildal M. A simplified domain structure of the burn-specific health scale-brief (BSHS-B): a tool to improve its value in routine clinical work. J Trauma. 2008 Jun;64(6):1581–6.
AC C
EP
TE D
M AN U
SC
RI PT
229
ACCEPTED MANUSCRIPT
15.
Kazis LE, Liang MH, Lee A, Ren XS, Phillips CB, Hinson M, et al. The development, validation, and testing of a health outcomes burn questionnaire for infants and children 5 years of age and younger: American Burn Association/Shriners Hospitals for Children. J Burn Care Rehabil. 2002 Jun;23(3):196–207.
270 271 272
16.
Daltroy LH, Liang MH, Phillips CB, Daugherty MB, Hinson M, Jenkins M, et al. American Burn Association/Shriners Hospitals for Children burn outcomes questionnaire: construction and psychometric properties. J Burn Care Rehabil. 2000 Feb;21(1 Pt 1):29–39.
273 274 275
17.
Meyer WJ, Lee AF, Kazis LE, Li N-C, Sheridan RL, Herndon DN, et al. Adolescent survivors of burn injuries and their parents’ perceptions of recovery outcomes: do they agree or disagree? J Trauma Acute Care Surg. 2012 Sep;73(3 Suppl 2):S213-220.
276 277 278 279
18.
Ryan CM, Schneider JC, Kazis LE, Lee A, Li N-C, Hinson M, et al. Benchmarks for multidimensional recovery after burn injury in young adults: the development, validation, and testing of the American Burn Association/Shriners Hospitals for Children young adult burn outcome questionnaire. J Burn Care Res. 2013 Jun;34(3):e121-142.
280 281 282
19.
Kazis LE, Marino M, Ni P, Soley Bori M, Amaya F, Dore E, et al. Development of the life impact burn recovery evaluation (LIBRE) profile: assessing burn survivors’ social participation. Qual Life Res. 2017 May 10.
283 284
20.
Marino M, Soley-Bori M, Jette AM, Slavin MD, Ryan CM, Schneider JC, et al. Measuring the Social Impact of Burns on Survivors. J Burn Care Res. 2017 Feb;38(1):e377–83.
285 286 287
21.
Marino M, Soley-Bori M, Jette AM, Slavin MD, Ryan CM, Schneider JC, et al. Development of a Conceptual Framework to Measure the Social Impact of Burns. J Burn Care Res. 2016 Dec;37(6):e569–78.
288 289 290
22.
Gibbons RD, Bock RD, Hedeker D, Weiss DJ, Segawa E, Bhaumik DK, et al. Full-Information Item Bifactor Analysis of Graded Response Data. Applied Psychological Measurement. 2007 Jan 1;31(1):4–19.
291 292
23.
Edelen MO, Reeve BB. Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res. 2007;16 Suppl 1:5–18.
293 294 295
24.
Orlando M, Thissen D. Further Investigation of the Performance of S - X2: An Item Fit Index for Use With Dichotomous Item Response Theory Models. Applied Psychological Measurement. 2003 Jul 1;27(4):289–98.
296 297 298
25.
Langer M. A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation (Unpublished doctoral dissertation). University of North Carolina, Chapel Hill; 2008.
299 300 301
26.
Woods CM, Cai L, Wang M. The Langer-Improved Wald Test for DIF Testing With Multiple Groups Evaluation and Comparison to Two-Group IRT. Educational and Psychological Measurement. 2012 Oct 31;0013164412464875.
302 303
27.
Edelen MO, Stucky BD, Chandra A. Quantifying “problematic” DIF within an IRT framework: application to a cancer stigma index. Qual Life Res. 2015 Jan;24(1):95–103.
AC C
EP
TE D
M AN U
SC
RI PT
266 267 268 269
ACCEPTED MANUSCRIPT
28.
Choi SW, Reise SP, Pilkonis PA, Hays RD, Cella D. Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Qual Life Res. 2010 Feb;19(1):125–36.
307 308 309
29.
Heinemann AW, Dijkers MP, Ni P, Tulsky DS, Jette A. Measurement properties of the Spinal Cord Injury-Functional Index (SCI-FI) short forms. Arch Phys Med Rehabil. 2014 Jul;95(7):1289– 1297.e5.
310 311
30.
Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care. 2000 Sep;38(9 Suppl):II28-42.
312 313
31.
Revicki DA, Cella DF. Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997 Aug;6(6):595–600.
314 315
32.
Thissen D, Nelson L, Rosa K, McLeod L. Item response theory for items scored in more than two categories. In: Test Scoring. Mahwah, NJ: Lawrence Erlbaum Associates; 2001. p. 141–86.
316 317
33.
Kolen MJ, Brennan RL. Test Equating, Scaling, and Linking - Methods and Practices. 2nd ed. New York: Springer; 2004.
318
34.
Thissen D, Wainer H, editors. Test Scoring. Mahwah, NJ: Lawrence Erlbaum Associates; 2001.
319 320 321
35.
Varni JW, Stucky BD, Thissen D, Dewitt EM, Irwin DE, Lai J-S, et al. PROMIS Pediatric Pain Interference Scale: an item response theory analysis of the pediatric pain item bank. J Pain. 2010 Nov;11(11):1109–19.
322 323 324
36.
Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult selfreported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010 Nov;63(11):1179–94.
SC
M AN U
TE D
EP AC C
325
RI PT
304 305 306
ACCEPTED MANUSCRIPT
Figure Legend
327
Figures 1a-1f. Test Information Function
328
Figure 2. Sample Item Map for Score Interpretation of the Family & Friends Scale
AC C
EP
TE D
M AN U
SC
RI PT
326
ACCEPTED MANUSCRIPT
1
Figures 1a-1f. Test Information Function
2
1a. Family & Friends
RI PT
25
FULL ITEM BANK
20
BURN SHORT FORM
SC
15 10 5 0 0
20
M AN U
TEST INFORMATION FUNCTION
30
40
60
80
100
T SCALE
3 1b. Social Interactions
TE D
4
60
FULL ITEM BANK
EP
50 40 30
BURN SHORT FORM
AC C
TEST INFORMATION FUNCTION
70
20 10 0
0
20
40
60 T SCALE
5 6
80
100
ACCEPTED MANUSCRIPT
7
1c. Social Activities
RI PT
20 FULL ITEM BANK
15
BURN SHORT FORM
SC
10 5 0 0
20
M AN U
TEST INFORMATION FUNCTION
25
40
60
80
100
T SCALE
8 9
1d. Work & Employment
TE D
45 40 35 30
FULL ITEM BANK BURN SHORT FORM
EP
25 20 15
AC C
TEST INFORMATION FUNCTION
50
10 5 0
0
20
40
60 T SCALE
10 11 12
80
100
ACCEPTED MANUSCRIPT
13
1e. Romantic Relationships
45
RI PT
40 35
FULL ITEM BANK
30
NORMATIVE/BURN BURN SHORT FORM SHORT FORM
25 20
SC
15 10 5 0 0
20
M AN U
TEST INFORMATION FUNCTION
50
40
60
80
100
T SCALE
14 15
1f. Sexual Relationships
TE D
20
FULL ITEM BANK BURN SHORT FORM
EP
15 10
AC C
TEST INFORMATION FUNCTION
25
5 0
0
20
40
60 T SCALE
16
80
100
ACCEPTED MANUSCRIPT
17
Figure 2. Sample Item Map for Score Interpretation of the Family & Friends Scale
RI PT
I have many friends in the city where I live. My family is comfortable talking about burns. I would rather be alone than with my family.
SC
Members of my family give me the support that I need.
I am comfortable being helped by my family. I don't like the way most family members act around me. I don't get along with my family.
TE D
As much as possible, I avoid members of my family.
M AN U
Members of my family enjoy meeting my friends.
Most family members are comfortable being with me.
AC C
18
EP
0
10
20
30
40
50 T Score
60
70
80
90
100
S0003-9993(17)31072-9
DOI:
10.1016/j.apmr.2017.06.037
Reference:
YAPMR 57001
To appear in:
ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION
Received Date: 7 June 2017 Accepted Date: 28 June 2017
Please cite this article as: Marino M, Dore EC, Ni P, Ryan CM, Schneider JC, Acton A, Jette AM, Kazis LE, Developing Item Response Theory-Based Short Forms to Measure the Social Impact of Burn Injuries, ARCHIVES OF PHYSICAL MEDICINE AND REHABILITATION (2017), doi: 10.1016/ j.apmr.2017.06.037. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Title: Developing Item Response Theory-Based Short Forms to Measure the Social Impact of
Running head: Short Forms for the Social Impact of Burns
RI PT
Burn Injuries
SC
Authors: Molly Marino, PhD, MPH1; Emily C. Dore MSW2; Pengsheng Ni, MD, MPH1; Colleen M. Ryan, MD3,4,5; Jeff C. Schneider, MD4,6 Amy Acton RN, BSN7; and Alan M. Jette PT PhD2 and Lewis E. Kazis, ScD1;
M AN U
Affiliations: (1) Boston University School of Public Health, Health & Disabilities Research Institute (2) Boston University School of Public Health, Center for the Assessment of Pharmaceutical Practices (CAPP) Department of Health Law, Policy, and Management (3) Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts. (4) Harvard Medical School, Boston, Massachusetts (5) Shriners Hospitals for Children –Boston®, Boston, Massachusetts (6) Department of Physical Medicine and Rehabilitation, Spaulding Rehabilitation Hospital, Boston, Massachusetts (7) The Phoenix Society for Burn Survivors.
Conflicts of Interest: None
TE D
Acknowledgments: This work is supported by National Institute on Disability, Independent Living, and Rehabilitation Research Award Number 90DP0055.
AC C
EP
Corresponding Author: Molly Marino, PhD, MPH Health and Disability Research Institute Boston University School of Public Health 715 Albany Street Talbot T5W Boston, MA 02118 Telephone: 617-638-1990 Fax: 617-638-1999 Email: [email protected]
ACCEPTED MANUSCRIPT
Title: Developing Item Response Theory-Based Short Forms to Measure the Social Impact of Burn
2
Injuries
3
Abstract:
4
Objective: To develop self-reported short forms for the LIBRE Profile.
5
Design: Participants were selected for inclusion in the short forms based upon the item parameters of
6
discrimination and average difficulty. Test information function, as well as floor and ceiling effects, were
7
evaluated for both the full item bank and the short forms.
8
Setting: Participants were recruited were from the Phoenix Society (a support network for burn
9
survivors), peer support networks, social media, and mailings.
SC
RI PT
1
Participants: 601 burn survivors over 18 years of age in North America.
11
Interventions: none
12
Main Outcome Measures: The LIBRE-Profile
13
Results: Ten-item short forms were developed to cover the six LIBRE Profile scales: Relationships with
14
Family & Friends, Social Interactions, Social Activities, Work & Employment, Romantic Relationships,
15
and Sexual Relationships. Ceiling effects were ≤15% for all scales; floor effects were all <1%. The
16
marginal reliability of the short forms ranged from 0.85-0.89.
17
Conclusions: The LIBRE Profile-Short Forms demonstrated credible psychometric properties. The short
18
form version provides a viable alternative to administering the LIBRE Profile when resources do not
19
allow for computer/internet access. The full item bank, Computerized Adaptive Test, and short forms are
20
all scored along the same metric and therefore scores are comparable regardless of mode of
21
administration.
22
Key Words: Item Response Theory, Burn Injury, Outcome Measurement, Questionnaires, Psychometrics
23
Abbreviations: Total Body Surface Area Burned (TBSA), Item Response Theory (IRT), Graded
24
Response Model (GRM) Computerized Adaptive Testing (CAT), Life Impact Burn Recovery Evaluation
25
(LIBRE), Short Form (SF), Item Characteristic Curves (ICCs), Differential Item Functioning (DIF), Test
26
Information Function (TIF).
AC C
EP
TE D
M AN U
10
ACCEPTED MANUSCRIPT
Burn survivors face a wide-range of burn-related challenges, including physical and
28
psychological issues that potentially impact their ability to reintegrate into society. Physical issues such as
29
joint stiffness, chronic pain, chronic itch, temperature intolerance, scarring, inability to sweat, and fatigue
30
may limit a person’s ability to be active and to join in community events (1,2). Psychological issues may
31
result from the burn survivor’s altered appearance or from an abrupt and unexpected decrease in
32
independence. Anxiety, depression, and post-traumatic stress are common in almost 50% of burn
33
survivors, with 20%-30% of adult burn survivors reporting moderate-to-severe psychological and/or
34
social difficulties (3,4). Any of these conditions alone could isolate the burn survivor, but the combination
35
can make recovery especially difficult. Body image, return to work, and sexual function are some of the
36
psychosocial areas that have so far been covered in burn research (2,3,5–8).
M AN U
SC
RI PT
27
Individuals with facial disfigurements, such as a face burn, avoid social situations as a way to
38
deal with difficult emotions (5). Dissatisfaction with body image is a common issue for burn survivors,
39
and is linked to depression symptoms and social issues (5). Burn survivors report attracting unwanted
40
stares and comments from strangers due to their burn scars. This stigmatization is thought to result in
41
negative body image, anxiety, and depression (2). While some survivors return to work eventually, about
42
50%-60% need a change in employment and 28% of all burn survivors never return to work at all (3,8).
43
Barriers include being self-conscious about appearance, fear of leaving home, and fear of the workplace
44
(6). Burn survivors of both genders also experience decreased sexual satisfaction, which may relate to the
45
emotional and physical ramifications of the burn injury (3,9).
EP
AC C
46
TE D
37
Although some of these issues have previously been identified, burn survivors continue to report
47
that there is a lack of social support and treatment options. Until now, there has not been a measure
48
focused exclusively on the social impact of a burn injury. Studies, clinical observations, and burn survivor
49
feedback have all identified the social and psychological difficulties encountered during the rehabilitation
50
process (3,5,10). The ability to measure these difficulties would allow the clinician and patient to have a
51
directed discussion about treatment, optimizing and personalizing a burn survivor’s recovery (10–13).
ACCEPTED MANUSCRIPT
The Burn Specific Health Scale (BSHS) has a limited domain structure that does not examine
53
psychosocial aspects in depth (14). The Burn Outcomes Questionnaires (BOQ) developed by Shriners
54
Burn Hospital and the American Burn Association measure outcomes based on specific age ranges. These
55
measures include the BOQ0-4, which focuses on children younger than 5 years of age (15), the BOQ5-18 for
56
children 5 to 18 years of age (16), the BOQ11-18 for adolescents 11 to 18 years (17), and the Young Adult
57
Burn Outcomes Questionnaire (YABOQ) for ages 19 to 30 years (18). In addition, generic function and
58
quality of life assessments are limited in their applicability to burn survivors since patients with burns
59
face a complex and unique constellation of impairments that involve multiple domains. These domains
60
deserve acknowledgement and recognition by condition-specific instruments focusing on burn
61
survivors (14).
62
LIBRE PROFILE:
M AN U
SC
RI PT
52
The LIBRE Profile is a new self-reported measure of social participation for burn survivors.
64
Items were initially developed based upon a comprehensive literature review and focus groups with
65
clinical experts and burn survivors. We administered the initial group of 192 items to a convenience
66
sample of 601 burn survivors who were age 18 or older, having survived a burn that was at least 5%
67
TBSA or burns to one of four critical areas (face, hands, feet, and genitals), living in the United States or
68
Canada, and were able to read and understand English. Details on the sample are available elsewhere (19–
69
21).
EP
TE D
63
Initial analyses included an Exploratory Factor Analysis to examine the factor loading patterns
71
among all 192 items, Confirmatory Factor Analysis to confirm unidimensionality of each domain, and
72
calibration of the items using the two parameter Graded Response Model (GRM) Item Response Theory
73
(IRT) Model (22–24). We also examined differential item functioning (DIF), to identify whether people
74
at the same estimated ability level in a content domain respond differently to the same item based on
75
another variable. DIF was conducted based on age, gender, race, and time since burn injury using a 2-step
76
IRT based DIF method, and examined the difference in the Item Characteristic Curves (ICCs) using the
AC C
70
ACCEPTED MANUSCRIPT
77
weighted Area Between the expected score Curves (wABC) to examine DIF impact. Items that
78
demonstrated DIF were retained with calibrations specific to the relevant characteristic (25–27). The six scales of the LIBRE Profile are Relationships with Family & Friends, Social Interactions,
80
Social Activities, Work & Employment, Romantic Relationships, and Sexual Relationships. The scales
81
contain questions and statements with three different response categories. The first were agreement
82
responses: Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree, N/A. The
83
second was a frequency scale: Never, Almost Never, Sometimes, Often, Always, N/A. And the third type
84
of category was related to degree: Not at All, A Little Bit, Somewhat, Quite a Bit, Very Much, N/A.
85
Person IRT scores are transformed to a T-score distribution where the mean=50, SD=10 based on the
86
average of the overall sample of burn subjects used for the calibration phase of the study (T score=person
87
score*10+50). Lower scores correspond to poorer performance on the scale.
M AN U
SC
RI PT
79
The LIBRE Profile was originally developed for administration through Computerized Adaptive
89
Testing (CAT). The CAT draws upon items from the entire instrument, applying an algorithm to choose
90
which items to pose to the participant based on previous responses. The advantage of this administration
91
is that it achieves accurate and precise scores while administering fewer questionnaire items than a
92
traditional full-length measure (13,28). However, not every clinical setting is able to use the CAT because
93
of limited computer resources or expertise (12,29). For this reason, we set out to develop fixed short
94
forms—a subset of questions from the larger instrument—which can be administered and scored on
95
paper.
EP
AC C
96
TE D
88
This paper discusses the development and assessment of the psychometric properties of the IRT-
97
based LIBRE Profile-Short Forms (SF). The LIBRE Profile-SF will allow more people to have access to
98
the tool who do not have the ability to administer the CAT. This study was approved by the Boston
99
University Institutional Review Board.
100 101
Methods
102
Sample:
ACCEPTED MANUSCRIPT
103 104
The LIBRE Profile-SFs were developed using the data from the same sample of 601 burn survivors who provided the information for the development of the initial LIBRE Profile.
105 Strategy for SF Construction:
RI PT
106
Among the candidate items in each LIBRE Profile item pool, we used IRT methods to select
108
questions for SF inclusion based upon high item discrimination, large score range coverage, matching
109
between the item score range and the sample distribution, and a distribution of average item difficulty
110
across the continuum of a scale. Item characteristic curves reflect the relationship between each item and
111
the latent trait that is being assessed. The item discrimination parameter reflects the steepness (slope) of
112
the item characteristic curve. The higher the discrimination parameter for an item, the more that item is
113
able to distinguish among subjects at a similar latent trait level (30,31). Items that displayed DIF were not
114
candidates for LIBRE Profile-SF.
M AN U
SC
107
We calculated the target test information (TIF) value for the whole SF that would be needed to
116
achieve the acceptable reliability of ≥0.80. TIF was weighted by the outpatient sample distribution (using
117
the formula: I = ʃinfo(Θ)g(Θ)d Θ, I) in order to select items with information matched to the score range
118
of the sample. The information value (I) was calculated for each item, then for all possible two item
119
combinations, using the item with the highest I, and each of the other items. The two items with the
120
highest I were then selected, and the process was repeated for 3 items. This process was repeated until all
121
possible items were exhausted.
122
Statistical Analysis
EP
AC C
123
TE D
115
After we arrived at the final items for each SF, we examined their content compared to all the
124
items of each of the six scales. We calculated the internal consistency reliability of all SFs to determine
125
the number of items in the final accepted SF.
126
Using the item parameters obtained from the full item bank analysis, we scored the SF for each
127
respondent and calculated the proportion of the sample at the ceiling and floor of the two administration
128
methods for each scale. Last, we compared the marginal reliability (calculated by score variance-average
ACCEPTED MANUSCRIPT
or squared standard error/score variance) and information provided (called the test information function)
130
for each SF and the full item bank across the entire score distribution for each content domain. To
131
facilitate scoring the SFs, we generated conversion tables between the summed raw score and the scale T-
132
score. For each raw score, we collected all likelihoods of the response patterns yielding the same raw
133
score and formed the joint likelihood, then applied Expected A Posteriori method to calculate the scale
134
score based on the joint likelihood (32).
RI PT
129
Finally, to provide an example of one way LIBRE Profile scores can be displayed in the future,
136
we generated score a sample item map, illustrating where a person’s score lies relative to the item content
137
across the Family & Friends short form scale.
138
Results:
M AN U
SC
135
139
The sample includes 601 burn survivors (Table 1); 54.74% were female, the mean age at survey
140
administration was 44.55 (SD 15.98) years, mean TBSA was 40.45%, and the average time since burn
141
injury was 15.36 years.
There are six LIBRE Profile short forms with 10 items each. Table 2 presents the item content of
143
all SFs, the item difficulty, and slope for each item. Average item difficulty was the most narrow for the
144
Romantic Relationships scale (36.7-41.62), and widest for the Social Interactions scale (29.91-42.88).
145
Score ranges (Table 3) indicate coverage of almost three standard deviations below the mean on all
146
scales, and between one and two standard deviations above the mean on all scales.
EP
TE D
142
Table 3 presents the comparisons of several key psychometric properties of the two measures—
148
the full item bank and SF subset of items. The marginal reliability of the full item bank ranged from 0.87
149
to 0.93. For the SFs, marginal reliability ranged from 0.85-0.89. Ceiling effects were ≤15% for all scales;
150
floor effects were all <1%.
AC C
147
151
Figures 1a-1f illustrate the Test Information Function (TIF) of the full item bank compared to the
152
SFs. There is some sacrifice in score precision for each SF compared to the full item bank, but the SFs
153
generate reliability above 0.9 for a wide range of scale scores. The peaks of the TIF curves occur at the
154
same score ranges, which is also where most of the scores are located. At a maximum, the Sexual
ACCEPTED MANUSCRIPT
155
Relationships SF TIF curve covers 76.9% of the full item bank; the least amount of coverage is the Social
156
Interactions SF, which covers only 42.9% of the full item bank TIF curve. Figure 2 displays one possible way scores can currently be interpreted, using the Family &
158
Friends scale as an example. The items are ordered from hardest at the top (most difficult) to easiest (least
159
difficulty) along the vertical axis. The responses are ordered along the horizontal axis indicating the T-
160
score of that scale. Each shadow bar represents the expected category score range. For example, the first
161
item (“I have many friends in the city where I live.”), the 5 shadow bars from left to right represent 5
162
response categories. For the “Strongly disagree” category, the corresponding T score range is from 0 to
163
24.7; for “Disagree,” the corresponding T score ranges from 24.7 to 38; for “Neither Agree nor Disagree,”
164
the corresponding T score ranges from 38 to 48.6; for “Agree,” the corresponding T score ranges from
165
48.6 to 62.7; for “Strongly Agree,” the corresponding T score ranges from 62.7 to 100. A figure such as
166
this facilitates score interpretation by presenting a person’s score relative to all of the item content.
M AN U
SC
RI PT
157
167 Discussion
TE D
168
The LIBRE Profile-SF is made up of six ten-item short forms, one for each of the LIBRE Profile
170
scales. Results suggest the LIBRE Profile-SF can be an accurate and reliable way to measure social
171
participation in burn survivors.
EP
169
There is some loss of precision with the SF compared with the entire item bank. This is expected,
173
since the SFs contain fewer items. However, the SFs all have ceiling effects less than 15%, and the
174
marginal reliability ranged from 0.85 to 0.89, which is considered credible (30). These findings show that
175
the LIBRE Profile-SF demonstrates appropriate psychometric properties. The differences in scores occur
176
at the extremes, with lower reliability for people who have either very low or very high scores. Therefore,
177
although the LIBRE Profile-SF has fewer items compared to the entire item bank, it also can provide an
178
accurate level score as a substitute for the full item bank.
AC C
172
179
It is important to keep measures brief so that they can be used in a fast-paced clinical setting. We
180
developed ten-item short forms as an appropriate balance between precision and feasibility. Participants
ACCEPTED MANUSCRIPT
would likely prefer a precise score without spending a lengthy amount of time responding to items. More
182
items in a SF increase the precision of the score, but would also increase the respondent burden. A ten-
183
item SF provides an accurate score and is still feasible to administer. Other SFs could be developed using
184
different items or a different number of items (more or fewer).
RI PT
181
The Relationships with Family & Friends SF contains eight items about family and two items
186
about friends, compared to the entire item pool which contains 14 family items and 10 friend items,
187
suggesting scores in that domain are driven by items related to family. Four of the Social Interactions
188
items focus on strangers or meeting new people, reflecting the presence of that aspect of the domain in the
189
full scale. All three of the items in the full Social Activities scale that mention burns are included in the
190
SF, which is beneficial to keeping the items relevant to the burn survivor experience. The Work &
191
Employment SF focuses primarily on global satisfaction with work performance (8 of the 10 items),
192
whereas the full item bank has more of a balance between those items (11) and others involving social
193
dynamics in the workplace (8). The full item bank of the Romantic Relationships scale is the largest in the
194
LIBRE Profile (28 items), whereas the Sexual Relationships is one of the smallest (15 items); item
195
content for both SFs are generally representative of the full scales.
TE D
M AN U
SC
185
The short forms are meant to be an alternative to the CAT for situations where it cannot be
197
administered due to lack of access to a computer or the internet. The SFs are scored on the same metric as
198
the CAT. This means the scores of the SFs are comparable to those of the CAT on the similar metric. This
199
is important so that results from the two instruments can be analyzed within the same context as all other
200
LIBRE Profile scores. Scores will inform patients as to how they are faring within a context of social
201
participation following a burn injury. Currently, scores can be compared to those of other burn survivors
202
from the survey of 601 subjects as part of the original LIBRE Profile calibration study.
203
Study Limitations
AC C
EP
196
204
Limitations to this study include the sampling method. A convenience sample of 601 burn
205
survivors participated in the calibration phase of the study. Although they represent a wide range of
206
various demographic and clinical characteristics, the participants may represent a population that is more
ACCEPTED MANUSCRIPT
likely to be engaged in social support programs. This connection to resources may help to explain some of
208
the ceiling effects observed in the data set. There are also limitations inherent to the Graded Response
209
IRT model (GRM) chosen for the LIBRE Profile that are worth noting. In the GRM 2 parameter model,
210
the IRT scores from two different samples are not directly comparable; in order to make scores
211
comparable across samples we had to link the scores onto a common metric (33). However, we feel that
212
this limitation is worth the trade off because the 2-parameter approach provided us with the better model
213
fit (34). This approach has also been used throughout the literature, including the PROMIS initiative
214
(35,36).
SC
RI PT
207
Finally, because the LIBRE Profile is still in its early stages of development there is currently no
216
clinically meaningful guide to facilitate score interpretation beyond the item map example presented here.
217
Although future work will hopefully further this aim, the main guidance for score interpretation is based
218
upon the scale and score distribution (with a mean of 50 and standard deviation of 10).
M AN U
215
219 Conclusions
TE D
220
As a patient-reported outcome measure, the LIBRE Profile-SF can be used to better engage the
222
patient and inform treatment, contributing to a more complete rehabilitation process. The short form
223
version of the LIBRE Profile ensures accessibility when resources are scarce, either on an individual or
224
health care facility level. The LIBRE Profile-SF will allow more people access to the LIBRE Profile who
225
do not have the ability to administer the CAT. To obtain the LIBRE Profile SF, complete with scoring
226
instructions and conversion tables, email [email protected].
228
AC C
227
EP
221
ACCEPTED MANUSCRIPT
References
230 231
1.
Schneider JC, Qu HD. Neurologic and musculoskeletal complications of burn injuries. Phys Med Rehabil Clin N Am. 2011 May;22(2):261–275, vi.
232 233
2.
Lawrence JW, Mason ST, Schomer K, Klein MB. Epidemiology and impact of scarring after burn injury: a systematic review of the literature. J Burn Care Res. 2012 Feb;33(1):136–46.
234 235
3.
Blakeney PE, Rosenberg L, Rosenberg M, Faber AW. Psychosocial care of persons with severe burns. Burns. 2008 Jun;34(4):433–40.
236 237
4.
Madianos MG, Papaghelis M, Ioannovich J, Dafni R. Psychiatric Disorders in Burn Patients: A Follow-Up Study. Psychother Psychosom. 2001 Jan 3;70(1):30–7.
238 239 240
5.
Thombs BD, Notes LD, Lawrence JW, Magyar-Russell G, Bresnick MG, Fauerbach JA. From survival to socialization: a longitudinal study of body image in survivors of severe burn injury. J Psychosom Res. 2008 Feb;64(2):205–12.
241 242
6.
Esselman PC. Community integration outcome after burn injury. Phys Med Rehabil Clin N Am. 2011 May;22(2):351–356, vii.
243 244
7.
Mackey SP, Diba R, McKeown D, Wallace C, Booth S, Gilbert PM, et al. Return to work after burns: A qualitative research study. Burns. 2009 May;35(3):338–42.
245 246 247 248 249
8.
Mason ST, Esselman P, Fraser R, Schomer K, Truitt A, Johnson K. Return to Work After Burn Injury: A Systematic Review : Journal of Burn Care & Research [Internet]. LWW. [cited 2016 Nov 10]. Available from: http://journals.lww.com/burncareresearch/Fulltext/2012/01000/Return_to_Work_After_Burn_Injury ___A_Systematic.13.aspx
250 251
9.
Bianchi TL. Aspects of sexuality after burn injury: outcomes in men. J Burn Care Rehabil. 1997 Apr;18(2):183–186; discussion 182.
252 253 254
10.
Kvannli L, Finlay V, Edgar DW, Wu A, Wood FM. Using the Burn Specific Health Scale-Brief as a measure of quality of life after a burn—What score should clinicians expect? Burns. 2011 Feb;37(1):54–60.
255 256 257 258
11.
Ryan CM, Lee AF, Kazis LE, Shapiro GD, Schneider JC, Goverman J, et al. Is Real-Time Feedback of Burn-Specific Patient-Reported Outcome Measures in Clinical Settings Practical and Useful? A Pilot Study Implementing the Young Adult Burn Outcome Questionnaire. J Burn Care Res. 2016 Feb;37(1):64–74.
259 260 261
12.
McDonough CM, Ni P, Coster WJ, Haley SM, Jette AM. Development of an IRT-Based Short Form to Assess Applied Cognitive Function in Outpatient Rehabilitation. Am J Phys Med Rehabil. 2016 Jan;95(1):62–71.
262 263
13.
Cella D, Gershon R, Lai J-S, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16 Suppl 1:133–41.
264 265
14.
Willebrand M, Kildal M. A simplified domain structure of the burn-specific health scale-brief (BSHS-B): a tool to improve its value in routine clinical work. J Trauma. 2008 Jun;64(6):1581–6.
AC C
EP
TE D
M AN U
SC
RI PT
229
ACCEPTED MANUSCRIPT
15.
Kazis LE, Liang MH, Lee A, Ren XS, Phillips CB, Hinson M, et al. The development, validation, and testing of a health outcomes burn questionnaire for infants and children 5 years of age and younger: American Burn Association/Shriners Hospitals for Children. J Burn Care Rehabil. 2002 Jun;23(3):196–207.
270 271 272
16.
Daltroy LH, Liang MH, Phillips CB, Daugherty MB, Hinson M, Jenkins M, et al. American Burn Association/Shriners Hospitals for Children burn outcomes questionnaire: construction and psychometric properties. J Burn Care Rehabil. 2000 Feb;21(1 Pt 1):29–39.
273 274 275
17.
Meyer WJ, Lee AF, Kazis LE, Li N-C, Sheridan RL, Herndon DN, et al. Adolescent survivors of burn injuries and their parents’ perceptions of recovery outcomes: do they agree or disagree? J Trauma Acute Care Surg. 2012 Sep;73(3 Suppl 2):S213-220.
276 277 278 279
18.
Ryan CM, Schneider JC, Kazis LE, Lee A, Li N-C, Hinson M, et al. Benchmarks for multidimensional recovery after burn injury in young adults: the development, validation, and testing of the American Burn Association/Shriners Hospitals for Children young adult burn outcome questionnaire. J Burn Care Res. 2013 Jun;34(3):e121-142.
280 281 282
19.
Kazis LE, Marino M, Ni P, Soley Bori M, Amaya F, Dore E, et al. Development of the life impact burn recovery evaluation (LIBRE) profile: assessing burn survivors’ social participation. Qual Life Res. 2017 May 10.
283 284
20.
Marino M, Soley-Bori M, Jette AM, Slavin MD, Ryan CM, Schneider JC, et al. Measuring the Social Impact of Burns on Survivors. J Burn Care Res. 2017 Feb;38(1):e377–83.
285 286 287
21.
Marino M, Soley-Bori M, Jette AM, Slavin MD, Ryan CM, Schneider JC, et al. Development of a Conceptual Framework to Measure the Social Impact of Burns. J Burn Care Res. 2016 Dec;37(6):e569–78.
288 289 290
22.
Gibbons RD, Bock RD, Hedeker D, Weiss DJ, Segawa E, Bhaumik DK, et al. Full-Information Item Bifactor Analysis of Graded Response Data. Applied Psychological Measurement. 2007 Jan 1;31(1):4–19.
291 292
23.
Edelen MO, Reeve BB. Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res. 2007;16 Suppl 1:5–18.
293 294 295
24.
Orlando M, Thissen D. Further Investigation of the Performance of S - X2: An Item Fit Index for Use With Dichotomous Item Response Theory Models. Applied Psychological Measurement. 2003 Jul 1;27(4):289–98.
296 297 298
25.
Langer M. A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation (Unpublished doctoral dissertation). University of North Carolina, Chapel Hill; 2008.
299 300 301
26.
Woods CM, Cai L, Wang M. The Langer-Improved Wald Test for DIF Testing With Multiple Groups Evaluation and Comparison to Two-Group IRT. Educational and Psychological Measurement. 2012 Oct 31;0013164412464875.
302 303
27.
Edelen MO, Stucky BD, Chandra A. Quantifying “problematic” DIF within an IRT framework: application to a cancer stigma index. Qual Life Res. 2015 Jan;24(1):95–103.
AC C
EP
TE D
M AN U
SC
RI PT
266 267 268 269
ACCEPTED MANUSCRIPT
28.
Choi SW, Reise SP, Pilkonis PA, Hays RD, Cella D. Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Qual Life Res. 2010 Feb;19(1):125–36.
307 308 309
29.
Heinemann AW, Dijkers MP, Ni P, Tulsky DS, Jette A. Measurement properties of the Spinal Cord Injury-Functional Index (SCI-FI) short forms. Arch Phys Med Rehabil. 2014 Jul;95(7):1289– 1297.e5.
310 311
30.
Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care. 2000 Sep;38(9 Suppl):II28-42.
312 313
31.
Revicki DA, Cella DF. Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997 Aug;6(6):595–600.
314 315
32.
Thissen D, Nelson L, Rosa K, McLeod L. Item response theory for items scored in more than two categories. In: Test Scoring. Mahwah, NJ: Lawrence Erlbaum Associates; 2001. p. 141–86.
316 317
33.
Kolen MJ, Brennan RL. Test Equating, Scaling, and Linking - Methods and Practices. 2nd ed. New York: Springer; 2004.
318
34.
Thissen D, Wainer H, editors. Test Scoring. Mahwah, NJ: Lawrence Erlbaum Associates; 2001.
319 320 321
35.
Varni JW, Stucky BD, Thissen D, Dewitt EM, Irwin DE, Lai J-S, et al. PROMIS Pediatric Pain Interference Scale: an item response theory analysis of the pediatric pain item bank. J Pain. 2010 Nov;11(11):1109–19.
322 323 324
36.
Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult selfreported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010 Nov;63(11):1179–94.
SC
M AN U
TE D
EP AC C
325
RI PT
304 305 306
ACCEPTED MANUSCRIPT
Figure Legend
327
Figures 1a-1f. Test Information Function
328
Figure 2. Sample Item Map for Score Interpretation of the Family & Friends Scale
AC C
EP
TE D
M AN U
SC
RI PT
326
ACCEPTED MANUSCRIPT
1
Figures 1a-1f. Test Information Function
2
1a. Family & Friends
RI PT
25
FULL ITEM BANK
20
BURN SHORT FORM
SC
15 10 5 0 0
20
M AN U
TEST INFORMATION FUNCTION
30
40
60
80
100
T SCALE
3 1b. Social Interactions
TE D
4
60
FULL ITEM BANK
EP
50 40 30
BURN SHORT FORM
AC C
TEST INFORMATION FUNCTION
70
20 10 0
0
20
40
60 T SCALE
5 6
80
100
ACCEPTED MANUSCRIPT
7
1c. Social Activities
RI PT
20 FULL ITEM BANK
15
BURN SHORT FORM
SC
10 5 0 0
20
M AN U
TEST INFORMATION FUNCTION
25
40
60
80
100
T SCALE
8 9
1d. Work & Employment
TE D
45 40 35 30
FULL ITEM BANK BURN SHORT FORM
EP
25 20 15
AC C
TEST INFORMATION FUNCTION
50
10 5 0
0
20
40
60 T SCALE
10 11 12
80
100
ACCEPTED MANUSCRIPT
13
1e. Romantic Relationships
45
RI PT
40 35
FULL ITEM BANK
30
NORMATIVE/BURN BURN SHORT FORM SHORT FORM
25 20
SC
15 10 5 0 0
20
M AN U
TEST INFORMATION FUNCTION
50
40
60
80
100
T SCALE
14 15
1f. Sexual Relationships
TE D
20
FULL ITEM BANK BURN SHORT FORM
EP
15 10
AC C
TEST INFORMATION FUNCTION
25
5 0
0
20
40
60 T SCALE
16
80
100
ACCEPTED MANUSCRIPT
17
Figure 2. Sample Item Map for Score Interpretation of the Family & Friends Scale
RI PT
I have many friends in the city where I live. My family is comfortable talking about burns. I would rather be alone than with my family.
SC
Members of my family give me the support that I need.
I am comfortable being helped by my family. I don't like the way most family members act around me. I don't get along with my family.
TE D
As much as possible, I avoid members of my family.
M AN U
Members of my family enjoy meeting my friends.
Most family members are comfortable being with me.
AC C
18
EP
0
10
20
30
40
50 T Score
60
70
80
90
100