- Email: [email protected]
Development and Evaluation of the Activities Measure for Upper Limb Amputees
Archives of Physical Medicine and Rehabilitation journal homepage: www.archives-pmr.org Archives of Physical Medicine and Rehabilitation 2013;94:488-94
ORIGINAL ARTICLE
Development and Evaluation of the Activities Measure for Upper Limb Amputees Linda Resnik, PT, PhD,a,b Laurel Adams, MOT, OTR/L,c Matthew Borgia, MS,a Jemy Delikat, MOT, OTR/L,c Roxanne Disla, OTR,d Christopher Ebner, MS, OTR,e,f Lisa Smurr Walters, MS, OTR/L, CHTe From the aProvidence Veterans Administration Medical Center, Providence, RI; bHealth Services, Policy and Practice Brown University, Providence, RI; cHealth Services Research and Development/Rehabilitation Research and Development Center of Excellence, James A. Haley Veterans Hospital, Tampa, FL; dNew York Harbor Healthcare SystemeManhattan Veterans Administration, New York, NY; eCenter for the Intrepid, Department of Orthopaedics and Rehabilitation, Brooke Army Medical Center, Fort Sam Houston, TX; fcurrently with Warrior Transition Battalion-Europe, Nachrichten Kaserne, Heidelberg Rohrbach, Germany.
Abstract Objectives: (1) To develop a measure of activities for adults with upper limb amputation: the Activities Measure for Upper Limb Amputees (AM-ULA); and (2) to conduct initial psychometric evaluation of the measure. Design: This was a cohort study where the prototype measure was administered twice within 1 week. Tests were videotaped and graded by 2 independent raters. Interrater reliability, test-retest reliability, internal consistency, and minimal detectable change were estimated. Known group validity was examined using analyses of variance comparing scores of transradial, transhumeral, and shoulder level amputees. Convergent validity was examined by correlating AM-ULA scores with dexterity tests and self-reported function. Setting: Hospital outpatient. Participants: Subjects (NZ52) with upper limb amputation. Interventions: Not applicable. Main Outcome Measures: Not applicable. Results: Intraclass correlation coefficients (ICCs) for test-retest reliability were .88 to .91. ICCs for interrater reliability were .84 to .89. Cronbach alphas were .89 to .91. The minimal detectable change at the 90% confidence interval was 3.7 points. Subjects with more distal levels of limb loss had better scores than those with more proximal levels (P<.01). The AM-ULA was moderately correlated with most dexterity tests and self-reported function. Conclusions: The AM-ULA is a new measure of activity performance for adults with upper limb amputation that considers task completion, speed, movement quality, skillfulness of prosthetic use, and independence in its rating system. It has good interrater reliability, test-retest reliability, and demonstrated known group validity. Archives of Physical Medicine and Rehabilitation 2013;94:488-94 ª 2013 by the American Congress of Rehabilitation Medicine
Outcome measures are needed to assess rehabilitation progress and document effectiveness of prosthetic devices.1-4 Yet, it is challenging to systematically collect or analyze outcomes data for persons with upper limb amputation, because there are few measures developed for or validated with adults, and limited research to guide
Supported by the Office of Research and Development Rehabilitation Research and Development Service, Department of Veterans Affairs (grant no. A6780I). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.
measure selection and interpretation. Although some instruments specific to adults with upper limb amputation do exist, the majority were developed for and used with pediatric amputees.1,5 Efforts to develop consensus on outcome measures for upper limb amputees coalesced in 2005 and continued through an Upper Limb Prosthetic Outcomes Measures (ULPOM) Working Group. The ULPOM Working Group aimed to develop a toolkit of validated measures addressing each major domain of the International Classification of Functioning, Disability and Health (ICF)6 (ie, body structures/functions, activities, participation), and identify the need for new measures.4 This group considered Wright’s comprehensive
0003-9993/13/$36 - see front matter ª 2013 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2012.10.004
Activities measure for upper limb amputees review5 of existing measures, recommended several measures, and concluded that there was a need for new tests of function for adults with amputation, as well as new measures for use with higher-level amputees, bilateral amputees, and body-powered users.7 Wright’s review5 reported that there were only 7 measures developed for adults. Furthermore, most measures were not validated with amputees, and 5 did not assess activities. The Orthotics and Prosthetics Users’ Survey (OPUS) Upper Extremity Functional Status (UEFS) module8 was the only prosthetic-specific measure of functional activities, and the Disabilities of Arm, Shoulder and Hand Questionnaire (DASH)9 was the only generic measure of upper limb functional activities. In 2010, Lindner et al10 summarized psychometric properties of 8 ULPOM and classified their content by linking items to ICF taxonomy. Lindner10 identified 3 measures appropriate for adults: Assessment of Capacity for Myoelectric Control (ACMC),11,12 UEFS,8 and Trinity Amputation and Prosthesis Experience Scales (TAPES).13 Both reviews drew similar conclusions about available instruments and recommended 3 measures: ACMC, OPUS,8 and TAPES. Additionally, Wright5 suggested 1 generic measure not on Lindner’s list10: the DASH.9 Together, these reviews highlighted the lack of objective, performance-based measures for adults. Performance-based measures are scored by an outside observer and may consist of timed-based tests or measures of performance using a rating scale. Self-reported measures are generally questionnaires completed by the persons themselves, although there may be special forms developed for proxy use. Performance-based measures are often considered more objective, while self-reported measures are considered more subjective. Both types are important in assessing outcomes. The only performance-based activity measure recommended by the ULPOM Working Group and the 2 literature reviews was the ACMC, a measure appropriate only for users of myoelectric prostheses. All other recommended measures were self-reported. The development of a new performance-based measure of activities for upper limb amputees began as part of the Department of Veterans Affairs (VA) Study to Optimize the DEKA Arm.14,a Investigators did not choose the ACMC, because the DEKA Arm may be controlled using foot controls, electromyography, and body-powered elements. This article reports on 2 study purposes: (1) to develop a new outcome measure of activities for adults with upper limb amputation, the Activities Measure for Upper Limb Amputees (AM-ULA) and (2) to conduct initial psychometric evaluation of the measure.
List of abbreviations: ACMC AM-ULA ANOVA DASH ICC ICF MDC MDC90 MDC95 OPUS TAPES UEFS ULPOM VA
Assessment of Capacity for Myoelectric Control Activities Measure for Upper Limb Amputees analysis of variance Disabilities of Arm, Shoulder and Hand Questionnaire intraclass correlation coefficient International Classification of Functioning, Disability and Health minimal detectable change minimal detectable change at the 90% confidence interval minimal detectable change at the 95% confidence interval Orthotics and Prosthetics Users’ Survey Trinity Amputations and Prosthetics Experience Scale Upper Extremity Functional Status Upper Limb Prosthetic Outcomes Measures Department of Veterans Affairs
www.archives-pmr.org
489
Methods Initial prototype development A set of activities from the widely used Atkins activities of daily living checklist was selected for the initial prototype.15,16 To further face validity, activities were chosen that were also included in the UEFS from the OPUS to represent a range of everyday functional skills.8 A simple grading scale, similar to that employed by Atkins and Meier15 was employed. Basic instructions for tasks were constructed. Therapists read the following instructions prior to each activity: “I want to see how you do some everyday activities. Please use your prosthetic arm to do these activities. Please use your sound arm only on those activities that require two hands.”
Evaluation of the prototype measure The first prototype activity measure was used in the testing of early subjects in the VA Study to Optimize the DEKA Arm.14 Therapists were oriented to the measure and provided written grading instructions. All testing was videotaped. During the first few months, site therapists and the first author (L.R.) independently graded subjects’ performance, and results were compared. This led us to conclude that more specific instructions for tasks and grading were needed to enable consistency in administration and scoring.
Revision of the prototype measure A series of conference calls with site occupational therapists helped identify and refine subtasks within activity items and scoring criteria (see supplemental appendix S1, available online only at the Archives website: www.archives-pmr.org). A standardized set of instructions was developed for each item (see supplemental appendix S2, available online only at the Archives website: www. archives-pmr.org). Initial discussions were used to gain insight into implicit elements used when assigning grades. Therapists watched a videotaped testing session and graded subject task performance. The group then watched the video together and discussed each item to understand the rationale for assigned grades. After each session, the first author (L.R.) circulated the list of elements therapists identified when assigning grades and asked for confirmation. All agreed that excellent should be used only when performance was comparable with that of a sound limb. We expected that use of a high bar would avoid a ceiling effectd whereby there would be no room for measurable improvement with advances in technology. Scoring criteria were revised and refined in 2 subsequent conference calls after viewing, grading, and reviewing new videos. After review and discussion of the final video, the first author (L.R.) concluded that the instructions were acceptable, because no additional scoring considerations were suggested. Final scoring criteria considered the: (1) extent of completion of all activity subtasks; (2) speed of completion; (3) movement quality; (4) skillfulness of prosthetic use and control over voluntary grip functions; and (5) independence. A synopsis of scoring criteria is subsequently presented. Scoring criteria Application of scoring criteria is summarized in supplemental appendix S2. When implementing scoring, the lowest score in any category is the one that is awarded for the overall score.
490 Completion of all subtask elements All subparts of a task should be completed to assign a grade higher than 0 (unable to perform). Thus, if the amputee could perform every aspect of a task except one, for example taking the cap off the toothpaste tube, he would be graded a 0 for the task of brushing teeth. Speed of completion (as compared with nondisabled) Speed of task performance is a relevant element. Amputees who perform tasks more slowly would be scored lower than those who perform tasks more quickly. Movement quality Movement quality, including awkward-looking movements caused by either inherent limitations of the prosthesis, poor planning, lack of prepositioning, or by compensatory movement strategies is important to consider when grading prosthetic use. The bar for the highest grade of movement quality is natural-looking movement similar to that of a nondisabled upper limb. Skillfulness of prosthetic use Subjects should be scored higher if they perform tasks using the prosthesis as a prime mover or an assist, rather than a passive stabilizer. Skillful control of the terminal device is a critical aspect of skilled use, and instances of involuntary loss of grip should be considered in the scoring system. Independence The last grading criterion is whether or not the amputee needed an assistive device or piece of adaptive equipment to complete the activity. Equipment can and should be used in testing, but prosthetic users cannot achieve the highest grade of excellent if they use equipment to complete a task.
Psychometric evaluation of the AM-ULA Therapists for the VA Study to Optimize the DEKA Arm administered the revised prototype for the AM-ULA to 49 subjects using their current prostheses on 2 occasions within 1 week. The study was approved by the Institutional Review Boards of the Providence VA Medical Center, the NY VA Health Harbor System, the James Haley VA Medical Center, the Long Beach VA Medical Center, and the Center for the Intrepid, Brooke Army Medical Center. Tests were videotaped and independently rated by 2 independent raters. Six site occupational therapists graded the performance of 16 subjects. All site therapists had at least 3 years of clinical experience; several had over 10 years experience with a range of experience in prosthetic rehabilitation from less than 1 year to more than 10 years. All had been trained in administration of standardized tests and measures for upper limb amputees as part of the research study protocol. Independent rater training The independent raters were certified hand therapists: 1 was an occupational therapist, the other was a physical therapist. Both had over 15 years of clinical experience and were experienced with prosthetic rehabilitation. Independent rater training took place over three 2-hour sessions. At the first session, the raters were introduced to the scoring system, viewed a videotaped test subject, and discussed task scoring. Scoring criteria were reviewed a second time after which videotapes of 2 additional subjects were viewed and discussed. At the second session, raters independently rated testing videotapes of 2 pilot subjects. Discrepancies in scores
L. Resnik et al were discussed. Prior to the third session, raters watched test videotapes of 2 additional subjects and independently rated performance. Scores were reviewed and discussed. At the end of the third training session, it was apparent that raters had good familiarity with each test and scoring criteria.
Statistical analyses Examination of test-retest reliability Test-retest reliability comparing scores for visits 1 and 2 were calculated using repeated-measures analysis of variance (ANOVA) and Shrout and Fleiss ICC (type3,1), a 2-way mixed model, single measure of reliability. Test-retest reliability of the scores of each rater was evaluated separately. Examination of interrater reliability Interrater reliability of the certified hand therapist raters at visit 1 were calculated using an intraclass correlation coefficient (ICC2,1), a 2-way random-effects single measure of reliability where the target and the raters are considered random effects. ICC2,1 allows for the generalizability of the findings to all possible raters. Consistency of ratings between the 2 certified hand therapist raters and the 6 onsite therapists was examined using ICC2,2, a 2-way mixed model of average measure reliability. For this comparison of data on only 16 subjects, mean rating scores from both visits 1 and 2 were used to decrease error variance and increase reliability estimates, because means are considered better estimates of true scores.17 Selection of items for final measure Only those items that had at least moderate (0.5) test-retest and interrater reliability were included in the final version of the AM-ULA.18 Table 1 and 2
Characteristics of subjects who completed both visits 1
Characteristic
Value
Age, mean SD (range) Sex Male Female Race White Other Veteran Nonveteran Veteran Active duty Prosthetic user (active device only) Not current user Full time Part time Type of prosthesis used for testing Body powered Myoelectric Hybrid Amputation of dominant side
45.816.5 (21.5e82.8) 44 (89.8) 5 (10.3) 44 (89.8) 5 (10.2) 10 (20.4) 27 (55.1) 12 (24.5) 1 (2.0) 31 (63.3) 17 (34.7) 25 20 4 26
(51.0) (40.8) (8.1) (54.2)*
NOTE. Values are n (%) or as otherwise indicated. * One subject was a bilateral congenital amputee and was omitted from this count.
www.archives-pmr.org
Activities measure for upper limb amputees Table 2
491
Test-retest and interrater reliability of AM-ULA items Test-Retest ICC
Interrater
Rater 1
Rater 2
Raters 1 and 2
Raters 1, 2, and Site Therapist
Item
ICC3,1
ICC3,1
ICC2,1
ICC2,2
Brush teethy Brush hair Put on t-shirt Remove t-shirt Button shirt Zip jacket Socks Tie shoes Use cup Use fork Use spoon Cut meaty Pour soda Write word Use scissors Door knob Key in locky Carry laundryy Use phone Hammer Fold towel Open envelopey Stir bowly Reach overhead Total score (24 items) Total score (revised scale 18 items)*
0.25 0.73 0.66 0.76 0.81 0.76 0.79 0.72 0.72 0.53 0.65 0.19 0.54 0.85 0.68 0.67 0.46 0.47 0.65 0.52 0.53 0.47 0.24 0.63 0.91 0.91
0.43 0.68 0.75 0.95 0.84 0.86 0.61 0.86 0.73 0.65 0.72 0.29 0.59 0.74 0.52 0.67 0.69 0.47 0.55 0.64 0.72 0.42 0.34 0.61 0.91 0.88
0.54 0.72 0.62 0.64 0.81 0.86 0.69 0.85 0.77 0.65 0.64 0.85 0.84 0.73 0.77 0.47 0.70 0.47 0.65 0.82 0.60 0.74 0.63 0.81 0.86 0.84
0.79 0.93 0.89 0.79 0.90 0.95 0.87 0.91 0.85 0.78 0.79 0.86 0.84 0.88 0.91 0.89 0.69 0.79 0.87 0.86 0.85 0.91 0.83 0.94 0.95 0.95
(0.07 to 0.53) (0.56 to 0.84) (0.26 to 0.87) (0.29 to 0.94) (0.63 to 0.91) (0.60 to 0.86) (0.60 to 0.90) (0.55 to 0.84) (0.55 to 0.83) (0.27 to 0.72) (0.44 to 0.79) (0.21 to 0.53) (0.28 to 0.72) (0.64 to 0.94) (0.49 to 0.82) (0.29 to 0.87) (0.08 to 0.73) (0.21 to 0.67) (0.45 to 0.79) (0.27 to 0.71) (0.29 to 0.71) (0.21 to 0.67) (0.05 to 0.48) (0.41 to 0.79) (0.85 to 0.95) (0.84 to 0.95)
(0.15 to 0.64) (0.49 to 0.81) (0.45 to 0.90) (0.78 to 0.99) (0.65 to 0.93) (0.75 to 0.92) (0.28 to 0.80) (0.74 to 0.92) (0.56 to 0.85) (0.43 to 0.80) (0.54 to 0.83) (e0.10 to 0.60) (0.36 to 0.76) (0.41 to 0.90) (0.27 to 0.71) (0.26 to 0.87) (0.42 to 0.84) (0.20 to 0.67) (0.30 to 0.73) (0.42 to 0.79) (0.54 to 0.84) (0.14 to 0.63) (0.06 to 0.57) (0.35 to 0.78) (0.85 to 0.95) (0.80 to 0.93)
(0.29 to 0.72) (0.55 to 0.83) (0.21 to 0.84) (0.10 to 0.89) (0.61 to 0.91) (0.77 to 0.92) (0.45 to 0.84) (0.74 to 0.92) (0.61 to 0.87) (0.43 to 0.79) (0.44 to 0.78) (0.71 to 0.93) (0.72 to 0.91) (0.40 to 0.89) (0.62 to 0.87) (0.01 to 0.78) (0.45 to 0.85) (0.21 to 0.66) (0.45 to 0.79) (0.69 to 0.90) (0.37 to 0.76) (0.58 to 0.85) (0.43 to 0.77) (0.68 to 0.90) (0.76 to 0.92) (0.74 to 0.91)
(0.51 (0.83 (0.70 (0.29 (0.76 (0.88 (0.65 (0.80 (0.66 (0.43 (0.45 (0.66 (0.63 (0.72 (0.79 (0.71 (0.21 (0.51 (0.70 (0.67 (0.65 (0.78 (0.61 (0.86 (0.89 (0.88
to to to to to to to to to to to to to to to to to to to to to to to to to to
0.93) 0.97) 0.96) 0.95) 0.97) 0.98) 0.96) 0.97) 0.95) 0.93) 0.93) 0.95) 0.94) 0.96) 0.97) 0.96) 0.90) 0.92) 0.95) 0.95) 0.94) 0.96) 0.94) 0.98) 0.98) 0.98)
* Revised score is the total score with items with ICCs <.50 deleted (6 items). y These are the items with ICC<.05 that were deleted from the final measure.
Examination of internal consistency The item to test, item to total, and scale alphas of each of the certified hand therapist’s ratings for the AM-ULA at the first test administration were examined. Minimal detectable change Minimal detectable change (MDC) is a statistical measure of change, defined as the minimum amount of change that exceeds the measurement error.19 Coefficients from the ICC were used to calculate the MDC at 90% (MDC90) and 95% confidence interval (MDC95). Known group validity ANOVAs were used to compare scores at the first testing session (visit 1) by level of amputation, expecting that subjects with more proximal levels of amputation would have lower (worse) scores than those with more distal (ie, transradial) amputations. ANOVAs were used to compare mean scores (mean of raters 1 and 2) for each AM-ULA item by amputation level. Convergent validity Pearson product-moment correlations were used to examine relations between mean rater scores of the AM-ULA and scores of 2 dexterity tests, the Modified Box and Block Test20 and the Jebsen-Taylor Hand Function Test,21 as well as a modified version www.archives-pmr.org
of the UEFS.8,22 The Jebsen-Taylor Hand Function subtests are traditionally scored by recording the number of seconds required to complete each task, and time is not capped. Subjects in our study were given 2 minutes to complete each subtest and number of items completed per second was calculated. The UEFS items ask amputees to evaluate ease of performing everyday activities. Higher scores indicate worse functioning. In our study, 22 UEFS items were used, and 1 item related to washing was omitted because of the expectation that users of myoelectric devices would not be able to get them wet and would not use their prostheses for washing. Because this item was omitted, item response theory methods in Winsteps23,b were used to recalibrate the measure and calculate person level summary scores.
Results Reliability Forty-nine subjects completed 2 visits within 1 week. Table 1 shows subject characteristics. Results of the reliability analysis are shown in table 2. ICCs for test retest reliability were .19 to .85 for rater 1 and .29 to .95 for rater 2. Six items had ICCs <.50. ICCs for interrater reliability of raters 1 and 2 ranged from .47 to .86. Two items had ICCs <.50. ICCs for interrater reliability of the
492
L. Resnik et al
site therapist and raters 1 and 2 (for 16 subjects graded by all 3 therapists) ranged from .69 to .95.
Refining the AM-ULA measure Six of 24 items had ICCs <0.5 for either test-retest reliability or interrater reliability. These items were eliminated. Summary scores were calculated by averaging the scores for the 18 remaining items and multiplying by 10 to improve interpretability. Thus, potential scores ranged from 0 to 40. ICCs for test-retest reliability were .88 to .91. ICCs for interrater reliability were .84 to .89. MDC90 of the refined measure was 3.7 points, and the MDC95 was 4.4.
Internal consistency All items in the revised item set had acceptable item-to-rest, and item-to-test correlations (table 3) and were retained in the final AM-ULA scale. The alphas of the full scale ranged between .89 and .91.
Table 4 ANOVAs comparing AM-ULA summary scores by amputation level Item
Transradial (nZ23)
Transhumeral (nZ23)
Shoulder (nZ6)
P
Site therapist Rater 1 Rater 2
226 194 205
166 125 155
144 94 104
<.01 <.001 <.001
NOTE. Values are mean SD or as otherwise indicated.
Convergent validity Correlations between the AM-ULA and most dexterity measures were moderate and statistically significant, ranging from .42 to .69 (table 6). The correlation between the AM-ULA and the JebsenTaylor writing was weak. The AM-ULA was moderately negatively correlated with the UEFS score (.44) and was not correlated with a number of activities that the prosthesis was used for in the UEFS.
Known group validity
Discussion ANOVAs showed that subjects with transradial amputation scored the best, and subjects with transhumeral amputation had lower (ie, worse) scores. Scores were even lower for subjects with amputation at the shoulder level (table 4). An item-by-item analysis of mean scores (mean of rater 1 and rater 2 scores) by amputation level shows that scores followed a similar pattern for most items (table 5).
Table 3 by rater
Item-to-total, item-to-rest, and alphas of AM-ULA items Rater 1
Item Brush hair Put on t-shirt Remove t-shirt Button shirt Zip jacket Socks Tie shoes Use cup Use fork Use spoon Pour soda Write word Use scissors Door knob Use phone Hammer Fold towel Reach overhead Total score
We developed and refined a new performance-based activity measure for adults with upper limb amputation and demonstrated that the measure has acceptable interrater reliability, test-retest reliability, internal consistency, known group validity, and convergent validity. Additionally, we presented data on the minimum amount of change in scores needed to be confident that a difference in scores is more than the result of measurement error. One advantage of the AM-ULA is that it can be used to assess activity performance for users of all
Rater 2 a
Itemtest
Itemrest
a
Table 5 ANOVAs comparing mean scores of raters 1 and 2 on AM-ULA items by amputation level
0.89 0.89
0.69 0.73
0.61 0.67
0.90 0.90
Items
Transradial (nZ23)
Transhumeral (nZ23)
Shoulder (nZ6)
P
0.69
0.89
0.60
0.54
0.90
0.51 0.56 0.61 0.72 0.49 0.50 0.49 0.55 0.51 0.37 0.33 0.40 0.64 0.54 0.61
0.89 0.89 0.89 0.88 0.89 0.89 0.89 0.89 0.89 0.90 0.89 0.89 0.88 0.89 0.89
0.50 0.68 0.50 0.75 0.68 0.69 0.72 0.67 0.65 0.48 0.39 0.70 0.67 0.63 0.62
0.42 0.60 0.41 0.70 0.59 0.61 0.66 0.59 0.59 0.38 0.31 0.64 0.60 0.56 0.51
0.91 0.90 0.91 0.90 0.90 0.90 0.90 0.90 0.90 0.91 0.91 0.90 0.90 0.90 0.91
Brush hair Put on T-shirt Remove T-shirt Button shirt Zip jacket Socks Tie shoes Use cup Use fork Use spoon Pour soda Write word Use scissors Door knob Use phone Hammer Fold towel
1.90.9 1.80.9 1.91.0 1.00.6 1.70.8 2.20.8 2.20.8 2.01.2 1.70.9 1.60.8 2.10.9 2.10.6 1.91.1 1.71.0 2.60.6 2.00.7 2.40.6
0.80.8 1.01.0 1.31.2 1.41.0 1.41.1 1.30.8 1.31.0 0.70.8 0.80.9 0.50.8 1.60.9 1.60.7 1.81.0 1.10.9 2.20.7 1.40.9 2.00.7
0.40.5 1.30.6 1.50.7 1.01.0 0.90.5 1.50.4 1.30.8 0.10.2 0.60.5 0.30.4 1.21.1 1.20.8 1.60.9 1.21.0 1.71.0 0.81.0 1.30.9
.000 .069 .414 .368 .203 .006 .004 .000 .004 .000 .073 .109 .785 .426 .012 .010 .003
Itemtest
Itemrest
0.57 0.73
0.47 0.69
0.75 0.58 0.63 0.67 0.77 0.61 0.59 0.59 0.64 0.56 0.47 0.44 0.48 0.71 0.61 0.69
0.89
0.91
NOTE. Values are mean SD or as otherwise indicated.
www.archives-pmr.org
Activities measure for upper limb amputees
493
Table 6 Pearson correlation between the AM-ULA and concurrent measures (NZ49) Measure
Correlation
Box and Block 0.63 Jebsen-Taylor Hand Function Test (items/s) Writing 0.27 Page turning 0.52 Small items 0.55 Feeding 0.61 Checkers 0.42 Light cans 0.69 Heavy cans 0.60 UEFS summary score 0.44 UEFS number of items 0.09
P <.0001 .061 .001 <.0001 <.0001 .002 <.0001 <.0001 .002 .540
types of upper limb prostheses: body powered, hybrid, or myoelectric. The measurement development process involved experienced occupational and physical therapists. An iterative process helped refine content, scoring elements, and criteria, ensuring that the AM-ULA had good content validity. We found good concordance between 2 independent rater’s ratings, and independent raters and the site therapist ratings. This is somewhat remarkable in that the hand therapist raters could watch the videotaped test performance repeatedly, while the site therapists scored tests immediately after administration. These findings suggest that the AM-ULA can be implemented and scored reliably by clinicians in practice. That said, the site therapists (L.A., J.D., R.D., C.E., L.S.W.) were involved in development of the AM-ULA test, and thus were very familiar with scoring. Therefore, like any other performance-based measure of function, we expect that training will be necessary to insure that clinicians understand subtask definitions and scoring criteria. Further study is needed to examine interrater reliability with a different group of raters not involved in measurement development. There are limitations to interpretation of the MDC. A change greater than the MDC does not indicate whether the changes in scores are clinically important. Future studies are needed to determine the magnitude of score change associated with clinically relevant improvements and deterioration. The analyses demonstrated moderate convergent validity between the AM-ULA and related measures and moderate correlation between all measures of dexterity and the AM-ULA, with the exception of the Jebsen-Taylor writing test. A finding of moderate correlation between dexterity and performance of daily activities is consistent with the findings from other studies examining relations between measures of dexterity and measures of functional performance.24 The Jebsen writing test is a timed test that requires the administrator to count the number of legible letters written in the allotted time. Thus, this measure assesses penmanship as well as dexterity, which may explain the weaker association. A moderate negative correlation between the AM-ULA and the UEFS was observed. In the AM-ULA higher scores indicate better functioning, while in the UEFS higher scores indicate worse functioning. This finding is consistent with previous research reporting only moderate correlations between www.archives-pmr.org
self-reported and performance measures. Further, the AM-ULA and UEFS assess different, but related, constructs. The AM-ULA is scored by the clinician, while the amputee is using or attempting to use the prosthesis. Difficulty of performing items on the UEFS is rated by amputees regardless of whether or not they use their prosthesis. Thus, UEFS total scores do not take use of the prosthesis into consideration. However, amputees are asked to indicate whether they usually use the prosthesis to perform the task. In our study, the number of activities for which the prosthesis is usually used was not correlated to observed function using the AM-ULA.
Study limitations The UEFS scores that we obtained were based on a 22-item version of the measure, not the original 23-item measure developed by Heinemann et al.8 Because 1 UEFS item was omitted during our administration (washing face), we cannot say with certainty that identical results would have been obtained if all the original UEFS items had been used. The impact of amputation of the dominant compared with the nondominant upper limb on AM-ULA scores was not examined. Data were collected on hand dominance prior to amputation and after amputation. We found that 100% of the sample (not including 1 congenital bilateral amputee) reported that they had shifted their hand dominance after amputation of the dominant extremity. It is possible that responsiveness of the AM-ULA would vary for new amputees undergoing prosthetic rehabilitation depending on whether or not the amputation was of the dominant or the nondominant side. However, our study was not designed to evaluate this. Future studies are needed to examine the influence of preamputation hand dominance on change in AM-ULA scores over time. A limitation of our approach is that the item set for the AM-ULA was drawn from items that were common to the UEFS and OPUS measure, and no additional content validation was performed. Thus, there was no expert consensus as to the appropriateness or comprehensiveness of the items that were selected for the new measure.
Conclusions The AM-ULA is a new 18-item measure of activity performance for adults with upper limb amputation that considers task completion, speed, movement quality, skillfulness of prosthetic use, and independence in its rating system. It has excellent internal consistency, good interrater reliability, test-retest reliability, and demonstrated known group and convergent validity. The AM-ULA is appropriate for users of all types of prosthetic devices and addresses the recognized need for a validated performancebased measure of activity for adults. It is scored from 0 to 40, with higher scores indicating better functional performance. The 3.7-point MDC90 can be used to assist clinicians in interpreting change in test scores with repeated administration.
Suppliers a. DEKA Integrated Solutions, Inc, 340 Commercial St, #401, Manchester, NH 03101-1121. b. Linacre JM. (2012) Winsteps Rasch measurement computer program. Beaverton, OR. Winsteps.com.
494
Keywords Amputation; Disability evaluation; Psychometrics; Rehabilitation; Upper extremity
Corresponding author Linda Resnik, PT, PhD, Research Health Scientist, Providence VA Medical Center, 830 Chalkstone Ave, Providence, RI 02908. E-mail address: [email protected].
Supplementary data Supplementary data related to this article can be found online at http://dx.doi.org/10.1016/j.apmr.2012.10.004
References 1. Wright V. Measurement of functional outcome with individuals who use upper extremity prosthetic devices: current and future directions. J Prosthet Orthot 2006;18:46-56. 2. Miller LA, Swanson S. Introduction to the Academy’s State of the Science Conference on upper limb prosthetic outcome measures. J Prosthet Orthot 2009;21(Suppl):P1-2. 3. Hill W, Stavdahl O, Hermansson L, et al. Functional outcomes in the WHO-ICF model: establishment of the Upper Limb Prosthetic Outcome Measures Group. J Prosthet Orthot 2009;21:115-9. 4. Hill W, Kyberd P, Hermansson LN, et al. Upper Limb Prosthetic Outcome Measures (UPLOM): a working group and their findings. J Prosthet Orthot 2009;21(4 Suppl):P69-82. 5. Wright V. Prosthetic outcome measures for use with upper limb amputees: a systematic review of the peer-reviewed literature, 1970 to 2009. J Prosthet Orthot 2009;21(4 Suppl):P3-63. 6. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001. 7. Miller L, Swanson S. Summary and recommendations of the Academy’s State of the Science Conference on upper limb prosthetic outcome measures. J Prosthet Orthot 2009;21(4 Suppl):83-9. 8. Heinemann AW, Bode RK, O’Reilly C. Development and measurement properties of the Orthotics and Prosthetics Users’ Survey (OPUS): a comprehensive set of clinical outcome instruments. Prosthet Orthot Int 2003;27:191-206.
L. Resnik et al 9. Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]. The Upper Extremity Collaborative Group (UECG). Am J Ind Med 1996;29:602-8. 10. Lindner HY, Natterlund BS, Hermansson LM. Upper limb prosthetic outcome measures: review and content comparison based on International Classification of Functioning, Disability and Health. Prosthet Orthot Int 2010;34:109-28. 11. Hermansson LM, Bodin L, Eliasson AC. Intra- and inter-rater reliability of the assessment of capacity for myoelectric control. J Rehabil Med 2006;38:118-23. 12. Hermansson LM, Fisher AG, Bernspa˚ng B, Eliasson AC. Assessment of capacity for myoelectric control: a new Rasch-built measure of prosthetic hand control. J Rehabil Med 2005;37:166-71. 13. Gallagher P, Franchignoni F, Giordano A, MacLachlan M. Trinity amputation and prosthesis experience scales: a psychometric assessment using classical test theory and Rasch analysis. Am J Phys Med Rehabil 2010;89:487-96. 14. Resnik L. Research update: VA study to optimize DEKA arm. J Rehabil Res Dev 2010;47:ix-x. 15. Atkins D, Meier R. Comprehensive management of the upper-limb amputee. New York: Springer-Verlag; 1989. 16. Smurr LM, Gulick K, Yancosek K, Ganz O. Managing the upper extremity amputee: a protocol for success. J Hand Ther 2008;21:16075, quiz 176. 17. Portney L, Watkins M. Foundations of clinical research applications to practice. Stamford: Appleton & Lange; 1993. 18. Streiner D, Norman G. Health measurement scales: a practical guide to their development and use. New York: Oxford Univ Pr; 2003. 19. Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intraindividual changes in health-related quality of life. J Clin Epidemiol 1999;52:861-73. 20. Mathiowetz V, Volland G, Kashman N, Weber K. Adult norms for the Box and Block Test of manual dexterity. Am J Occup Ther 1985;39:386-91. 21. Jebsen RH, Taylor N, Trieschmann RB, Trotter MJ, Howard LA. An objective and standardized test of hand function. Arch Phys Med Rehabil 1969;50:311-9. 22. Burger H, Franchignoni F, Heinemann AW, Kotnik S, Giordano A. Validation of the orthotics and prosthetics user survey upper extremity functional status module in people with unilateral upper limb amputation. J Rehabil Med 2008;40:393-9. 23. Linacre JM. A user’s guide to WINSTEPS, MINISTEP Rasch-model computer programs. Chicago: 2006. 24. Lin KC, Chuang LL, Wu CY, Hsieh YW, Chang WY. Responsiveness and validity of three dexterous function measures in stroke rehabilitation. J Rehabil Res Dev 2010;47:563-71.
www.archives-pmr.org
Activities measure for upper limb amputees
494.e1
Supplemental Appendix S1 Definition of Tasks 1. )Put toothpaste on a toothbrush a) Grasp a tube of toothpaste b) Remove the cap from the toothpaste c) Grasp/position a toothbrush d) Squeeze toothpaste onto toothbrush e) Replace the cap f) Release the tube of toothpaste 2. Brush/comb hair (unilateral) a) Grasp comb b) Bring comb to head c) Comb hair or perform a combing motion d) Release the comb from grasp 3. Put on t-shirt a) Grasp the t-shirt b) Thread head through neck opening c) Thread arms through sleeves d) Pull shirt down to fit properly 4. Remove t-shirt a) Grasp the t-shirt b) Lift the shirt over head c) Undress arms d) Place shirt on table e) Release grasp 5. Button shirt with front buttons a) Grasp the shirt b) Push button through hole c) Pull button out other side d) Complete buttoning of 3 buttons e) Release grasp on shirt 6. Attach end of zipper and zip jacket a) Grasp zipper and vest b) Initiate the zipper c) Pull zipper up at least two-thirds of the way d) Unzip the zipper 7. Put on socks a) Grasp the sock b) Pull the sock over toes c) Pull the sock over the heel so that it is fully donned 8. Tie shoelaces a) Grasp a lace with each hand b) Criss-cross the laces c) Make loops d) Pull the bow tight e) Release grasp on the laces 9. Drink from a paper cup (unilateral) a) Pick up a paper cup from a table b) Bring the cup to touch the mouth c) Tilt the cup and simulate drinking (or drink) d) Return the cup to the table e) Release the grip on the cup 10. Use a fork (unilateral) a) Grasp fork b) Bring fork all the way to the mouth as if taking a bite of food c) Move fork away from mouth and return to table d) Release grip on the fork 11. Use a spoon (unilateral) a) Grasp spoon
www.archives-pmr.org
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
b) Bring spoon all the way to the mouth as if taking a bite of food c) Move spoon away from mouth and return to table d) Release grip on the spoon ) Cut meat with knife and fork a) Grasp the utensils b) Position the utensils c) Spear dough/“meat” with fork d) Make 3 cuts with knife e) Release the utensils f) Set utensils down on table or plate Pour from a 12oz can a) Pick up soda can b) Grasp cup with opposite hand to stabilize c) Pour soda into cup d) Set can down on table e) Release cup and can Write the word “LETTER” legibly (unilateral) a) Grasp the pen or pencil b) Write the word “LETTER” c) Set the pen down d) Release grip Use a pair of scissors a) Grasp the scissors b) Grasp paper in the other hand c) Cut the paper with scissors (make at least 3 cuts) d) Release paper e) Release scissors Turn a round door knob (unilateral) a) Reach for the door knob b) Grasp the door knob c) Turn the door knob to release the latch d) Release the knob ) Use a key in lock (unilateral) a) Grasp the key b) Insert key into keyhole c) Turn key to lock or unlock d) Remove key from keyhole e) Release the key ) Carry a laundry basket a) Grasp opposite sides of laundry basket b) Pick up the basket c) Ambulate with the basket held securely d) Set basket on a counter height surface e) Release the hands from basket Dial a touch tone phone a) Grasp cell phone in 1 hand b) Position phone to see touch pad c) Use appropriate part of terminal device or sound hand to press buttons to dial a phone number Use a hammer and nail a) Pick up the hammer with 1 hand and the nail with the other hand b) Position the nail vertically on top of wood c) Lift hammer and bring it down to drive the nail into the wood d) Remove hand that was positioning the nail away and continue to hammer nail into wood e) Release hammer from grip Fold a bath towel a) Grasp ends of towel b) Brings ends together to fold twice
494.e2 c) Release grip on the towel 22. )Open an envelope a) Secure the envelope in 1 hand b) Using a letter opener, pen, or finger break the top seal of envelope c) Open envelope d) Release the envelope 23. )Stir in a bowl a) Grasp a spoon b) Place the spoon in a bowl c) Stabilize the bowl with other hand as needed d) Stir the spoon within the bowl 24. Reach overhead (ie, to the top of refrigerator door) (unilateral) a) Lift the arm overhead b) Bring arm/hand to top of surface c) Grasp object on shelf d) Bring the arm down with object in hand ) Indicates that item was not included in final AM-ULA measure.
Supplemental Appendix S2 AM-ULA Instructions Read the following to the subject: “I want to see how you do some everyday activities. Please use your prosthetic arm to do these activities. Please use your sound arm only on those activities that require 2 hands.” Use the instructions below to guide the subject in performing the testing tasks: 1. Please take this toothbrush and toothpaste, remove the cap from the tube, and squeeze the toothpaste on the toothbrush. Then replace the cap and release the tube. Be sure to use your prosthesis while doing this.) 2. Please take this comb and, while grasping it, run it through your hair (or over the top of their head if bald). Then put the comb down. Try to use only your prosthesis to do this task. 3. Please put this t-shirt on. I want you to grasp the t-shirt, thread your head through the neck opening, thread your arms through the arm holes, and pull the shirt down over your body. Be sure to use your prosthesis while doing this. 4. Now I’d like you to remove the t-shirt. While grasping the tshirt, lift it over your head, remove your arms from the sleeves, and place the shirt on the table. Be sure to use your prosthesis while doing this. 5. Please put on this button-up shirt (therapist may help don shirt) and then button 3 buttons by putting the buttons through the holes and pulling them out the other side. Be sure to use your prosthesis while doing this. 6. Please put this vest on (therapist may help don vest), start the zipper, and pull it at least two-thirds of the way up. Then unzip the zipper. Be sure to use your prosthesis while doing this. 7. Please take this sock and put it on, pull the sock over your heel and pull it up all the way so that it fits well. Then remove the sock from your foot. Be sure to use your prosthesis while doing this. 8. Please take this shoe (placed either on a tabletop or on the floor) and tie the laces. Grasp a lace in each hand, criss-cross the laces, making loops, and tie the bow tight. Be sure to use your prosthesis while doing this.
L. Resnik et al 9. Pick up this paper cup from the table, bring it all the way to your mouth, and pretend to drink from it. Then put the cup back on the table. Try to use only your prosthesis to do this task. 10. Please grasp this fork and bring it to your mouth as if you were going to take a bite of food from it. Then put the fork back on the table. Try to use only your prosthesis to do this task. 11. Please grasp this spoon with your prosthesis and bring it to your mouth as if you were going to take a bite of food from it. Then put the spoon back on the table. Be sure to use your prosthesis while doing this. 12. On the table in front of you is a plate, knife, fork, and a piece of play dough in the shape of a hot dog. Please spear the “meat” with the fork, cut the meat 3 times with the knife. When you finish, place the utensils back on the table or the plate. Be sure to use your prosthesis while doing this.) 13. Please pick up the soda can from the table with 1 hand and grasp the cup with your other hand. Then pour the soda from the can into the cup. When finished set the can and cup (if necessary) down on the table. Be sure to use your prosthesis while doing this.) 14. Please grasp this pen with your prosthesis and write the word “LETTER” on the blank sheet of paper on the clipboard. Then set the pen down on the table. Try to use only your prosthesis to do this task. 15. Please take this blank piece of paper and, grasping scissors in 1 hand and with the paper in the other, make at least 3 cuts in the paper. Then place the cut paper and scissors on the table. 16. Please reach for this door knob (use a round door knob), grasp the knob, and turn it until the latch is released. Then release the knob. Try to use only your prosthesis to do this task. 17. Please take this key and insert it into the keyhole. Turn the key to lock (or unlock) the door, then remove the key from the keyhole and give it back to me. Try to use only your prosthesis to do this task.) 18. Please grasp both sides of this laundry basket, pick it up, and carry it over here (please show subject where). Set the basket down and release your hands from it.) 19. Please take this cell phone and dial a phone number with it. Be sure to use your prosthesis while doing this. 20. Please pick up the hammer in 1 hand and the nail in the other. Position the nail vertically on the piece of wood and use the hammer to drive the nail into it. Remove your hand that was holding the nail and continue hammering the nail into the wood. Then put the hammer down. 21. Please grasp the ends of this bath towel, bringing the ends of the towel together to fold it twice. Then place the folded towel back on the table. Be sure to use your prosthesis while doing this. 22. Please secure this envelope in 1 hand and, using a letter opener, pen, or finger, break the seal on it. Then open the envelope and place it on the table. Be sure to use your prosthesis while doing this.) 23. Please grasp this spoon with your prosthetic hand and, using your other hand to stabilize the bowl, use the spoon to stir the cereal in the bowl.) 24. Using your prosthetic side, reach up to this overhead shelf and grasp the object (a lightweight cup) on the shelf, now lower your arm and hand the object to me. Try to use only your prosthesis to do this task. ) Indicates item was deleted from the final AM-ULA measure. www.archives-pmr.org
Activities measure for upper limb amputees
Scoring Instructions Overview The purpose of this measure is to provide a clinician rated measure of an upper limb amputee’s performance of daily functional activities using the prosthesis. The scale is graded on a scale of 0-4 (unable to excellent). Rules of Administration Tester should familiarize themselves with all tasks and necessary sub-components (see attached Task List). Tester asks the Amputee to perform all of the unilateral tasks using the prosthesis and to use the prosthesis as much as possible when performing tasks that require bilateral engagement. Tester is not to provide assistance, except for safety. The amputee may use an aide, such as a button hook or other adaptive equipment. Scoring Rules Amputees are scored on 5 elements: Extent of completion. Grade the extent of completion of all sub-tasks of the activity.
www.archives-pmr.org
494.e3 Speed of completion of entire activity. Grade the speed of task performance as compared to performance with a sound limb. Movement quality: Grade the amount of awkwardness or compensatory movements resulting in/from lack of prepositioning, limitations of the device, lack of skilled use or any other reason. Skillfulness of prosthetic use. Grade the type of use (no active use, use as a stabilizer, assist, or prime mover), control over voluntary grip functions. Independence. Grade the use of assistive device or adaptive equipment. The grid below shows the scoring criteria for each element. The lowest score in any category is the one that is awarded for the overall score.
Note Use of “excellent” category compares performance of the activity to performance with a sound limb. Test administrators should be aware that there will be few amputees who will be graded an “excellent.” This is a purposefully high bar for comparison; however, it is included to avoid a ceiling effectdwhereby there is no room for improvement, with advances in technology and methods of controls that will be evident in the near future.
494.e4
Grade
Completion of Sub-tasks
Speed of Completion (as Compared to Non-disabled)
0 Unable 1 Poor
Less than all All
2 Fair
Skillfulness of Prosthesis use
Independence
N/A Very slow to slow
N/A Very awkward, many compensatory movements
N/A May or may not use assistive device
All
Slow to medium
Some awkwardness or compensatory movement
3 Good
All
Medium-fast to normal
Minimal to no awkwardness or compensatory movement
4 Excellent
All
Equivalent to nondisabled
Excellent movement quality, no awkwardness or compensatory movement
No prosthetic use Inappropriate choice of grip for task (if choice is available) Loses grip multiple times during task, lack of proportional control (if available) Multiple unintentional activation of a control Prosthesis only used as a stabilizer during bimanual activities, ie, there is no active use or use of grip, but arm or terminal device used to stabilize during the task Sub-optimal choice of grip for task (if choice is available) Use of prosthesis as an assist for bimanual activities or prime mover unilateral activities Loses grip once during task More than 1 attempt needed to pre-position object within grasp and/or more than minimal awkwardness in pre-positioning object One incidence of unintentional activation of a control Fair proportional control Skilled use of prosthesis as an assist for bimanual activities or as a prime mover for unilateral activities Quick and easy pre-positioning of object within grasp No unintentional loss of grip Optimal choice of grip for task (if choice is available) No inadvertent loss of grip or unwanted movement Optimal choice of grip for task (if choice is available) Sound side NOT used to pre-position object within grasp
May or may not use assistive device
May or may not make use of assistive device
No assistive device use
L. Resnik et al
www.archives-pmr.org
Movement Quality
ORIGINAL ARTICLE
Development and Evaluation of the Activities Measure for Upper Limb Amputees Linda Resnik, PT, PhD,a,b Laurel Adams, MOT, OTR/L,c Matthew Borgia, MS,a Jemy Delikat, MOT, OTR/L,c Roxanne Disla, OTR,d Christopher Ebner, MS, OTR,e,f Lisa Smurr Walters, MS, OTR/L, CHTe From the aProvidence Veterans Administration Medical Center, Providence, RI; bHealth Services, Policy and Practice Brown University, Providence, RI; cHealth Services Research and Development/Rehabilitation Research and Development Center of Excellence, James A. Haley Veterans Hospital, Tampa, FL; dNew York Harbor Healthcare SystemeManhattan Veterans Administration, New York, NY; eCenter for the Intrepid, Department of Orthopaedics and Rehabilitation, Brooke Army Medical Center, Fort Sam Houston, TX; fcurrently with Warrior Transition Battalion-Europe, Nachrichten Kaserne, Heidelberg Rohrbach, Germany.
Abstract Objectives: (1) To develop a measure of activities for adults with upper limb amputation: the Activities Measure for Upper Limb Amputees (AM-ULA); and (2) to conduct initial psychometric evaluation of the measure. Design: This was a cohort study where the prototype measure was administered twice within 1 week. Tests were videotaped and graded by 2 independent raters. Interrater reliability, test-retest reliability, internal consistency, and minimal detectable change were estimated. Known group validity was examined using analyses of variance comparing scores of transradial, transhumeral, and shoulder level amputees. Convergent validity was examined by correlating AM-ULA scores with dexterity tests and self-reported function. Setting: Hospital outpatient. Participants: Subjects (NZ52) with upper limb amputation. Interventions: Not applicable. Main Outcome Measures: Not applicable. Results: Intraclass correlation coefficients (ICCs) for test-retest reliability were .88 to .91. ICCs for interrater reliability were .84 to .89. Cronbach alphas were .89 to .91. The minimal detectable change at the 90% confidence interval was 3.7 points. Subjects with more distal levels of limb loss had better scores than those with more proximal levels (P<.01). The AM-ULA was moderately correlated with most dexterity tests and self-reported function. Conclusions: The AM-ULA is a new measure of activity performance for adults with upper limb amputation that considers task completion, speed, movement quality, skillfulness of prosthetic use, and independence in its rating system. It has good interrater reliability, test-retest reliability, and demonstrated known group validity. Archives of Physical Medicine and Rehabilitation 2013;94:488-94 ª 2013 by the American Congress of Rehabilitation Medicine
Outcome measures are needed to assess rehabilitation progress and document effectiveness of prosthetic devices.1-4 Yet, it is challenging to systematically collect or analyze outcomes data for persons with upper limb amputation, because there are few measures developed for or validated with adults, and limited research to guide
Supported by the Office of Research and Development Rehabilitation Research and Development Service, Department of Veterans Affairs (grant no. A6780I). No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.
measure selection and interpretation. Although some instruments specific to adults with upper limb amputation do exist, the majority were developed for and used with pediatric amputees.1,5 Efforts to develop consensus on outcome measures for upper limb amputees coalesced in 2005 and continued through an Upper Limb Prosthetic Outcomes Measures (ULPOM) Working Group. The ULPOM Working Group aimed to develop a toolkit of validated measures addressing each major domain of the International Classification of Functioning, Disability and Health (ICF)6 (ie, body structures/functions, activities, participation), and identify the need for new measures.4 This group considered Wright’s comprehensive
0003-9993/13/$36 - see front matter ª 2013 by the American Congress of Rehabilitation Medicine http://dx.doi.org/10.1016/j.apmr.2012.10.004
Activities measure for upper limb amputees review5 of existing measures, recommended several measures, and concluded that there was a need for new tests of function for adults with amputation, as well as new measures for use with higher-level amputees, bilateral amputees, and body-powered users.7 Wright’s review5 reported that there were only 7 measures developed for adults. Furthermore, most measures were not validated with amputees, and 5 did not assess activities. The Orthotics and Prosthetics Users’ Survey (OPUS) Upper Extremity Functional Status (UEFS) module8 was the only prosthetic-specific measure of functional activities, and the Disabilities of Arm, Shoulder and Hand Questionnaire (DASH)9 was the only generic measure of upper limb functional activities. In 2010, Lindner et al10 summarized psychometric properties of 8 ULPOM and classified their content by linking items to ICF taxonomy. Lindner10 identified 3 measures appropriate for adults: Assessment of Capacity for Myoelectric Control (ACMC),11,12 UEFS,8 and Trinity Amputation and Prosthesis Experience Scales (TAPES).13 Both reviews drew similar conclusions about available instruments and recommended 3 measures: ACMC, OPUS,8 and TAPES. Additionally, Wright5 suggested 1 generic measure not on Lindner’s list10: the DASH.9 Together, these reviews highlighted the lack of objective, performance-based measures for adults. Performance-based measures are scored by an outside observer and may consist of timed-based tests or measures of performance using a rating scale. Self-reported measures are generally questionnaires completed by the persons themselves, although there may be special forms developed for proxy use. Performance-based measures are often considered more objective, while self-reported measures are considered more subjective. Both types are important in assessing outcomes. The only performance-based activity measure recommended by the ULPOM Working Group and the 2 literature reviews was the ACMC, a measure appropriate only for users of myoelectric prostheses. All other recommended measures were self-reported. The development of a new performance-based measure of activities for upper limb amputees began as part of the Department of Veterans Affairs (VA) Study to Optimize the DEKA Arm.14,a Investigators did not choose the ACMC, because the DEKA Arm may be controlled using foot controls, electromyography, and body-powered elements. This article reports on 2 study purposes: (1) to develop a new outcome measure of activities for adults with upper limb amputation, the Activities Measure for Upper Limb Amputees (AM-ULA) and (2) to conduct initial psychometric evaluation of the measure.
List of abbreviations: ACMC AM-ULA ANOVA DASH ICC ICF MDC MDC90 MDC95 OPUS TAPES UEFS ULPOM VA
Assessment of Capacity for Myoelectric Control Activities Measure for Upper Limb Amputees analysis of variance Disabilities of Arm, Shoulder and Hand Questionnaire intraclass correlation coefficient International Classification of Functioning, Disability and Health minimal detectable change minimal detectable change at the 90% confidence interval minimal detectable change at the 95% confidence interval Orthotics and Prosthetics Users’ Survey Trinity Amputations and Prosthetics Experience Scale Upper Extremity Functional Status Upper Limb Prosthetic Outcomes Measures Department of Veterans Affairs
www.archives-pmr.org
489
Methods Initial prototype development A set of activities from the widely used Atkins activities of daily living checklist was selected for the initial prototype.15,16 To further face validity, activities were chosen that were also included in the UEFS from the OPUS to represent a range of everyday functional skills.8 A simple grading scale, similar to that employed by Atkins and Meier15 was employed. Basic instructions for tasks were constructed. Therapists read the following instructions prior to each activity: “I want to see how you do some everyday activities. Please use your prosthetic arm to do these activities. Please use your sound arm only on those activities that require two hands.”
Evaluation of the prototype measure The first prototype activity measure was used in the testing of early subjects in the VA Study to Optimize the DEKA Arm.14 Therapists were oriented to the measure and provided written grading instructions. All testing was videotaped. During the first few months, site therapists and the first author (L.R.) independently graded subjects’ performance, and results were compared. This led us to conclude that more specific instructions for tasks and grading were needed to enable consistency in administration and scoring.
Revision of the prototype measure A series of conference calls with site occupational therapists helped identify and refine subtasks within activity items and scoring criteria (see supplemental appendix S1, available online only at the Archives website: www.archives-pmr.org). A standardized set of instructions was developed for each item (see supplemental appendix S2, available online only at the Archives website: www. archives-pmr.org). Initial discussions were used to gain insight into implicit elements used when assigning grades. Therapists watched a videotaped testing session and graded subject task performance. The group then watched the video together and discussed each item to understand the rationale for assigned grades. After each session, the first author (L.R.) circulated the list of elements therapists identified when assigning grades and asked for confirmation. All agreed that excellent should be used only when performance was comparable with that of a sound limb. We expected that use of a high bar would avoid a ceiling effectd whereby there would be no room for measurable improvement with advances in technology. Scoring criteria were revised and refined in 2 subsequent conference calls after viewing, grading, and reviewing new videos. After review and discussion of the final video, the first author (L.R.) concluded that the instructions were acceptable, because no additional scoring considerations were suggested. Final scoring criteria considered the: (1) extent of completion of all activity subtasks; (2) speed of completion; (3) movement quality; (4) skillfulness of prosthetic use and control over voluntary grip functions; and (5) independence. A synopsis of scoring criteria is subsequently presented. Scoring criteria Application of scoring criteria is summarized in supplemental appendix S2. When implementing scoring, the lowest score in any category is the one that is awarded for the overall score.
490 Completion of all subtask elements All subparts of a task should be completed to assign a grade higher than 0 (unable to perform). Thus, if the amputee could perform every aspect of a task except one, for example taking the cap off the toothpaste tube, he would be graded a 0 for the task of brushing teeth. Speed of completion (as compared with nondisabled) Speed of task performance is a relevant element. Amputees who perform tasks more slowly would be scored lower than those who perform tasks more quickly. Movement quality Movement quality, including awkward-looking movements caused by either inherent limitations of the prosthesis, poor planning, lack of prepositioning, or by compensatory movement strategies is important to consider when grading prosthetic use. The bar for the highest grade of movement quality is natural-looking movement similar to that of a nondisabled upper limb. Skillfulness of prosthetic use Subjects should be scored higher if they perform tasks using the prosthesis as a prime mover or an assist, rather than a passive stabilizer. Skillful control of the terminal device is a critical aspect of skilled use, and instances of involuntary loss of grip should be considered in the scoring system. Independence The last grading criterion is whether or not the amputee needed an assistive device or piece of adaptive equipment to complete the activity. Equipment can and should be used in testing, but prosthetic users cannot achieve the highest grade of excellent if they use equipment to complete a task.
Psychometric evaluation of the AM-ULA Therapists for the VA Study to Optimize the DEKA Arm administered the revised prototype for the AM-ULA to 49 subjects using their current prostheses on 2 occasions within 1 week. The study was approved by the Institutional Review Boards of the Providence VA Medical Center, the NY VA Health Harbor System, the James Haley VA Medical Center, the Long Beach VA Medical Center, and the Center for the Intrepid, Brooke Army Medical Center. Tests were videotaped and independently rated by 2 independent raters. Six site occupational therapists graded the performance of 16 subjects. All site therapists had at least 3 years of clinical experience; several had over 10 years experience with a range of experience in prosthetic rehabilitation from less than 1 year to more than 10 years. All had been trained in administration of standardized tests and measures for upper limb amputees as part of the research study protocol. Independent rater training The independent raters were certified hand therapists: 1 was an occupational therapist, the other was a physical therapist. Both had over 15 years of clinical experience and were experienced with prosthetic rehabilitation. Independent rater training took place over three 2-hour sessions. At the first session, the raters were introduced to the scoring system, viewed a videotaped test subject, and discussed task scoring. Scoring criteria were reviewed a second time after which videotapes of 2 additional subjects were viewed and discussed. At the second session, raters independently rated testing videotapes of 2 pilot subjects. Discrepancies in scores
L. Resnik et al were discussed. Prior to the third session, raters watched test videotapes of 2 additional subjects and independently rated performance. Scores were reviewed and discussed. At the end of the third training session, it was apparent that raters had good familiarity with each test and scoring criteria.
Statistical analyses Examination of test-retest reliability Test-retest reliability comparing scores for visits 1 and 2 were calculated using repeated-measures analysis of variance (ANOVA) and Shrout and Fleiss ICC (type3,1), a 2-way mixed model, single measure of reliability. Test-retest reliability of the scores of each rater was evaluated separately. Examination of interrater reliability Interrater reliability of the certified hand therapist raters at visit 1 were calculated using an intraclass correlation coefficient (ICC2,1), a 2-way random-effects single measure of reliability where the target and the raters are considered random effects. ICC2,1 allows for the generalizability of the findings to all possible raters. Consistency of ratings between the 2 certified hand therapist raters and the 6 onsite therapists was examined using ICC2,2, a 2-way mixed model of average measure reliability. For this comparison of data on only 16 subjects, mean rating scores from both visits 1 and 2 were used to decrease error variance and increase reliability estimates, because means are considered better estimates of true scores.17 Selection of items for final measure Only those items that had at least moderate (0.5) test-retest and interrater reliability were included in the final version of the AM-ULA.18 Table 1 and 2
Characteristics of subjects who completed both visits 1
Characteristic
Value
Age, mean SD (range) Sex Male Female Race White Other Veteran Nonveteran Veteran Active duty Prosthetic user (active device only) Not current user Full time Part time Type of prosthesis used for testing Body powered Myoelectric Hybrid Amputation of dominant side
45.816.5 (21.5e82.8) 44 (89.8) 5 (10.3) 44 (89.8) 5 (10.2) 10 (20.4) 27 (55.1) 12 (24.5) 1 (2.0) 31 (63.3) 17 (34.7) 25 20 4 26
(51.0) (40.8) (8.1) (54.2)*
NOTE. Values are n (%) or as otherwise indicated. * One subject was a bilateral congenital amputee and was omitted from this count.
www.archives-pmr.org
Activities measure for upper limb amputees Table 2
491
Test-retest and interrater reliability of AM-ULA items Test-Retest ICC
Interrater
Rater 1
Rater 2
Raters 1 and 2
Raters 1, 2, and Site Therapist
Item
ICC3,1
ICC3,1
ICC2,1
ICC2,2
Brush teethy Brush hair Put on t-shirt Remove t-shirt Button shirt Zip jacket Socks Tie shoes Use cup Use fork Use spoon Cut meaty Pour soda Write word Use scissors Door knob Key in locky Carry laundryy Use phone Hammer Fold towel Open envelopey Stir bowly Reach overhead Total score (24 items) Total score (revised scale 18 items)*
0.25 0.73 0.66 0.76 0.81 0.76 0.79 0.72 0.72 0.53 0.65 0.19 0.54 0.85 0.68 0.67 0.46 0.47 0.65 0.52 0.53 0.47 0.24 0.63 0.91 0.91
0.43 0.68 0.75 0.95 0.84 0.86 0.61 0.86 0.73 0.65 0.72 0.29 0.59 0.74 0.52 0.67 0.69 0.47 0.55 0.64 0.72 0.42 0.34 0.61 0.91 0.88
0.54 0.72 0.62 0.64 0.81 0.86 0.69 0.85 0.77 0.65 0.64 0.85 0.84 0.73 0.77 0.47 0.70 0.47 0.65 0.82 0.60 0.74 0.63 0.81 0.86 0.84
0.79 0.93 0.89 0.79 0.90 0.95 0.87 0.91 0.85 0.78 0.79 0.86 0.84 0.88 0.91 0.89 0.69 0.79 0.87 0.86 0.85 0.91 0.83 0.94 0.95 0.95
(0.07 to 0.53) (0.56 to 0.84) (0.26 to 0.87) (0.29 to 0.94) (0.63 to 0.91) (0.60 to 0.86) (0.60 to 0.90) (0.55 to 0.84) (0.55 to 0.83) (0.27 to 0.72) (0.44 to 0.79) (0.21 to 0.53) (0.28 to 0.72) (0.64 to 0.94) (0.49 to 0.82) (0.29 to 0.87) (0.08 to 0.73) (0.21 to 0.67) (0.45 to 0.79) (0.27 to 0.71) (0.29 to 0.71) (0.21 to 0.67) (0.05 to 0.48) (0.41 to 0.79) (0.85 to 0.95) (0.84 to 0.95)
(0.15 to 0.64) (0.49 to 0.81) (0.45 to 0.90) (0.78 to 0.99) (0.65 to 0.93) (0.75 to 0.92) (0.28 to 0.80) (0.74 to 0.92) (0.56 to 0.85) (0.43 to 0.80) (0.54 to 0.83) (e0.10 to 0.60) (0.36 to 0.76) (0.41 to 0.90) (0.27 to 0.71) (0.26 to 0.87) (0.42 to 0.84) (0.20 to 0.67) (0.30 to 0.73) (0.42 to 0.79) (0.54 to 0.84) (0.14 to 0.63) (0.06 to 0.57) (0.35 to 0.78) (0.85 to 0.95) (0.80 to 0.93)
(0.29 to 0.72) (0.55 to 0.83) (0.21 to 0.84) (0.10 to 0.89) (0.61 to 0.91) (0.77 to 0.92) (0.45 to 0.84) (0.74 to 0.92) (0.61 to 0.87) (0.43 to 0.79) (0.44 to 0.78) (0.71 to 0.93) (0.72 to 0.91) (0.40 to 0.89) (0.62 to 0.87) (0.01 to 0.78) (0.45 to 0.85) (0.21 to 0.66) (0.45 to 0.79) (0.69 to 0.90) (0.37 to 0.76) (0.58 to 0.85) (0.43 to 0.77) (0.68 to 0.90) (0.76 to 0.92) (0.74 to 0.91)
(0.51 (0.83 (0.70 (0.29 (0.76 (0.88 (0.65 (0.80 (0.66 (0.43 (0.45 (0.66 (0.63 (0.72 (0.79 (0.71 (0.21 (0.51 (0.70 (0.67 (0.65 (0.78 (0.61 (0.86 (0.89 (0.88
to to to to to to to to to to to to to to to to to to to to to to to to to to
0.93) 0.97) 0.96) 0.95) 0.97) 0.98) 0.96) 0.97) 0.95) 0.93) 0.93) 0.95) 0.94) 0.96) 0.97) 0.96) 0.90) 0.92) 0.95) 0.95) 0.94) 0.96) 0.94) 0.98) 0.98) 0.98)
* Revised score is the total score with items with ICCs <.50 deleted (6 items). y These are the items with ICC<.05 that were deleted from the final measure.
Examination of internal consistency The item to test, item to total, and scale alphas of each of the certified hand therapist’s ratings for the AM-ULA at the first test administration were examined. Minimal detectable change Minimal detectable change (MDC) is a statistical measure of change, defined as the minimum amount of change that exceeds the measurement error.19 Coefficients from the ICC were used to calculate the MDC at 90% (MDC90) and 95% confidence interval (MDC95). Known group validity ANOVAs were used to compare scores at the first testing session (visit 1) by level of amputation, expecting that subjects with more proximal levels of amputation would have lower (worse) scores than those with more distal (ie, transradial) amputations. ANOVAs were used to compare mean scores (mean of raters 1 and 2) for each AM-ULA item by amputation level. Convergent validity Pearson product-moment correlations were used to examine relations between mean rater scores of the AM-ULA and scores of 2 dexterity tests, the Modified Box and Block Test20 and the Jebsen-Taylor Hand Function Test,21 as well as a modified version www.archives-pmr.org
of the UEFS.8,22 The Jebsen-Taylor Hand Function subtests are traditionally scored by recording the number of seconds required to complete each task, and time is not capped. Subjects in our study were given 2 minutes to complete each subtest and number of items completed per second was calculated. The UEFS items ask amputees to evaluate ease of performing everyday activities. Higher scores indicate worse functioning. In our study, 22 UEFS items were used, and 1 item related to washing was omitted because of the expectation that users of myoelectric devices would not be able to get them wet and would not use their prostheses for washing. Because this item was omitted, item response theory methods in Winsteps23,b were used to recalibrate the measure and calculate person level summary scores.
Results Reliability Forty-nine subjects completed 2 visits within 1 week. Table 1 shows subject characteristics. Results of the reliability analysis are shown in table 2. ICCs for test retest reliability were .19 to .85 for rater 1 and .29 to .95 for rater 2. Six items had ICCs <.50. ICCs for interrater reliability of raters 1 and 2 ranged from .47 to .86. Two items had ICCs <.50. ICCs for interrater reliability of the
492
L. Resnik et al
site therapist and raters 1 and 2 (for 16 subjects graded by all 3 therapists) ranged from .69 to .95.
Refining the AM-ULA measure Six of 24 items had ICCs <0.5 for either test-retest reliability or interrater reliability. These items were eliminated. Summary scores were calculated by averaging the scores for the 18 remaining items and multiplying by 10 to improve interpretability. Thus, potential scores ranged from 0 to 40. ICCs for test-retest reliability were .88 to .91. ICCs for interrater reliability were .84 to .89. MDC90 of the refined measure was 3.7 points, and the MDC95 was 4.4.
Internal consistency All items in the revised item set had acceptable item-to-rest, and item-to-test correlations (table 3) and were retained in the final AM-ULA scale. The alphas of the full scale ranged between .89 and .91.
Table 4 ANOVAs comparing AM-ULA summary scores by amputation level Item
Transradial (nZ23)
Transhumeral (nZ23)
Shoulder (nZ6)
P
Site therapist Rater 1 Rater 2
226 194 205
166 125 155
144 94 104
<.01 <.001 <.001
NOTE. Values are mean SD or as otherwise indicated.
Convergent validity Correlations between the AM-ULA and most dexterity measures were moderate and statistically significant, ranging from .42 to .69 (table 6). The correlation between the AM-ULA and the JebsenTaylor writing was weak. The AM-ULA was moderately negatively correlated with the UEFS score (.44) and was not correlated with a number of activities that the prosthesis was used for in the UEFS.
Known group validity
Discussion ANOVAs showed that subjects with transradial amputation scored the best, and subjects with transhumeral amputation had lower (ie, worse) scores. Scores were even lower for subjects with amputation at the shoulder level (table 4). An item-by-item analysis of mean scores (mean of rater 1 and rater 2 scores) by amputation level shows that scores followed a similar pattern for most items (table 5).
Table 3 by rater
Item-to-total, item-to-rest, and alphas of AM-ULA items Rater 1
Item Brush hair Put on t-shirt Remove t-shirt Button shirt Zip jacket Socks Tie shoes Use cup Use fork Use spoon Pour soda Write word Use scissors Door knob Use phone Hammer Fold towel Reach overhead Total score
We developed and refined a new performance-based activity measure for adults with upper limb amputation and demonstrated that the measure has acceptable interrater reliability, test-retest reliability, internal consistency, known group validity, and convergent validity. Additionally, we presented data on the minimum amount of change in scores needed to be confident that a difference in scores is more than the result of measurement error. One advantage of the AM-ULA is that it can be used to assess activity performance for users of all
Rater 2 a
Itemtest
Itemrest
a
Table 5 ANOVAs comparing mean scores of raters 1 and 2 on AM-ULA items by amputation level
0.89 0.89
0.69 0.73
0.61 0.67
0.90 0.90
Items
Transradial (nZ23)
Transhumeral (nZ23)
Shoulder (nZ6)
P
0.69
0.89
0.60
0.54
0.90
0.51 0.56 0.61 0.72 0.49 0.50 0.49 0.55 0.51 0.37 0.33 0.40 0.64 0.54 0.61
0.89 0.89 0.89 0.88 0.89 0.89 0.89 0.89 0.89 0.90 0.89 0.89 0.88 0.89 0.89
0.50 0.68 0.50 0.75 0.68 0.69 0.72 0.67 0.65 0.48 0.39 0.70 0.67 0.63 0.62
0.42 0.60 0.41 0.70 0.59 0.61 0.66 0.59 0.59 0.38 0.31 0.64 0.60 0.56 0.51
0.91 0.90 0.91 0.90 0.90 0.90 0.90 0.90 0.90 0.91 0.91 0.90 0.90 0.90 0.91
Brush hair Put on T-shirt Remove T-shirt Button shirt Zip jacket Socks Tie shoes Use cup Use fork Use spoon Pour soda Write word Use scissors Door knob Use phone Hammer Fold towel
1.90.9 1.80.9 1.91.0 1.00.6 1.70.8 2.20.8 2.20.8 2.01.2 1.70.9 1.60.8 2.10.9 2.10.6 1.91.1 1.71.0 2.60.6 2.00.7 2.40.6
0.80.8 1.01.0 1.31.2 1.41.0 1.41.1 1.30.8 1.31.0 0.70.8 0.80.9 0.50.8 1.60.9 1.60.7 1.81.0 1.10.9 2.20.7 1.40.9 2.00.7
0.40.5 1.30.6 1.50.7 1.01.0 0.90.5 1.50.4 1.30.8 0.10.2 0.60.5 0.30.4 1.21.1 1.20.8 1.60.9 1.21.0 1.71.0 0.81.0 1.30.9
.000 .069 .414 .368 .203 .006 .004 .000 .004 .000 .073 .109 .785 .426 .012 .010 .003
Itemtest
Itemrest
0.57 0.73
0.47 0.69
0.75 0.58 0.63 0.67 0.77 0.61 0.59 0.59 0.64 0.56 0.47 0.44 0.48 0.71 0.61 0.69
0.89
0.91
NOTE. Values are mean SD or as otherwise indicated.
www.archives-pmr.org
Activities measure for upper limb amputees
493
Table 6 Pearson correlation between the AM-ULA and concurrent measures (NZ49) Measure
Correlation
Box and Block 0.63 Jebsen-Taylor Hand Function Test (items/s) Writing 0.27 Page turning 0.52 Small items 0.55 Feeding 0.61 Checkers 0.42 Light cans 0.69 Heavy cans 0.60 UEFS summary score 0.44 UEFS number of items 0.09
P <.0001 .061 .001 <.0001 <.0001 .002 <.0001 <.0001 .002 .540
types of upper limb prostheses: body powered, hybrid, or myoelectric. The measurement development process involved experienced occupational and physical therapists. An iterative process helped refine content, scoring elements, and criteria, ensuring that the AM-ULA had good content validity. We found good concordance between 2 independent rater’s ratings, and independent raters and the site therapist ratings. This is somewhat remarkable in that the hand therapist raters could watch the videotaped test performance repeatedly, while the site therapists scored tests immediately after administration. These findings suggest that the AM-ULA can be implemented and scored reliably by clinicians in practice. That said, the site therapists (L.A., J.D., R.D., C.E., L.S.W.) were involved in development of the AM-ULA test, and thus were very familiar with scoring. Therefore, like any other performance-based measure of function, we expect that training will be necessary to insure that clinicians understand subtask definitions and scoring criteria. Further study is needed to examine interrater reliability with a different group of raters not involved in measurement development. There are limitations to interpretation of the MDC. A change greater than the MDC does not indicate whether the changes in scores are clinically important. Future studies are needed to determine the magnitude of score change associated with clinically relevant improvements and deterioration. The analyses demonstrated moderate convergent validity between the AM-ULA and related measures and moderate correlation between all measures of dexterity and the AM-ULA, with the exception of the Jebsen-Taylor writing test. A finding of moderate correlation between dexterity and performance of daily activities is consistent with the findings from other studies examining relations between measures of dexterity and measures of functional performance.24 The Jebsen writing test is a timed test that requires the administrator to count the number of legible letters written in the allotted time. Thus, this measure assesses penmanship as well as dexterity, which may explain the weaker association. A moderate negative correlation between the AM-ULA and the UEFS was observed. In the AM-ULA higher scores indicate better functioning, while in the UEFS higher scores indicate worse functioning. This finding is consistent with previous research reporting only moderate correlations between www.archives-pmr.org
self-reported and performance measures. Further, the AM-ULA and UEFS assess different, but related, constructs. The AM-ULA is scored by the clinician, while the amputee is using or attempting to use the prosthesis. Difficulty of performing items on the UEFS is rated by amputees regardless of whether or not they use their prosthesis. Thus, UEFS total scores do not take use of the prosthesis into consideration. However, amputees are asked to indicate whether they usually use the prosthesis to perform the task. In our study, the number of activities for which the prosthesis is usually used was not correlated to observed function using the AM-ULA.
Study limitations The UEFS scores that we obtained were based on a 22-item version of the measure, not the original 23-item measure developed by Heinemann et al.8 Because 1 UEFS item was omitted during our administration (washing face), we cannot say with certainty that identical results would have been obtained if all the original UEFS items had been used. The impact of amputation of the dominant compared with the nondominant upper limb on AM-ULA scores was not examined. Data were collected on hand dominance prior to amputation and after amputation. We found that 100% of the sample (not including 1 congenital bilateral amputee) reported that they had shifted their hand dominance after amputation of the dominant extremity. It is possible that responsiveness of the AM-ULA would vary for new amputees undergoing prosthetic rehabilitation depending on whether or not the amputation was of the dominant or the nondominant side. However, our study was not designed to evaluate this. Future studies are needed to examine the influence of preamputation hand dominance on change in AM-ULA scores over time. A limitation of our approach is that the item set for the AM-ULA was drawn from items that were common to the UEFS and OPUS measure, and no additional content validation was performed. Thus, there was no expert consensus as to the appropriateness or comprehensiveness of the items that were selected for the new measure.
Conclusions The AM-ULA is a new 18-item measure of activity performance for adults with upper limb amputation that considers task completion, speed, movement quality, skillfulness of prosthetic use, and independence in its rating system. It has excellent internal consistency, good interrater reliability, test-retest reliability, and demonstrated known group and convergent validity. The AM-ULA is appropriate for users of all types of prosthetic devices and addresses the recognized need for a validated performancebased measure of activity for adults. It is scored from 0 to 40, with higher scores indicating better functional performance. The 3.7-point MDC90 can be used to assist clinicians in interpreting change in test scores with repeated administration.
Suppliers a. DEKA Integrated Solutions, Inc, 340 Commercial St, #401, Manchester, NH 03101-1121. b. Linacre JM. (2012) Winsteps Rasch measurement computer program. Beaverton, OR. Winsteps.com.
494
Keywords Amputation; Disability evaluation; Psychometrics; Rehabilitation; Upper extremity
Corresponding author Linda Resnik, PT, PhD, Research Health Scientist, Providence VA Medical Center, 830 Chalkstone Ave, Providence, RI 02908. E-mail address: [email protected].
Supplementary data Supplementary data related to this article can be found online at http://dx.doi.org/10.1016/j.apmr.2012.10.004
References 1. Wright V. Measurement of functional outcome with individuals who use upper extremity prosthetic devices: current and future directions. J Prosthet Orthot 2006;18:46-56. 2. Miller LA, Swanson S. Introduction to the Academy’s State of the Science Conference on upper limb prosthetic outcome measures. J Prosthet Orthot 2009;21(Suppl):P1-2. 3. Hill W, Stavdahl O, Hermansson L, et al. Functional outcomes in the WHO-ICF model: establishment of the Upper Limb Prosthetic Outcome Measures Group. J Prosthet Orthot 2009;21:115-9. 4. Hill W, Kyberd P, Hermansson LN, et al. Upper Limb Prosthetic Outcome Measures (UPLOM): a working group and their findings. J Prosthet Orthot 2009;21(4 Suppl):P69-82. 5. Wright V. Prosthetic outcome measures for use with upper limb amputees: a systematic review of the peer-reviewed literature, 1970 to 2009. J Prosthet Orthot 2009;21(4 Suppl):P3-63. 6. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001. 7. Miller L, Swanson S. Summary and recommendations of the Academy’s State of the Science Conference on upper limb prosthetic outcome measures. J Prosthet Orthot 2009;21(4 Suppl):83-9. 8. Heinemann AW, Bode RK, O’Reilly C. Development and measurement properties of the Orthotics and Prosthetics Users’ Survey (OPUS): a comprehensive set of clinical outcome instruments. Prosthet Orthot Int 2003;27:191-206.
L. Resnik et al 9. Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]. The Upper Extremity Collaborative Group (UECG). Am J Ind Med 1996;29:602-8. 10. Lindner HY, Natterlund BS, Hermansson LM. Upper limb prosthetic outcome measures: review and content comparison based on International Classification of Functioning, Disability and Health. Prosthet Orthot Int 2010;34:109-28. 11. Hermansson LM, Bodin L, Eliasson AC. Intra- and inter-rater reliability of the assessment of capacity for myoelectric control. J Rehabil Med 2006;38:118-23. 12. Hermansson LM, Fisher AG, Bernspa˚ng B, Eliasson AC. Assessment of capacity for myoelectric control: a new Rasch-built measure of prosthetic hand control. J Rehabil Med 2005;37:166-71. 13. Gallagher P, Franchignoni F, Giordano A, MacLachlan M. Trinity amputation and prosthesis experience scales: a psychometric assessment using classical test theory and Rasch analysis. Am J Phys Med Rehabil 2010;89:487-96. 14. Resnik L. Research update: VA study to optimize DEKA arm. J Rehabil Res Dev 2010;47:ix-x. 15. Atkins D, Meier R. Comprehensive management of the upper-limb amputee. New York: Springer-Verlag; 1989. 16. Smurr LM, Gulick K, Yancosek K, Ganz O. Managing the upper extremity amputee: a protocol for success. J Hand Ther 2008;21:16075, quiz 176. 17. Portney L, Watkins M. Foundations of clinical research applications to practice. Stamford: Appleton & Lange; 1993. 18. Streiner D, Norman G. Health measurement scales: a practical guide to their development and use. New York: Oxford Univ Pr; 2003. 19. Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intraindividual changes in health-related quality of life. J Clin Epidemiol 1999;52:861-73. 20. Mathiowetz V, Volland G, Kashman N, Weber K. Adult norms for the Box and Block Test of manual dexterity. Am J Occup Ther 1985;39:386-91. 21. Jebsen RH, Taylor N, Trieschmann RB, Trotter MJ, Howard LA. An objective and standardized test of hand function. Arch Phys Med Rehabil 1969;50:311-9. 22. Burger H, Franchignoni F, Heinemann AW, Kotnik S, Giordano A. Validation of the orthotics and prosthetics user survey upper extremity functional status module in people with unilateral upper limb amputation. J Rehabil Med 2008;40:393-9. 23. Linacre JM. A user’s guide to WINSTEPS, MINISTEP Rasch-model computer programs. Chicago: 2006. 24. Lin KC, Chuang LL, Wu CY, Hsieh YW, Chang WY. Responsiveness and validity of three dexterous function measures in stroke rehabilitation. J Rehabil Res Dev 2010;47:563-71.
www.archives-pmr.org
Activities measure for upper limb amputees
494.e1
Supplemental Appendix S1 Definition of Tasks 1. )Put toothpaste on a toothbrush a) Grasp a tube of toothpaste b) Remove the cap from the toothpaste c) Grasp/position a toothbrush d) Squeeze toothpaste onto toothbrush e) Replace the cap f) Release the tube of toothpaste 2. Brush/comb hair (unilateral) a) Grasp comb b) Bring comb to head c) Comb hair or perform a combing motion d) Release the comb from grasp 3. Put on t-shirt a) Grasp the t-shirt b) Thread head through neck opening c) Thread arms through sleeves d) Pull shirt down to fit properly 4. Remove t-shirt a) Grasp the t-shirt b) Lift the shirt over head c) Undress arms d) Place shirt on table e) Release grasp 5. Button shirt with front buttons a) Grasp the shirt b) Push button through hole c) Pull button out other side d) Complete buttoning of 3 buttons e) Release grasp on shirt 6. Attach end of zipper and zip jacket a) Grasp zipper and vest b) Initiate the zipper c) Pull zipper up at least two-thirds of the way d) Unzip the zipper 7. Put on socks a) Grasp the sock b) Pull the sock over toes c) Pull the sock over the heel so that it is fully donned 8. Tie shoelaces a) Grasp a lace with each hand b) Criss-cross the laces c) Make loops d) Pull the bow tight e) Release grasp on the laces 9. Drink from a paper cup (unilateral) a) Pick up a paper cup from a table b) Bring the cup to touch the mouth c) Tilt the cup and simulate drinking (or drink) d) Return the cup to the table e) Release the grip on the cup 10. Use a fork (unilateral) a) Grasp fork b) Bring fork all the way to the mouth as if taking a bite of food c) Move fork away from mouth and return to table d) Release grip on the fork 11. Use a spoon (unilateral) a) Grasp spoon
www.archives-pmr.org
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
b) Bring spoon all the way to the mouth as if taking a bite of food c) Move spoon away from mouth and return to table d) Release grip on the spoon ) Cut meat with knife and fork a) Grasp the utensils b) Position the utensils c) Spear dough/“meat” with fork d) Make 3 cuts with knife e) Release the utensils f) Set utensils down on table or plate Pour from a 12oz can a) Pick up soda can b) Grasp cup with opposite hand to stabilize c) Pour soda into cup d) Set can down on table e) Release cup and can Write the word “LETTER” legibly (unilateral) a) Grasp the pen or pencil b) Write the word “LETTER” c) Set the pen down d) Release grip Use a pair of scissors a) Grasp the scissors b) Grasp paper in the other hand c) Cut the paper with scissors (make at least 3 cuts) d) Release paper e) Release scissors Turn a round door knob (unilateral) a) Reach for the door knob b) Grasp the door knob c) Turn the door knob to release the latch d) Release the knob ) Use a key in lock (unilateral) a) Grasp the key b) Insert key into keyhole c) Turn key to lock or unlock d) Remove key from keyhole e) Release the key ) Carry a laundry basket a) Grasp opposite sides of laundry basket b) Pick up the basket c) Ambulate with the basket held securely d) Set basket on a counter height surface e) Release the hands from basket Dial a touch tone phone a) Grasp cell phone in 1 hand b) Position phone to see touch pad c) Use appropriate part of terminal device or sound hand to press buttons to dial a phone number Use a hammer and nail a) Pick up the hammer with 1 hand and the nail with the other hand b) Position the nail vertically on top of wood c) Lift hammer and bring it down to drive the nail into the wood d) Remove hand that was positioning the nail away and continue to hammer nail into wood e) Release hammer from grip Fold a bath towel a) Grasp ends of towel b) Brings ends together to fold twice
494.e2 c) Release grip on the towel 22. )Open an envelope a) Secure the envelope in 1 hand b) Using a letter opener, pen, or finger break the top seal of envelope c) Open envelope d) Release the envelope 23. )Stir in a bowl a) Grasp a spoon b) Place the spoon in a bowl c) Stabilize the bowl with other hand as needed d) Stir the spoon within the bowl 24. Reach overhead (ie, to the top of refrigerator door) (unilateral) a) Lift the arm overhead b) Bring arm/hand to top of surface c) Grasp object on shelf d) Bring the arm down with object in hand ) Indicates that item was not included in final AM-ULA measure.
Supplemental Appendix S2 AM-ULA Instructions Read the following to the subject: “I want to see how you do some everyday activities. Please use your prosthetic arm to do these activities. Please use your sound arm only on those activities that require 2 hands.” Use the instructions below to guide the subject in performing the testing tasks: 1. Please take this toothbrush and toothpaste, remove the cap from the tube, and squeeze the toothpaste on the toothbrush. Then replace the cap and release the tube. Be sure to use your prosthesis while doing this.) 2. Please take this comb and, while grasping it, run it through your hair (or over the top of their head if bald). Then put the comb down. Try to use only your prosthesis to do this task. 3. Please put this t-shirt on. I want you to grasp the t-shirt, thread your head through the neck opening, thread your arms through the arm holes, and pull the shirt down over your body. Be sure to use your prosthesis while doing this. 4. Now I’d like you to remove the t-shirt. While grasping the tshirt, lift it over your head, remove your arms from the sleeves, and place the shirt on the table. Be sure to use your prosthesis while doing this. 5. Please put on this button-up shirt (therapist may help don shirt) and then button 3 buttons by putting the buttons through the holes and pulling them out the other side. Be sure to use your prosthesis while doing this. 6. Please put this vest on (therapist may help don vest), start the zipper, and pull it at least two-thirds of the way up. Then unzip the zipper. Be sure to use your prosthesis while doing this. 7. Please take this sock and put it on, pull the sock over your heel and pull it up all the way so that it fits well. Then remove the sock from your foot. Be sure to use your prosthesis while doing this. 8. Please take this shoe (placed either on a tabletop or on the floor) and tie the laces. Grasp a lace in each hand, criss-cross the laces, making loops, and tie the bow tight. Be sure to use your prosthesis while doing this.
L. Resnik et al 9. Pick up this paper cup from the table, bring it all the way to your mouth, and pretend to drink from it. Then put the cup back on the table. Try to use only your prosthesis to do this task. 10. Please grasp this fork and bring it to your mouth as if you were going to take a bite of food from it. Then put the fork back on the table. Try to use only your prosthesis to do this task. 11. Please grasp this spoon with your prosthesis and bring it to your mouth as if you were going to take a bite of food from it. Then put the spoon back on the table. Be sure to use your prosthesis while doing this. 12. On the table in front of you is a plate, knife, fork, and a piece of play dough in the shape of a hot dog. Please spear the “meat” with the fork, cut the meat 3 times with the knife. When you finish, place the utensils back on the table or the plate. Be sure to use your prosthesis while doing this.) 13. Please pick up the soda can from the table with 1 hand and grasp the cup with your other hand. Then pour the soda from the can into the cup. When finished set the can and cup (if necessary) down on the table. Be sure to use your prosthesis while doing this.) 14. Please grasp this pen with your prosthesis and write the word “LETTER” on the blank sheet of paper on the clipboard. Then set the pen down on the table. Try to use only your prosthesis to do this task. 15. Please take this blank piece of paper and, grasping scissors in 1 hand and with the paper in the other, make at least 3 cuts in the paper. Then place the cut paper and scissors on the table. 16. Please reach for this door knob (use a round door knob), grasp the knob, and turn it until the latch is released. Then release the knob. Try to use only your prosthesis to do this task. 17. Please take this key and insert it into the keyhole. Turn the key to lock (or unlock) the door, then remove the key from the keyhole and give it back to me. Try to use only your prosthesis to do this task.) 18. Please grasp both sides of this laundry basket, pick it up, and carry it over here (please show subject where). Set the basket down and release your hands from it.) 19. Please take this cell phone and dial a phone number with it. Be sure to use your prosthesis while doing this. 20. Please pick up the hammer in 1 hand and the nail in the other. Position the nail vertically on the piece of wood and use the hammer to drive the nail into it. Remove your hand that was holding the nail and continue hammering the nail into the wood. Then put the hammer down. 21. Please grasp the ends of this bath towel, bringing the ends of the towel together to fold it twice. Then place the folded towel back on the table. Be sure to use your prosthesis while doing this. 22. Please secure this envelope in 1 hand and, using a letter opener, pen, or finger, break the seal on it. Then open the envelope and place it on the table. Be sure to use your prosthesis while doing this.) 23. Please grasp this spoon with your prosthetic hand and, using your other hand to stabilize the bowl, use the spoon to stir the cereal in the bowl.) 24. Using your prosthetic side, reach up to this overhead shelf and grasp the object (a lightweight cup) on the shelf, now lower your arm and hand the object to me. Try to use only your prosthesis to do this task. ) Indicates item was deleted from the final AM-ULA measure. www.archives-pmr.org
Activities measure for upper limb amputees
Scoring Instructions Overview The purpose of this measure is to provide a clinician rated measure of an upper limb amputee’s performance of daily functional activities using the prosthesis. The scale is graded on a scale of 0-4 (unable to excellent). Rules of Administration Tester should familiarize themselves with all tasks and necessary sub-components (see attached Task List). Tester asks the Amputee to perform all of the unilateral tasks using the prosthesis and to use the prosthesis as much as possible when performing tasks that require bilateral engagement. Tester is not to provide assistance, except for safety. The amputee may use an aide, such as a button hook or other adaptive equipment. Scoring Rules Amputees are scored on 5 elements: Extent of completion. Grade the extent of completion of all sub-tasks of the activity.
www.archives-pmr.org
494.e3 Speed of completion of entire activity. Grade the speed of task performance as compared to performance with a sound limb. Movement quality: Grade the amount of awkwardness or compensatory movements resulting in/from lack of prepositioning, limitations of the device, lack of skilled use or any other reason. Skillfulness of prosthetic use. Grade the type of use (no active use, use as a stabilizer, assist, or prime mover), control over voluntary grip functions. Independence. Grade the use of assistive device or adaptive equipment. The grid below shows the scoring criteria for each element. The lowest score in any category is the one that is awarded for the overall score.
Note Use of “excellent” category compares performance of the activity to performance with a sound limb. Test administrators should be aware that there will be few amputees who will be graded an “excellent.” This is a purposefully high bar for comparison; however, it is included to avoid a ceiling effectdwhereby there is no room for improvement, with advances in technology and methods of controls that will be evident in the near future.
494.e4
Grade
Completion of Sub-tasks
Speed of Completion (as Compared to Non-disabled)
0 Unable 1 Poor
Less than all All
2 Fair
Skillfulness of Prosthesis use
Independence
N/A Very slow to slow
N/A Very awkward, many compensatory movements
N/A May or may not use assistive device
All
Slow to medium
Some awkwardness or compensatory movement
3 Good
All
Medium-fast to normal
Minimal to no awkwardness or compensatory movement
4 Excellent
All
Equivalent to nondisabled
Excellent movement quality, no awkwardness or compensatory movement
No prosthetic use Inappropriate choice of grip for task (if choice is available) Loses grip multiple times during task, lack of proportional control (if available) Multiple unintentional activation of a control Prosthesis only used as a stabilizer during bimanual activities, ie, there is no active use or use of grip, but arm or terminal device used to stabilize during the task Sub-optimal choice of grip for task (if choice is available) Use of prosthesis as an assist for bimanual activities or prime mover unilateral activities Loses grip once during task More than 1 attempt needed to pre-position object within grasp and/or more than minimal awkwardness in pre-positioning object One incidence of unintentional activation of a control Fair proportional control Skilled use of prosthesis as an assist for bimanual activities or as a prime mover for unilateral activities Quick and easy pre-positioning of object within grasp No unintentional loss of grip Optimal choice of grip for task (if choice is available) No inadvertent loss of grip or unwanted movement Optimal choice of grip for task (if choice is available) Sound side NOT used to pre-position object within grasp
May or may not use assistive device
May or may not make use of assistive device
No assistive device use
L. Resnik et al
www.archives-pmr.org
Movement Quality