- Email: [email protected]
Validity, Reliability, and Responsiveness of a Digital Version of the Visual Analog Scale
SCIENTIFIC/CLINICAL ARTICLE JHT READ
FOR
CREDIT ARTICLE #206.
Validity, Reliability, and Responsiveness of a Digital Version of the Visual Analog Scale Bhagwant S. Sindhu, PhD, OTR Department of Occupational Science and Technology, University of Wisconsin-Milwaukee, Milwaukee, Wisconsin
Orit Shechtman, PhD, OTR/L Laura Tuckey, MS, OTR/L Department of Occupational Therapy, University of Florida, Gainesville, Florida
Musculoskeletal pain puts a substantial burden on society because of loss of productivity4 and increased health care costs.5e7 Pain greatly reduces work capacity,8 is higher in women than in men, and increases with age.9,10 In particular, upper extremity musculoskeletal pain has a prevalence of 3e7% among U.S. adults.10 Various health agencies and regulatory bodies have mandated that pain level be assessed on contact with the health care providers.11e14 Pain intensity, the severity of perceived pain, is commonly measured by two assessments: the numerical rating scale (NRS) and visual analog scale (VAS).15 The NRS consists of a range of numbers, with the smaller numbers indicating less pain. An 11-point NRS rates pain intensity corresponding to an integer number between 0 and 10. People rate their pain by indicating a number, either verbally (NRS-V) or by marking a paper (NRS-P), that lists the numbers horizontally in ascending order.15 The VAS usually Correspondence and reprint requests to Bhagwant S. Sindhu, PhD, OTR, Department of Occupational Science and Technology, University of Wisconsin-Milwaukee, Milwaukee, WI 53201; e-mail:. 0894-1130/$ - see front matter Ó 2011 Hanley & Belfus, an imprint of Elsevier Inc. All rights reserved. doi:10.1016/j.jht.2011.06.003
356
JOURNAL OF HAND THERAPY
ABSTRACT: The design used in this study was a prospective cohort. Pain intensity levels recorded by the digital version of the visual analog scale (VAS-D) are easy to both score and share with other health care professionals. The purpose of the study was to examine the testeretest reliability, concurrent validity, and responsiveness of the VAS-D. Thirty-three people with upper extremity injuries reported pain intensity levels before and after performing four maximal grip contractions (pre- and postgripping). Our version of the VAS-D had high testeretest reliability (r ¼ 0.96) and good concurrent validity (r ¼ 0.84e0.97) with both the paper version of the VAS (VAS-P) and the verbal numerical rating scale (NRS-V). Responsiveness of the VAS-D was indicated by a significant increase in pain levels from pre- to postgripping. Similar responsiveness to that of the VAS-P and NRS-V was indicated by similar effect size coefficients and analysis of variance of pain change scores. In conclusion, the VAS-D is a reliable, valid, and responsive measure of pain intensity for people with upper extremity injuries. However, differences in accuracy (resolution) among the VAS-D, VAS-P, or NRS-V may render the three pain scales not fully compatible. Level of Evidence: Not applicable. J HAND THER. 2011;24:356–64.
consists of a 10-centimeter (cm) line placed horizontally on a piece of paper, with an anchor point on each end.16 One anchor of this line represents ‘‘no pain,’’ and the other anchor represents ‘‘maximum perceived pain intensity.’’15 People rate their perceived pain intensity by marking a vertical line across the horizontal scale line.16 The distance from the ‘‘no pain’’ anchor to this mark is measured by a ruler and used as the overall pain intensity score.15,16 This distance is measured in millimeters so that scores range from 0 to 100,16 forming a 101-point VAS, such as the one in the Short Form of the McGill Pain Questionnaire.17 Both NRS and VAS have wellestablished psychometric properties; they are valid,15,18e23 reliable,24 and equally sensitive to changes in pain intensity.25e27 Both measures are nonintrusive, easy to administer, suitable for repeated use, and require simple instructions; the NRS is easier to score.15,16,28 Many versions of the VAS have been used, including various paper and digital versions.1e3,17,29,30 The advantages of the digital versions of VAS (VAS-D) include easier scoring and record keeping.1 The VAS-P is scored using a ruler, which can be time consuming and has potential for human error. In contrast, the VAS-D is scored automatically and
immediately by a computer, without the potential for human error.29e31 In addition, the VAS-D generates and stores electronic patient data, which are portable, easy to share, instantly accessible, and time stamped.2,3 Despite these advantages, limited research has been conducted to determine the psychometric properties (reliability, validity, and responsiveness) of the VAS-D. Psychometric properties of various versions of the VAS-D have been studied. The VAS-D displayed on a personal digital assistant was found to be valid when compared with both an 11-point NRS-P3,29 and a 10-cm VAS-P despite differences in scale line length.2,3,29 The VAS-D displayed on a computer monitor was found to be valid when compared with a 10-cm VAS-P.1,17 However, the monitor display could be rated on either a 102-point scale1 or an 11-point scale instead of the standard 101-point VAS.17 Overall, the various versions of the VAS-D appear to be valid for use in general clinical populations. However, when reviewing the literature, we did not find studies reporting on the responsiveness (e.g., the ability of measure change in pain) of the VAS-D. Pain assessments must be responsive, that is, able to detect clinically meaningful changes over time32 so that therapists can examine the effect of their treatment on pain. When pain is assessed both before and after an intervention session, responsiveness is calculated as the change in pain level from pre- to post-therapy or ‘‘change scores.’’ A common statistical method used to calculate responsiveness is effect size (ES).33,34 ES is measured as the standardized difference between two means and thus evaluates the magnitude of differences or the extent of a treatment effect. ES is calculated by dividing the average change scores (in the nominator) by standard deviation (SD) of prescore (in the denominator).35 ES coefficients are generally interpreted on the basis of Cohen’s conventional criteria, in which 0.20 (or 20% of one SD) indicates a small effect, 0.50 (or 50% of one SD) indicates a moderate effect, and 0.8 (or 80% of one SD) indicates a large effect. Thus, a larger Cohen’s ES indicates a greater difference between pre- and post-therapy pain scores or a greater responsiveness. Similar responsiveness between two pain scales indicates no difference in the ability of these scales to measure change in pain. Similar responsiveness is implied when the ES of the two scales is similar; for example, when two assessments have a large ES.34 The responsiveness of the VAS-P was found to be similar to the NRS26,27 and greater than the McGill Pain Questionnaire,36,37 which measures the multidimensional nature of pain.38 However, it is unknown if the VAS-P and various versions of the VAS-D have similar responsiveness. The purpose of the present study was to examine the psychometric properties of the VAS-D, including testeretest reliability, validity, and responsiveness.
METHODS Participants Thirty-three participants (17 males and 16 females) with unilateral musculoskeletal disorders and injuries of the elbow, forearm, or hand were recruited from various hand therapy clinics. The participants were between the ages of 18 and 65, and they were excluded if they verbally reported their pain intensity to be greater than 7 on a scale of 0e10 and were unable to safely perform four maximal grips with their affected extremity as determined by their hand therapists. These exclusion criteria were implemented for maintaining patient safety, allowing us to exclude individuals who were not sufficiently healed from participating in the study. Before completing the study protocol, each participant read and signed an informed consent form approved by the institutional review board.
Materials and Equipment A demographic questionnaire was used to collect information on age, gender, and injury-related variables, such as diagnosis and site of injury. Maximal grip efforts were performed using an electronic Jamar dynamometer (Thought Technology Ltd., Montreal, Quebec, Canada). Three different scales were used to record pain intensity before (pre) and after (post) four maximal grip contractions: the VAS-D, the paper version of the visual analog scale (VAS-P), and the verbal version of the NRS (NRS-V). Both the VAS-P and VAS-D consisted of a 10-cm horizontal line anchored by two extremes of pain, that is, ‘‘no pain’’ (numerical score of 0) and ‘‘pain as bad as it could be’’ (numerical score of 10). The VAS-P consisted of a horizontal 10-cm black line on a white piece of paper, and the pain level was marked by the participant as a vertical line crossing the scale. The pain score was measured to the closest millimeter using a ruler; in other words, the minimum measurable score was 1 mm or a score of 0.1 on a scale of 0e10 (Figure 1). In contrast, the VAS-D, which was a modified version of a computer program developed by Arvidsson,39 was administered on a touch screen (AccuSync LCD; 52V, 15-inch, NEC Display Solutions, Itasca, IL) with a resolution of 1,024 3 768 pixels. Pain level was marked by moving a slider (0.8 cm or 28 pixels long) on a horizontal line (10 cm or 334 pixels long). The pain score was measured by the MediaLab software (Version 2006.2.34; Empirisoft Corp., New York, NY) as the final location of the center of the slider (Figure 2). OctobereDecember 2011 357
FIGURE 1. Picture of the paper version of the visual analog scale that was used for assessing pain. Note: Thickness of pencil graphite ¼ 1e2 mm.
Protocol
Statistical Analysis
Each participant performed a total of four maximal grip contractions with their injured extremity while sitting in the position recommended by the American Society of Hand Therapists.40 Each grip lasted 6 seconds with a 1-minute rest interval between grips. Pain level was recorded with all three pain intensity scales (VAS-P, VAS-D, and NRS-V) both before (pre) and after (post) gripping. Baseline (pregripping) pain level was recorded twice (5e10 minutes apart) to examine testeretest reliability. Postgripping pain level was recorded once, immediately after the fourth maximal grip contraction. To control order effects, half of the participants completed the VAS-P first and the other half completed the VAS-D first. The NRS-V was always completed last to prevent the participants from replicating their verbalized scores on the VAS-P and VAS-D. Participants were instructed to rate their current level of pain using the three different pain intensity scales. Standardized verbal instructions were used throughout the study as follows: for the VAS-P: ‘‘Mark a vertical line between 0 and 10 at a point that indicates your pain level,’’ for the VAS-D: ‘‘Move the slider on the line between 0 and 10 to the point that indicates your pain level,’’ and for the NRS-V: ‘‘State a number between 0 and 10 that indicates your pain level.’’
Testeretest reliability was determined by calculating intraclass correlation coefficients (ICCs) between the two baseline (pregripping) pain scores. Concurrent validity was determined by calculating Pearson product-moment correlation coefficients between pain scores on the VAS-P, VAS-D, and NRS-V for both pregripping pain levels (the second baseline pain scores) and postgripping pain levels. To ensure agreement between correlational and inferential analysis, a repeated-measures analysis of variance (ANOVA) was conducted to compare pre- and postgripping pain levels as recorded by the VAS-P, VAS-D, and NRS-V. The ANOVA consisted of two within-subject variables: scale (VAS-P, VAS-D, NRSV) and time (pregripping vs. postgripping). Post hoc paired samples t-tests were used to discern the differences among the three scales. The Bonferroni correction for the six comparisons set the alpha level at 0.008. All other comparisons were deemed significant at the 0.05 alpha level. Responsiveness, which is the ability of the scale to measure change in pain, was determined by calculating change scores between pre- and postgripping pain levels. Change score (postgripping pregripping) was calculated for each participant on each scale and compared using a repeated-measures ANOVA with one within-subjects variable: scale (VAS-P, VAS-D, NRS-V). Furthermore, the change
FIGURE 2. Picture of the digital version of the visual analog scale that was used for assessing pain. Note: Slider thickness ¼ 8 mm. 358
JOURNAL OF HAND THERAPY
scores were used to calculate the ES coefficients for the VAS-P, VAS-D, and NRS-V. The ES coefficients were obtained by dividing the average change score by the SD of the pregripping pain score.35 Greater ES coefficients indicate larger change in pain intensity from pre- to postgripping. Data analysis was conducted using the SPSS 16.0 software package (SPSS Inc., Chicago, IL).
RESULTS The average age of the study participants was 39 6 12 years, and the most common location of the injury was the hand (39% of all participants); other demographic variables are reported in Table 1. Preand postgripping average pain intensity levels as measured by the VAS-P, VAS-D, and NRS-V are reported in Figure 3. Testeretest reliability of the VAS-D, as measured by ICC between the two pregripping (baseline) pain scores, was r ¼ 0.96 (Table 2). Concurrent validity of the VAS-D, as measured by Pearson product-moment correlation coefficients between pain scores on the VAS-P, VAS-D, and NRS-V ranged from r ¼ 0.84 to 0.97 (Table 3). The ANOVA (Table 4) comparing pain scores on the three scales during pre- and postgripping revealed significant main effects of both within-subject variables, time
(F ¼ 9.31; p # 0.005) and scale (F ¼ 8.02; p # 0.008). For the main effect of time, postgripping pain scores were significantly greater than pregripping pain scores. For the main effect of scale, the significant difference in pain scores among the three scales (VAS-P, VAS-D, and NRS-V) was further investigated using post hoc paired samples t-tests. Pain scores were significantly greater for the VAS-D than the VAS-P both at pregripping (t ¼ 4.24; p , 0.001) and postgripping (t ¼ 3.27; p # 0.003); also, pain scores were significantly smaller for the VAS-P than the NRS-V at postgripping (t ¼ 3.53; p # 0.001). All other differences were not significant (Table 5). Responsiveness coefficients revealed a small ES (0.20e0.50) for all three scales, the NRS-V, VAS-D, and VAS-P (Table 3). The ANOVA on change scores revealed no significant differences in responsiveness between the three scales (F ¼ 1.36; p # 0.25).
DISCUSSION The purpose of this study was to examine the psychometric properties, that is, reliability, validity, and responsiveness, of a VAS-D and compare them to two other pain assessment scales, namely the VAS-P and NRS-V. When an assessment is used for clinical decision making, acceptable reliability is indicated by
TABLE 1. Demographic and Injury-related Characteristics of Study Participants Demographic and Injury-related Characteristics
Men (N ¼ 17)
Women (N ¼ 16)
All (N ¼ 33)
Mean (SD) or Number (%)
Mean (SD) or Number (%)
Mean (SD) or Number (%)
Age (yr)
37 (11)
Grip strength (lbs)
68 (34.03)
Current work status Full-time Part-time Not working
4 (23.5) 2 (11.8) 11 (64.7)
8 (50) 5 (31.25) 3 (18.75)
12 (36.4) 8 (21.2) 13 (42.4)
Dominant extremity Left Right
4 (23.5) 13 (76.5)
3 (18.75) 13 (81.25)
7 (21.2) 26 (78.8)
Injured extremity Left Right
10 (58.8) 7 (41.2)
6 (37.5) 10 (62.5)
16 (48.5) 17 (51.5)
Location of injury Hand Wrist Forearm Elbow
8 4 1 4
Duration of injury (mo)
7 (14.6)
(47.06) (23.53) (5.88) (23.53)
39 (13.7) 47.26 (23.62)
5 6 3 2
(31.25) (37.5) (18.75) (12.5)
31 (58.1)
39 (12.3) 58.13 (30.91)
13 10 4 6
(39.4) (30.3) (12.1) (18.2)
19 (42.9)
Cause of injury Traumatic Cumulative trauma Do not know
16 (94.1) 1 (5.9) —
6 (37.5) 9 (56.25) 1 (6.25)
22 (66.7) 10 (30.3) 1 (3)
Medical management Currently taking pain medications Undergone surgical intervention
3 (17.64) 10 (58.8)
5 (31.25) 9 (56.25)
8 (24.24) 19 (57.6)
SD ¼ standard deviation.
OctobereDecember 2011 359
2.5 First pre-gripping Second pre-gripping Post-gripping
Pain Intensity Score
2.0
1.5
1.0
0.5
0.0 VAS-P
VAS-D
NRS-V
FIGURE 3. Averages and standard errors of pre- and postgripping pain scores using the VAS-P, VAS-D, and NRS-V. VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale. correlation coefficients exceeding r ¼ 0.9.41 In the present study, we found the testeretest reliability of two consecutive pregripping pain scores on the VAS-D to be excellent (r ¼ 0.96) and identical to that of the VAS-P (Table 2). This high correlation coefficient indicates that participants identified almost the same location on a horizontal line of the VAS-P and VAS-D when reporting the level of pain. In contrast, testeretest reliability of the verbal form of NRS (NRS-V) was perfect (r ¼ 1.0), indicating that individuals reported identical pain scores 5e10 minutes apart, likely because of verbal recall. The testeretest reliability was examined at a short time interval in an effort to capture a constant pain level. In addition, the VAS was designed to reduce the ability of people to recall their scores by having a line devoid of any marking or numbers. Previous studies have reported the VAS to be reliable when pain is repeatedly TABLE 2. TesteRetest Reliability of Baseline (Pregripping) Pain Scores (ICC), As Well As Change Scores (Postgripping Pregripping), and ES Coefficients for the VAS-P, VAS-D, and NRS-V Pain Scale
ICC
CSA
PSS
ES
VAS-P VAS-D NRS-V
0.96 0.96 1.00
0.40 0.48 0.54
1.39 1.52 1.74
0.29 0.32 0.37
ICC ¼ intraclass correlation coefficient; ES ¼ effect size; VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale; CSA ¼ change score average; PSS ¼pregripping score standard deviation. ES ¼ CSA divided by standard deviation of pregripping score ¼ CSA/PSS.
360
JOURNAL OF HAND THERAPY
measured over a time interval as short as few minutes.18,42 Revill et al.42 reported significantly less variance for repeated pain ratings than for repeated rating of a random marks on the straight line, suggesting that reproducible ratings of constant pain are less likely because of recall of the original line markings. An instrument is considered to be valid when Pearson correlation coefficients are high (r . 0.75).41 In the present study, we found that the VAS-D had high correlation coefficients (r ¼ 0.84e0.97) with the VAS-P and NRS-V (Table 3), suggesting strong consistency in pain scores among the three pain scales. These findings indicate that the VAS-D is a valid measure of upper extremity pain level. Consequently, our findings indicate that when used under the same TABLE 3. Validity of Pre- and Postgripping Pain Scores as Indicated by Pearson Product-Moment Correlation Matrix Constructed for the VAS-P, VAS-D, and NRS-V Pain Measurement
VAS-P
VAS-D
NRS-V
Pregripping (second baseline) pain score VAS-P VAS-D NRS-V
1 0.97 0.84
1 0.84
1
Postgripping pain score VAS-P VAS-D NRS-V
1 0.95 0.93
1 0.93
1
VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale.
TABLE 4. Results of a Two-Way ANOVA (Scale 3 Time) Comparing Pre- and Postgripping Pain Scores Using the Three Pain Scales Source Scale Time Scale 3 time
Sum of Squares
DF
Mean Square
F
p-Value
6.02 10.97 0.16
2 1 2
6.02 10.97 0.16
8.06 9.31 1.36
0.008* 0.005* 0.252
ANOVA ¼ analysis of variance; DF ¼ degrees of freedom. Scale ¼ paper version of the visual analog scale vs. digital version of the visual analog scale vs. verbal version of the numerical rating scale and time ¼ baseline vs. last maximal grip. *Indicates significant differences at the p # 0.05 alpha level.
conditions as described in our methodology, clinicians can use the VAS-D with confidence in its reliability and validity. Previous studies examining concurrent validity of the VAS-D report a wide range of correlation coefficients. Stronger associations are found when the physical characteristics of the VASD are more similar to the VAS-P. Cook et al.17 found lower correlation coefficients (r ¼ 0.68) than the present study between an 11-point scale VAS-D and 10cm VAS-P, whereas Swanston et al.1 found similar correlation coefficients (r ¼ 0.81e0.95) to those found in the present study between a 102-point VAS-D and a 101-point VAS-P. In the present study, the similarities in physical characteristics between the VAS-D and VAS-P were as follows: 1) a 101-point scale distributed over a 10-cm horizontal line, 2) a black line on a white background, 3) the same words used for the anchors, and 4) identical (standardized) instructions on how to complete the assessment. There were, however, some differences between our version of the VAS-D and the standard VAS-P. An obvious difference was in how the two scales were marked: the VAS-P was marked by placing a vertical line across the scale’s horizontal line, whereas the VAS-D was marked by moving a slider on the scale’s horizontal line (see
TABLE 5. Results of Post Hoc, Paired Comparison t-Tests between Pre- and Postgripping Pain Intensity Scores for the VAS-P, VAS-D, and NRS-V Paired Comparison Pregripping score (second baseline) VAS-D vs. VAS-P NRS-V vs. VAS-P NRS-V vs. VAS-D Postgripping score VAS-D vs. VAS-P NRS-V vs. VAS-P NRS-V vs. VAS-D
Mean SD 0.28 0.38 0.36 1.04 0.08 1.01
t
p-Value
4.25 0.0002* 1.98 0.0564 0.43 0.6704
0.36 0.64 3.27 0.0026* 0.50 0.81 3.53 0.0013* 0.13 0.75 1.02 0.3163
VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale; SD ¼ standard deviation. *Indicates significant differences at the p , 0.008 alpha level.
Figures 1 and 2). Another difference, of greater consequence, was the accuracy or resolution of the scale. This difference in accuracy stems from differences in thickness of the pain-level indicator (pencil thickness vs. slider thickness). The thickness of the graphite of a standard pencil is 1e2 mm, whereas the thickness of the slider used in our version of the VAS-D was 8 mm. The computer recorded the pain level at the center of the slider, but participants may have designated their pain level according to the location of either end of the slider. As each end was 4 mm away from the center of the slider, that is the possible amount of error in resolution. Therefore, the VAS-P was at least two times more accurate than the VAS-D, with the thickness of the pain-level indicator (pencil) being 1e2% of the 10-cm scale vs. the painlevel indicator of VAS-D (slider) being 4% of the whole scale. Thus, the combination of the screen resolution and slider thickness may have compromised the accuracy of our version of the VAS-D, making it less precise than the VAS-P. The NRS-V has a lower resolution than the VAS, as individuals are more likely to report their pain level using integer numbers (e.g., 1, 2, and 3) between 0 and 10 as opposed to rational numbers (e.g., 3.5, 4.3, and 5.7). Hence, the resolution or accuracy of the NRS-V, with the pain level indicator being an integer number, is at 10% of the whole scale. Because the accuracy of the VAS-D (pain level indicator is 4% of the entire scale) is more similar to that of the VAS-P (pain level indicator is 2% of the entire scale) than the NRS-V (pain level indicator is 10% of the entire scale), it is not surprising that we found the VAS-D to have a stronger association with the VAS-P (r ¼ 0.95e0.97) than with the NRS-V (r ¼ 0.84e0.93, Table 2). Despite the excellent concurrent validity correlation coefficients, the ANOVA revealed significant differences between pain scores on the VAS-D, VASP, and NRS-V (Table 3). Pain scores on the VAS-D were significantly higher compared with the VAS-P and significantly lower compared with the NRS-V (Figure 3). The significant difference in pain scores between the three scales is probably because of the difference in accuracy between the three scales as discussed above. It is interesting to note that the order of scale accuracy corresponds to the order of the magnitude of pain scores on the scales. In other words, the most accurate scale had the lowest pain scores, whereas the least accurate scale had the highest pain score (Figure 3). These findings suggest that the NRS-V, VAS-P, and VAS-D, although consistent, are not in total agreement, which may indicate limitations in the validity of our version of the VAS-D. Yet, the statistically significant differences between the three scales may not be clinically significant because the actual differences among the scales are at most 0.3 points (Figure 3), which is a smaller magnitude than a OctobereDecember 2011 361
reported pain level on the NRS-V (an integer number). Finding significant differences between two scales simultaneously with excellent concurrent validity indicates disagreement in statistical methodology; for example, different results are derived from correlational statistics than from ANOVA. Similar disagreement has been observed in other studies investigating concurrent validity as a result of differences in accuracy between the instruments being compared. For example, Shechtman et al.43 examined the reliability and validity of the DynEx dynamometer (MD Systems, Inc., Westerville, OH, USA) as compared with the Jamar dynamometer and found significant differences (p ¼ 0.014) yet high concurrent validity (r ¼ 0.986e0.989) between the two instruments. The authors stated that the significant differences in grip strength were because of a greater measurement accuracy of the digital DynEx dynamometer.43 Similarly, in the present study, the differences in pain scores among the three scales are likely because of differences in the accuracy of these scales. The VAS-D is more accurate than the NRS-V (in which integer numbers are used) but less accurate than the VAS-P because of the digital technology limitations (an insufficient screen resolution combined with a relatively large-sized digital slider compromising accuracy). Therefore, the VAS-D, VAS-P, and NRS-V may be used interchangeably, but with caution, as small but significant differences in pain scores exist between the three scales. Responsiveness is the ability of an assessment to detect change over time or after an intervention.41 A significant increase in pain levels was found from pre- to postgripping (from baseline to after performing four maximal grips). This increase indicates that the VAS-P, VAS-D, and NRS-V are sensitive to changes in pain level brought about by gripping. The change recorded by the three pain assessments was similar as evidenced by the indices of responsiveness. The ES coefficients indicated a small increase in pain from pre- to postgripping (Table 2). The ANOVA on change scores revealed that the increase in pain scores, which ranged from 0.40 to 0.54 points, was not significantly different among the three scales (Table 2). Therefore, the VAS-P, VAS-D, and NRS-V were equally responsive when measuring change in pain intensity associated with performing maximal grip efforts in people with upper extremity injuries. The limitations of the present study involve our participant sample and study design. First, generalizability of our findings may be limited by the lack of sample homogeneity (i.e., participants had different injuries, etiologies, and affected areas of the forearm and hand) and because of the gender differences in distribution of acute vs. cumulative trauma injuries (see Table 1). Also, recall of the original line markings 362
JOURNAL OF HAND THERAPY
may have inflated the testeretest reliability coefficients of the VAS-D because of a relatively short time interval between repeated ratings. To eliminate the effect of this short time interval, we asked participants to mark a straight line without any other visuals, such as numbers or verbal descriptors spelled out. Despite all our efforts, some recall may still be present. In addition, the increase in pain scores was triggered by performing four maximal grip repetitions, which elicited only a small change in pain level. A stimulus sufficient in eliciting greater changes in pain scores (greater ES) would have probably differentiated better the degree of responsiveness among the three scales. Finally, the accuracy of our VAS-D was compromised by the digital technology, that is, the computer monitor display, which had a low resolution relative to the large-sized pain-level indicator (slider). Using the same resolution of the pain-level indicator for the VAS-D and VAS-P may have eliminated the significant differences in pain scores between the two scales. We recommend that future studies use a homogeneous patient population, perform more than four maximal grip repetitions, and use a VAS-D scale, which possesses the same accuracy as the VAS-P scale.
CONCLUSION The clinical importance of this study stems from the need to ensure that electronic/digital assessments used in rehabilitation centers and hand therapy clinics are reliable, valid, and responsive. We found our version of the VAS-D to be reliable, valid, and responsive for measuring upper extremity pain intensity. However, we also found small yet statistically significant differences between pain scores on the VAS-D, VAS-P, and NRS-V. Although the VAS-D may be used interchangeably with the other two scales, therapists need to be aware that pain scores on the VAS-D may be slightly different than those on the VAS-P and NRS-V. Acknowledgments This work was supported in part by the Evelyn Mackin Research Grant of the American Society of Hand Therapists. Partial equipment support was provided by Thought Technology Ltd., Montreal, Quebec, Canada.
REFERENCES 1. Swanston M, Abraham C, Macrae WA, et al. Pain assessment with interactive computer animation. Pain. 1993;53: 347–51. 2. Jamison RN, Gracely RH, Raymond SA, et al. Comparative study of electronic vs. paper VAS ratings: a randomized, crossover trial using healthy volunteers. Pain. 2002;99:341–7. 3. Jamison RN, Raymond SA, Slawsby EA, McHugo GJ, Baird JC. Pain assessment in patients with low back pain: comparison of
4.
5.
6.
7. 8.
9.
10.
11. 12. 13. 14.
15.
16.
17.
18.
19.
20.
21. 22.
23.
weekly recall and momentary electronic data. J Pain. 2006;7: 192–9. Stewart WF, Ricci JA, Chee E, Hahn SR, Morganstein D. Cost of lost productive work time among US workers with depression. JAMA. 2003;289:3135–44. Rekola KE, Keinanen-Kiukaanniemi S, Takala J. Use of primary health services in sparsely populated country districts by patients with musculoskeletal symptoms: consultations with a physician. J Epidemiol Community Health. 1993;47: 153–7. Mantyselka P, Kumpusalo E, Ahonen R, et al. Pain as a reason to visit the doctor: a study in Finnish primary health care. Pain. 2001;89:175–80. Hasselstrom J, Liu-Palmgren J, Rasjo-Wraak G. Prevalence of pain in general practice. Eur J Pain. 2002;6:375–85. MacIver H, Smyth G, Bird HA. Occupational disorders: nonspecific forearm pain. Best Pract Res Clin Rheumatol. 2007; 21:349–65. Urwin M, Symmons D, Allison T, et al. Estimating the burden of musculoskeletal disorders in the community: the comparative prevalence of symptoms at different anatomical sites, and the relation to social deprivation. Ann Rheum Dis. 1998; 57:649–55. Cunningham LS, Kelsey JL. Epidemiology of musculoskeletal impairments and associated disability. Am J Public Health. 1984;74:574–9. Chapman R. New JCAHO Standards for Pain Management. Glenview, IL; Carpe Diem! APS Bulletin, 2000. Stephenson J. Veterans’ pain a vital sign. JAMA. 1999;281:978. Salcido RS. Is pain a vital sign? Adv Skin Wound Care. 2003;16: 214. Gordon DB, Dahl JL, Miaskowski C, et al. American pain society recommendations for improving the quality of acute and cancer pain management: American Pain Society Quality of Care Task Force. Arch Intern Med. 2005;165:1574–80. Jensen MP, Karoly P. Self-report scales and procedures for assessing pain in adults. In: Turk DC, Melzack R (eds). Handbook of Pain Assessment. 2nd ed. New York, NY: The Guilford Press, 2001. pp. 15–34. Butler PV. Linear analogue self-assessment and procrustean measurement: a critical review of visual analogue scaling in pain assessment. J Clin Psychol Med Settings. 1997;4: 111–29. Cook AJ, Roberts DA, Henderson MD, Van Winkle LC Chastain DC, Hamill-Ruth RJ. Electronic pain questionnaires: a randomized, crossover comparison with paper questionnaires for chronic pain assessment. Pain. 2004;110:310–7. Seymour RA. The use of pain scales in assessing the efficacy of analgesics in post-operative dental pain. Eur J Clin Pharmacol. 1982;23:441–4. Jensen MP, Karoly P, Braver S. The measurement of clinical pain intensity: a comparison of six methods. Pain. 1986;27: 117–26. Kremer E, Atkinson JH, Ignelzi RJ. Measurement of pain: patient preference does not confound pain measurement. Pain. 1981;10:241–8. Gramling SE, Elliott TR. Efficient pain assessment in clinical settings. Behav Res Ther. 1992;30:71–3. Teske K, Daut RL, Cleeland CS. Relationships between nurses’ observations and patients’ self-reports of pain. Pain. 1983;16: 289–96. Ahles TA, Ruckdeschel JC, Blanchard EB. Cancer-related pain—II. Assessment with visual analogue scales. J Psychosom Res. 1984;28:121–4.
24. Gatson-Johansson F. Measurement of pain: the psychometric properties of the Pain-O-Meter, a simple, inexpensive pain assessment tool that could change health care practices. J Pain Symptom Manage. 1996;12:172–81. 25. Breivik EK, Bjornsson GA, Skovlund E. A comparison of pain rating scales by sampling from clinical trial data. Clin J Pain. 2000;16:22–8. 26. Bolton JE, Wilkinson RC. Responsiveness of pain scales: a comparison of three pain intensity measures in chiropractic patients. J Manipulative Physiol Ther. 1998;21:1–7. 27. Lundeberg T, Lund I, Dahlin L, et al. Reliability and responsiveness of three different pain assessments. J Rehabil Med. 2001;33:279–83. 28. McCormack HM, Horne DJ, Sheather S. Clinical applications of visual analogue scales: a critical review. Psychol Med. 1988;18: 1007–19. 29. Junker U, Freynhagen R, Langler K, et al. Paper versus electronic rating scales for pain assessment: a prospective, randomised, cross-over validation study with 200 chronic pain patients. Curr Med Res Opin. 2008;24:1797–806. 30. Gerson GR, Gerson JN. Assessment of pain using a computer. Anaesthesia. 1980;35:815–7. 31. Stratton RJ, Stubbs RJ, Hughes D, King N, Blundell JE, Elia M. Comparison of the traditional paper visual analogue scale questionnaire with an Apple Newton electronic appetite rating system (EARS) in free living subjects feeding ad libitum. Eur J Clin Nutr. 1998;52:737–41. 32. Lehman LA, Sindhu BS, Shechtman O, Romero S, Velozo CA. A comparison of the ability of two upper extremity assessments to measure change in function. J Hand Ther. 2010;23: 31–9, quiz 40. 33. Stratford PW, Binkley FM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–23. 34. Lehman LA, Velozo CA. Ability to detect change in patient function: responsiveness designs and methods of calculation. J Hand Ther. 2010;23:361–70. 35. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27:S178–89. 36. Scrimshaw SV, Maher C. Responsiveness of visual analogue and McGill pain scale measures. J Manipulative Physiol Ther. 2001;24:501–4. 37. Gloth FM 3rd, Scheve AA, Stober CV, Chow S, Prosser J. The Functional Pain Scale: reliability, validity, and responsiveness in an elderly population. J Am Med Dir Assoc. 2001;2: 110–4. 38. Melzack R, Katz J. The McGill pain questionnaire: appraisal and current status. In: Turk DC, Melzack R (eds). Handbook of Pain Assessment. 2nd ed. New York, NY: The Guilford Press, 2001. pp. 35–52. 39. Slider Web FX. Slider. 1.02 ed. Gothenburg, Sweden: Slider WebFX, 2006. Available at: http://webfx.eae.net/dhtml/ slider/slider.html. 40. Fess EE. Grip strength. In: Casanova JS (ed). Clinical Assessment Recommendations. 2nd ed. Chicago, IL: The American Society of Hand Therapists, 1992. pp. 41–5. 41. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Norwalk, CT: Appleton & Lange, 2000. 42. Revill SI, Robinson JO, Rosen M, Hogg MI. The reliability of a linear analogue for evaluating pain. Anaesthesia. 1976;31: 1191–8. 43. Shechtman O, Gestewitz L, Kimble C. Reliability and validity of the DynEx dynamometer. J Hand Ther. 2005;18:339–47.
OctobereDecember 2011 363
JHT Read for Credit Quiz: Article #206
Record your answers on the Return Answer Form found on the tear-out coupon at the back of this issue or to complete online and use a credit card, go to JHTReadforCredit.com. There is only one best answer for each question. #1. Which of the following is NOT TRUE regarding responsiveness? a. responsiveness of an assessment means its ability to detect clinically meaningful change over time b. a common statistical method used to calculate responsiveness is effect size (ES) c. similar effect size (ES) coefficients between two pain scales indicate no difference in the ability of these scales to measure change in pain d. smaller effect size (ES) coefficients indicate greater responsiveness #2. In this study, test-retest reliability was determined by calculating a. Pearson moment correlation coefficients b. ANOVA c. Intraclass correlation coefficients (ICCs) d. Kappas
364
JOURNAL OF HAND THERAPY
#3. A key result was that the digital VAS model was a. responsive, reliable, and valid b. responsive but not valid because it was not reliable c. reliable and valid but not responsive d. too difficult to administer to render it clinically useful #4. Pain was assessed a. while a noxious stimulus (pin prick) was applied b. pre and post gripping c. at the conclusion of treatment d. at the beginning of treatment #5. The authors concluded that the 3 pain scales that were compared were essentially all very compatible a. true b. false When submitting to the HTCC for re-certification, please batch your JHT RFC certificates in groups of 3 or more to get full credit.
FOR
CREDIT ARTICLE #206.
Validity, Reliability, and Responsiveness of a Digital Version of the Visual Analog Scale Bhagwant S. Sindhu, PhD, OTR Department of Occupational Science and Technology, University of Wisconsin-Milwaukee, Milwaukee, Wisconsin
Orit Shechtman, PhD, OTR/L Laura Tuckey, MS, OTR/L Department of Occupational Therapy, University of Florida, Gainesville, Florida
Musculoskeletal pain puts a substantial burden on society because of loss of productivity4 and increased health care costs.5e7 Pain greatly reduces work capacity,8 is higher in women than in men, and increases with age.9,10 In particular, upper extremity musculoskeletal pain has a prevalence of 3e7% among U.S. adults.10 Various health agencies and regulatory bodies have mandated that pain level be assessed on contact with the health care providers.11e14 Pain intensity, the severity of perceived pain, is commonly measured by two assessments: the numerical rating scale (NRS) and visual analog scale (VAS).15 The NRS consists of a range of numbers, with the smaller numbers indicating less pain. An 11-point NRS rates pain intensity corresponding to an integer number between 0 and 10. People rate their pain by indicating a number, either verbally (NRS-V) or by marking a paper (NRS-P), that lists the numbers horizontally in ascending order.15 The VAS usually Correspondence and reprint requests to Bhagwant S. Sindhu, PhD, OTR, Department of Occupational Science and Technology, University of Wisconsin-Milwaukee, Milwaukee, WI 53201; e-mail:
356
JOURNAL OF HAND THERAPY
ABSTRACT: The design used in this study was a prospective cohort. Pain intensity levels recorded by the digital version of the visual analog scale (VAS-D) are easy to both score and share with other health care professionals. The purpose of the study was to examine the testeretest reliability, concurrent validity, and responsiveness of the VAS-D. Thirty-three people with upper extremity injuries reported pain intensity levels before and after performing four maximal grip contractions (pre- and postgripping). Our version of the VAS-D had high testeretest reliability (r ¼ 0.96) and good concurrent validity (r ¼ 0.84e0.97) with both the paper version of the VAS (VAS-P) and the verbal numerical rating scale (NRS-V). Responsiveness of the VAS-D was indicated by a significant increase in pain levels from pre- to postgripping. Similar responsiveness to that of the VAS-P and NRS-V was indicated by similar effect size coefficients and analysis of variance of pain change scores. In conclusion, the VAS-D is a reliable, valid, and responsive measure of pain intensity for people with upper extremity injuries. However, differences in accuracy (resolution) among the VAS-D, VAS-P, or NRS-V may render the three pain scales not fully compatible. Level of Evidence: Not applicable. J HAND THER. 2011;24:356–64.
consists of a 10-centimeter (cm) line placed horizontally on a piece of paper, with an anchor point on each end.16 One anchor of this line represents ‘‘no pain,’’ and the other anchor represents ‘‘maximum perceived pain intensity.’’15 People rate their perceived pain intensity by marking a vertical line across the horizontal scale line.16 The distance from the ‘‘no pain’’ anchor to this mark is measured by a ruler and used as the overall pain intensity score.15,16 This distance is measured in millimeters so that scores range from 0 to 100,16 forming a 101-point VAS, such as the one in the Short Form of the McGill Pain Questionnaire.17 Both NRS and VAS have wellestablished psychometric properties; they are valid,15,18e23 reliable,24 and equally sensitive to changes in pain intensity.25e27 Both measures are nonintrusive, easy to administer, suitable for repeated use, and require simple instructions; the NRS is easier to score.15,16,28 Many versions of the VAS have been used, including various paper and digital versions.1e3,17,29,30 The advantages of the digital versions of VAS (VAS-D) include easier scoring and record keeping.1 The VAS-P is scored using a ruler, which can be time consuming and has potential for human error. In contrast, the VAS-D is scored automatically and
immediately by a computer, without the potential for human error.29e31 In addition, the VAS-D generates and stores electronic patient data, which are portable, easy to share, instantly accessible, and time stamped.2,3 Despite these advantages, limited research has been conducted to determine the psychometric properties (reliability, validity, and responsiveness) of the VAS-D. Psychometric properties of various versions of the VAS-D have been studied. The VAS-D displayed on a personal digital assistant was found to be valid when compared with both an 11-point NRS-P3,29 and a 10-cm VAS-P despite differences in scale line length.2,3,29 The VAS-D displayed on a computer monitor was found to be valid when compared with a 10-cm VAS-P.1,17 However, the monitor display could be rated on either a 102-point scale1 or an 11-point scale instead of the standard 101-point VAS.17 Overall, the various versions of the VAS-D appear to be valid for use in general clinical populations. However, when reviewing the literature, we did not find studies reporting on the responsiveness (e.g., the ability of measure change in pain) of the VAS-D. Pain assessments must be responsive, that is, able to detect clinically meaningful changes over time32 so that therapists can examine the effect of their treatment on pain. When pain is assessed both before and after an intervention session, responsiveness is calculated as the change in pain level from pre- to post-therapy or ‘‘change scores.’’ A common statistical method used to calculate responsiveness is effect size (ES).33,34 ES is measured as the standardized difference between two means and thus evaluates the magnitude of differences or the extent of a treatment effect. ES is calculated by dividing the average change scores (in the nominator) by standard deviation (SD) of prescore (in the denominator).35 ES coefficients are generally interpreted on the basis of Cohen’s conventional criteria, in which 0.20 (or 20% of one SD) indicates a small effect, 0.50 (or 50% of one SD) indicates a moderate effect, and 0.8 (or 80% of one SD) indicates a large effect. Thus, a larger Cohen’s ES indicates a greater difference between pre- and post-therapy pain scores or a greater responsiveness. Similar responsiveness between two pain scales indicates no difference in the ability of these scales to measure change in pain. Similar responsiveness is implied when the ES of the two scales is similar; for example, when two assessments have a large ES.34 The responsiveness of the VAS-P was found to be similar to the NRS26,27 and greater than the McGill Pain Questionnaire,36,37 which measures the multidimensional nature of pain.38 However, it is unknown if the VAS-P and various versions of the VAS-D have similar responsiveness. The purpose of the present study was to examine the psychometric properties of the VAS-D, including testeretest reliability, validity, and responsiveness.
METHODS Participants Thirty-three participants (17 males and 16 females) with unilateral musculoskeletal disorders and injuries of the elbow, forearm, or hand were recruited from various hand therapy clinics. The participants were between the ages of 18 and 65, and they were excluded if they verbally reported their pain intensity to be greater than 7 on a scale of 0e10 and were unable to safely perform four maximal grips with their affected extremity as determined by their hand therapists. These exclusion criteria were implemented for maintaining patient safety, allowing us to exclude individuals who were not sufficiently healed from participating in the study. Before completing the study protocol, each participant read and signed an informed consent form approved by the institutional review board.
Materials and Equipment A demographic questionnaire was used to collect information on age, gender, and injury-related variables, such as diagnosis and site of injury. Maximal grip efforts were performed using an electronic Jamar dynamometer (Thought Technology Ltd., Montreal, Quebec, Canada). Three different scales were used to record pain intensity before (pre) and after (post) four maximal grip contractions: the VAS-D, the paper version of the visual analog scale (VAS-P), and the verbal version of the NRS (NRS-V). Both the VAS-P and VAS-D consisted of a 10-cm horizontal line anchored by two extremes of pain, that is, ‘‘no pain’’ (numerical score of 0) and ‘‘pain as bad as it could be’’ (numerical score of 10). The VAS-P consisted of a horizontal 10-cm black line on a white piece of paper, and the pain level was marked by the participant as a vertical line crossing the scale. The pain score was measured to the closest millimeter using a ruler; in other words, the minimum measurable score was 1 mm or a score of 0.1 on a scale of 0e10 (Figure 1). In contrast, the VAS-D, which was a modified version of a computer program developed by Arvidsson,39 was administered on a touch screen (AccuSync LCD; 52V, 15-inch, NEC Display Solutions, Itasca, IL) with a resolution of 1,024 3 768 pixels. Pain level was marked by moving a slider (0.8 cm or 28 pixels long) on a horizontal line (10 cm or 334 pixels long). The pain score was measured by the MediaLab software (Version 2006.2.34; Empirisoft Corp., New York, NY) as the final location of the center of the slider (Figure 2). OctobereDecember 2011 357
FIGURE 1. Picture of the paper version of the visual analog scale that was used for assessing pain. Note: Thickness of pencil graphite ¼ 1e2 mm.
Protocol
Statistical Analysis
Each participant performed a total of four maximal grip contractions with their injured extremity while sitting in the position recommended by the American Society of Hand Therapists.40 Each grip lasted 6 seconds with a 1-minute rest interval between grips. Pain level was recorded with all three pain intensity scales (VAS-P, VAS-D, and NRS-V) both before (pre) and after (post) gripping. Baseline (pregripping) pain level was recorded twice (5e10 minutes apart) to examine testeretest reliability. Postgripping pain level was recorded once, immediately after the fourth maximal grip contraction. To control order effects, half of the participants completed the VAS-P first and the other half completed the VAS-D first. The NRS-V was always completed last to prevent the participants from replicating their verbalized scores on the VAS-P and VAS-D. Participants were instructed to rate their current level of pain using the three different pain intensity scales. Standardized verbal instructions were used throughout the study as follows: for the VAS-P: ‘‘Mark a vertical line between 0 and 10 at a point that indicates your pain level,’’ for the VAS-D: ‘‘Move the slider on the line between 0 and 10 to the point that indicates your pain level,’’ and for the NRS-V: ‘‘State a number between 0 and 10 that indicates your pain level.’’
Testeretest reliability was determined by calculating intraclass correlation coefficients (ICCs) between the two baseline (pregripping) pain scores. Concurrent validity was determined by calculating Pearson product-moment correlation coefficients between pain scores on the VAS-P, VAS-D, and NRS-V for both pregripping pain levels (the second baseline pain scores) and postgripping pain levels. To ensure agreement between correlational and inferential analysis, a repeated-measures analysis of variance (ANOVA) was conducted to compare pre- and postgripping pain levels as recorded by the VAS-P, VAS-D, and NRS-V. The ANOVA consisted of two within-subject variables: scale (VAS-P, VAS-D, NRSV) and time (pregripping vs. postgripping). Post hoc paired samples t-tests were used to discern the differences among the three scales. The Bonferroni correction for the six comparisons set the alpha level at 0.008. All other comparisons were deemed significant at the 0.05 alpha level. Responsiveness, which is the ability of the scale to measure change in pain, was determined by calculating change scores between pre- and postgripping pain levels. Change score (postgripping pregripping) was calculated for each participant on each scale and compared using a repeated-measures ANOVA with one within-subjects variable: scale (VAS-P, VAS-D, NRS-V). Furthermore, the change
FIGURE 2. Picture of the digital version of the visual analog scale that was used for assessing pain. Note: Slider thickness ¼ 8 mm. 358
JOURNAL OF HAND THERAPY
scores were used to calculate the ES coefficients for the VAS-P, VAS-D, and NRS-V. The ES coefficients were obtained by dividing the average change score by the SD of the pregripping pain score.35 Greater ES coefficients indicate larger change in pain intensity from pre- to postgripping. Data analysis was conducted using the SPSS 16.0 software package (SPSS Inc., Chicago, IL).
RESULTS The average age of the study participants was 39 6 12 years, and the most common location of the injury was the hand (39% of all participants); other demographic variables are reported in Table 1. Preand postgripping average pain intensity levels as measured by the VAS-P, VAS-D, and NRS-V are reported in Figure 3. Testeretest reliability of the VAS-D, as measured by ICC between the two pregripping (baseline) pain scores, was r ¼ 0.96 (Table 2). Concurrent validity of the VAS-D, as measured by Pearson product-moment correlation coefficients between pain scores on the VAS-P, VAS-D, and NRS-V ranged from r ¼ 0.84 to 0.97 (Table 3). The ANOVA (Table 4) comparing pain scores on the three scales during pre- and postgripping revealed significant main effects of both within-subject variables, time
(F ¼ 9.31; p # 0.005) and scale (F ¼ 8.02; p # 0.008). For the main effect of time, postgripping pain scores were significantly greater than pregripping pain scores. For the main effect of scale, the significant difference in pain scores among the three scales (VAS-P, VAS-D, and NRS-V) was further investigated using post hoc paired samples t-tests. Pain scores were significantly greater for the VAS-D than the VAS-P both at pregripping (t ¼ 4.24; p , 0.001) and postgripping (t ¼ 3.27; p # 0.003); also, pain scores were significantly smaller for the VAS-P than the NRS-V at postgripping (t ¼ 3.53; p # 0.001). All other differences were not significant (Table 5). Responsiveness coefficients revealed a small ES (0.20e0.50) for all three scales, the NRS-V, VAS-D, and VAS-P (Table 3). The ANOVA on change scores revealed no significant differences in responsiveness between the three scales (F ¼ 1.36; p # 0.25).
DISCUSSION The purpose of this study was to examine the psychometric properties, that is, reliability, validity, and responsiveness, of a VAS-D and compare them to two other pain assessment scales, namely the VAS-P and NRS-V. When an assessment is used for clinical decision making, acceptable reliability is indicated by
TABLE 1. Demographic and Injury-related Characteristics of Study Participants Demographic and Injury-related Characteristics
Men (N ¼ 17)
Women (N ¼ 16)
All (N ¼ 33)
Mean (SD) or Number (%)
Mean (SD) or Number (%)
Mean (SD) or Number (%)
Age (yr)
37 (11)
Grip strength (lbs)
68 (34.03)
Current work status Full-time Part-time Not working
4 (23.5) 2 (11.8) 11 (64.7)
8 (50) 5 (31.25) 3 (18.75)
12 (36.4) 8 (21.2) 13 (42.4)
Dominant extremity Left Right
4 (23.5) 13 (76.5)
3 (18.75) 13 (81.25)
7 (21.2) 26 (78.8)
Injured extremity Left Right
10 (58.8) 7 (41.2)
6 (37.5) 10 (62.5)
16 (48.5) 17 (51.5)
Location of injury Hand Wrist Forearm Elbow
8 4 1 4
Duration of injury (mo)
7 (14.6)
(47.06) (23.53) (5.88) (23.53)
39 (13.7) 47.26 (23.62)
5 6 3 2
(31.25) (37.5) (18.75) (12.5)
31 (58.1)
39 (12.3) 58.13 (30.91)
13 10 4 6
(39.4) (30.3) (12.1) (18.2)
19 (42.9)
Cause of injury Traumatic Cumulative trauma Do not know
16 (94.1) 1 (5.9) —
6 (37.5) 9 (56.25) 1 (6.25)
22 (66.7) 10 (30.3) 1 (3)
Medical management Currently taking pain medications Undergone surgical intervention
3 (17.64) 10 (58.8)
5 (31.25) 9 (56.25)
8 (24.24) 19 (57.6)
SD ¼ standard deviation.
OctobereDecember 2011 359
2.5 First pre-gripping Second pre-gripping Post-gripping
Pain Intensity Score
2.0
1.5
1.0
0.5
0.0 VAS-P
VAS-D
NRS-V
FIGURE 3. Averages and standard errors of pre- and postgripping pain scores using the VAS-P, VAS-D, and NRS-V. VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale. correlation coefficients exceeding r ¼ 0.9.41 In the present study, we found the testeretest reliability of two consecutive pregripping pain scores on the VAS-D to be excellent (r ¼ 0.96) and identical to that of the VAS-P (Table 2). This high correlation coefficient indicates that participants identified almost the same location on a horizontal line of the VAS-P and VAS-D when reporting the level of pain. In contrast, testeretest reliability of the verbal form of NRS (NRS-V) was perfect (r ¼ 1.0), indicating that individuals reported identical pain scores 5e10 minutes apart, likely because of verbal recall. The testeretest reliability was examined at a short time interval in an effort to capture a constant pain level. In addition, the VAS was designed to reduce the ability of people to recall their scores by having a line devoid of any marking or numbers. Previous studies have reported the VAS to be reliable when pain is repeatedly TABLE 2. TesteRetest Reliability of Baseline (Pregripping) Pain Scores (ICC), As Well As Change Scores (Postgripping Pregripping), and ES Coefficients for the VAS-P, VAS-D, and NRS-V Pain Scale
ICC
CSA
PSS
ES
VAS-P VAS-D NRS-V
0.96 0.96 1.00
0.40 0.48 0.54
1.39 1.52 1.74
0.29 0.32 0.37
ICC ¼ intraclass correlation coefficient; ES ¼ effect size; VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale; CSA ¼ change score average; PSS ¼pregripping score standard deviation. ES ¼ CSA divided by standard deviation of pregripping score ¼ CSA/PSS.
360
JOURNAL OF HAND THERAPY
measured over a time interval as short as few minutes.18,42 Revill et al.42 reported significantly less variance for repeated pain ratings than for repeated rating of a random marks on the straight line, suggesting that reproducible ratings of constant pain are less likely because of recall of the original line markings. An instrument is considered to be valid when Pearson correlation coefficients are high (r . 0.75).41 In the present study, we found that the VAS-D had high correlation coefficients (r ¼ 0.84e0.97) with the VAS-P and NRS-V (Table 3), suggesting strong consistency in pain scores among the three pain scales. These findings indicate that the VAS-D is a valid measure of upper extremity pain level. Consequently, our findings indicate that when used under the same TABLE 3. Validity of Pre- and Postgripping Pain Scores as Indicated by Pearson Product-Moment Correlation Matrix Constructed for the VAS-P, VAS-D, and NRS-V Pain Measurement
VAS-P
VAS-D
NRS-V
Pregripping (second baseline) pain score VAS-P VAS-D NRS-V
1 0.97 0.84
1 0.84
1
Postgripping pain score VAS-P VAS-D NRS-V
1 0.95 0.93
1 0.93
1
VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale.
TABLE 4. Results of a Two-Way ANOVA (Scale 3 Time) Comparing Pre- and Postgripping Pain Scores Using the Three Pain Scales Source Scale Time Scale 3 time
Sum of Squares
DF
Mean Square
F
p-Value
6.02 10.97 0.16
2 1 2
6.02 10.97 0.16
8.06 9.31 1.36
0.008* 0.005* 0.252
ANOVA ¼ analysis of variance; DF ¼ degrees of freedom. Scale ¼ paper version of the visual analog scale vs. digital version of the visual analog scale vs. verbal version of the numerical rating scale and time ¼ baseline vs. last maximal grip. *Indicates significant differences at the p # 0.05 alpha level.
conditions as described in our methodology, clinicians can use the VAS-D with confidence in its reliability and validity. Previous studies examining concurrent validity of the VAS-D report a wide range of correlation coefficients. Stronger associations are found when the physical characteristics of the VASD are more similar to the VAS-P. Cook et al.17 found lower correlation coefficients (r ¼ 0.68) than the present study between an 11-point scale VAS-D and 10cm VAS-P, whereas Swanston et al.1 found similar correlation coefficients (r ¼ 0.81e0.95) to those found in the present study between a 102-point VAS-D and a 101-point VAS-P. In the present study, the similarities in physical characteristics between the VAS-D and VAS-P were as follows: 1) a 101-point scale distributed over a 10-cm horizontal line, 2) a black line on a white background, 3) the same words used for the anchors, and 4) identical (standardized) instructions on how to complete the assessment. There were, however, some differences between our version of the VAS-D and the standard VAS-P. An obvious difference was in how the two scales were marked: the VAS-P was marked by placing a vertical line across the scale’s horizontal line, whereas the VAS-D was marked by moving a slider on the scale’s horizontal line (see
TABLE 5. Results of Post Hoc, Paired Comparison t-Tests between Pre- and Postgripping Pain Intensity Scores for the VAS-P, VAS-D, and NRS-V Paired Comparison Pregripping score (second baseline) VAS-D vs. VAS-P NRS-V vs. VAS-P NRS-V vs. VAS-D Postgripping score VAS-D vs. VAS-P NRS-V vs. VAS-P NRS-V vs. VAS-D
Mean SD 0.28 0.38 0.36 1.04 0.08 1.01
t
p-Value
4.25 0.0002* 1.98 0.0564 0.43 0.6704
0.36 0.64 3.27 0.0026* 0.50 0.81 3.53 0.0013* 0.13 0.75 1.02 0.3163
VAS-P ¼ paper version of the visual analog scale; VAS-D ¼ digital version of the visual analog scale; NRS-V ¼ verbal version of the numerical rating scale; SD ¼ standard deviation. *Indicates significant differences at the p , 0.008 alpha level.
Figures 1 and 2). Another difference, of greater consequence, was the accuracy or resolution of the scale. This difference in accuracy stems from differences in thickness of the pain-level indicator (pencil thickness vs. slider thickness). The thickness of the graphite of a standard pencil is 1e2 mm, whereas the thickness of the slider used in our version of the VAS-D was 8 mm. The computer recorded the pain level at the center of the slider, but participants may have designated their pain level according to the location of either end of the slider. As each end was 4 mm away from the center of the slider, that is the possible amount of error in resolution. Therefore, the VAS-P was at least two times more accurate than the VAS-D, with the thickness of the pain-level indicator (pencil) being 1e2% of the 10-cm scale vs. the painlevel indicator of VAS-D (slider) being 4% of the whole scale. Thus, the combination of the screen resolution and slider thickness may have compromised the accuracy of our version of the VAS-D, making it less precise than the VAS-P. The NRS-V has a lower resolution than the VAS, as individuals are more likely to report their pain level using integer numbers (e.g., 1, 2, and 3) between 0 and 10 as opposed to rational numbers (e.g., 3.5, 4.3, and 5.7). Hence, the resolution or accuracy of the NRS-V, with the pain level indicator being an integer number, is at 10% of the whole scale. Because the accuracy of the VAS-D (pain level indicator is 4% of the entire scale) is more similar to that of the VAS-P (pain level indicator is 2% of the entire scale) than the NRS-V (pain level indicator is 10% of the entire scale), it is not surprising that we found the VAS-D to have a stronger association with the VAS-P (r ¼ 0.95e0.97) than with the NRS-V (r ¼ 0.84e0.93, Table 2). Despite the excellent concurrent validity correlation coefficients, the ANOVA revealed significant differences between pain scores on the VAS-D, VASP, and NRS-V (Table 3). Pain scores on the VAS-D were significantly higher compared with the VAS-P and significantly lower compared with the NRS-V (Figure 3). The significant difference in pain scores between the three scales is probably because of the difference in accuracy between the three scales as discussed above. It is interesting to note that the order of scale accuracy corresponds to the order of the magnitude of pain scores on the scales. In other words, the most accurate scale had the lowest pain scores, whereas the least accurate scale had the highest pain score (Figure 3). These findings suggest that the NRS-V, VAS-P, and VAS-D, although consistent, are not in total agreement, which may indicate limitations in the validity of our version of the VAS-D. Yet, the statistically significant differences between the three scales may not be clinically significant because the actual differences among the scales are at most 0.3 points (Figure 3), which is a smaller magnitude than a OctobereDecember 2011 361
reported pain level on the NRS-V (an integer number). Finding significant differences between two scales simultaneously with excellent concurrent validity indicates disagreement in statistical methodology; for example, different results are derived from correlational statistics than from ANOVA. Similar disagreement has been observed in other studies investigating concurrent validity as a result of differences in accuracy between the instruments being compared. For example, Shechtman et al.43 examined the reliability and validity of the DynEx dynamometer (MD Systems, Inc., Westerville, OH, USA) as compared with the Jamar dynamometer and found significant differences (p ¼ 0.014) yet high concurrent validity (r ¼ 0.986e0.989) between the two instruments. The authors stated that the significant differences in grip strength were because of a greater measurement accuracy of the digital DynEx dynamometer.43 Similarly, in the present study, the differences in pain scores among the three scales are likely because of differences in the accuracy of these scales. The VAS-D is more accurate than the NRS-V (in which integer numbers are used) but less accurate than the VAS-P because of the digital technology limitations (an insufficient screen resolution combined with a relatively large-sized digital slider compromising accuracy). Therefore, the VAS-D, VAS-P, and NRS-V may be used interchangeably, but with caution, as small but significant differences in pain scores exist between the three scales. Responsiveness is the ability of an assessment to detect change over time or after an intervention.41 A significant increase in pain levels was found from pre- to postgripping (from baseline to after performing four maximal grips). This increase indicates that the VAS-P, VAS-D, and NRS-V are sensitive to changes in pain level brought about by gripping. The change recorded by the three pain assessments was similar as evidenced by the indices of responsiveness. The ES coefficients indicated a small increase in pain from pre- to postgripping (Table 2). The ANOVA on change scores revealed that the increase in pain scores, which ranged from 0.40 to 0.54 points, was not significantly different among the three scales (Table 2). Therefore, the VAS-P, VAS-D, and NRS-V were equally responsive when measuring change in pain intensity associated with performing maximal grip efforts in people with upper extremity injuries. The limitations of the present study involve our participant sample and study design. First, generalizability of our findings may be limited by the lack of sample homogeneity (i.e., participants had different injuries, etiologies, and affected areas of the forearm and hand) and because of the gender differences in distribution of acute vs. cumulative trauma injuries (see Table 1). Also, recall of the original line markings 362
JOURNAL OF HAND THERAPY
may have inflated the testeretest reliability coefficients of the VAS-D because of a relatively short time interval between repeated ratings. To eliminate the effect of this short time interval, we asked participants to mark a straight line without any other visuals, such as numbers or verbal descriptors spelled out. Despite all our efforts, some recall may still be present. In addition, the increase in pain scores was triggered by performing four maximal grip repetitions, which elicited only a small change in pain level. A stimulus sufficient in eliciting greater changes in pain scores (greater ES) would have probably differentiated better the degree of responsiveness among the three scales. Finally, the accuracy of our VAS-D was compromised by the digital technology, that is, the computer monitor display, which had a low resolution relative to the large-sized pain-level indicator (slider). Using the same resolution of the pain-level indicator for the VAS-D and VAS-P may have eliminated the significant differences in pain scores between the two scales. We recommend that future studies use a homogeneous patient population, perform more than four maximal grip repetitions, and use a VAS-D scale, which possesses the same accuracy as the VAS-P scale.
CONCLUSION The clinical importance of this study stems from the need to ensure that electronic/digital assessments used in rehabilitation centers and hand therapy clinics are reliable, valid, and responsive. We found our version of the VAS-D to be reliable, valid, and responsive for measuring upper extremity pain intensity. However, we also found small yet statistically significant differences between pain scores on the VAS-D, VAS-P, and NRS-V. Although the VAS-D may be used interchangeably with the other two scales, therapists need to be aware that pain scores on the VAS-D may be slightly different than those on the VAS-P and NRS-V. Acknowledgments This work was supported in part by the Evelyn Mackin Research Grant of the American Society of Hand Therapists. Partial equipment support was provided by Thought Technology Ltd., Montreal, Quebec, Canada.
REFERENCES 1. Swanston M, Abraham C, Macrae WA, et al. Pain assessment with interactive computer animation. Pain. 1993;53: 347–51. 2. Jamison RN, Gracely RH, Raymond SA, et al. Comparative study of electronic vs. paper VAS ratings: a randomized, crossover trial using healthy volunteers. Pain. 2002;99:341–7. 3. Jamison RN, Raymond SA, Slawsby EA, McHugo GJ, Baird JC. Pain assessment in patients with low back pain: comparison of
4.
5.
6.
7. 8.
9.
10.
11. 12. 13. 14.
15.
16.
17.
18.
19.
20.
21. 22.
23.
weekly recall and momentary electronic data. J Pain. 2006;7: 192–9. Stewart WF, Ricci JA, Chee E, Hahn SR, Morganstein D. Cost of lost productive work time among US workers with depression. JAMA. 2003;289:3135–44. Rekola KE, Keinanen-Kiukaanniemi S, Takala J. Use of primary health services in sparsely populated country districts by patients with musculoskeletal symptoms: consultations with a physician. J Epidemiol Community Health. 1993;47: 153–7. Mantyselka P, Kumpusalo E, Ahonen R, et al. Pain as a reason to visit the doctor: a study in Finnish primary health care. Pain. 2001;89:175–80. Hasselstrom J, Liu-Palmgren J, Rasjo-Wraak G. Prevalence of pain in general practice. Eur J Pain. 2002;6:375–85. MacIver H, Smyth G, Bird HA. Occupational disorders: nonspecific forearm pain. Best Pract Res Clin Rheumatol. 2007; 21:349–65. Urwin M, Symmons D, Allison T, et al. Estimating the burden of musculoskeletal disorders in the community: the comparative prevalence of symptoms at different anatomical sites, and the relation to social deprivation. Ann Rheum Dis. 1998; 57:649–55. Cunningham LS, Kelsey JL. Epidemiology of musculoskeletal impairments and associated disability. Am J Public Health. 1984;74:574–9. Chapman R. New JCAHO Standards for Pain Management. Glenview, IL; Carpe Diem! APS Bulletin, 2000. Stephenson J. Veterans’ pain a vital sign. JAMA. 1999;281:978. Salcido RS. Is pain a vital sign? Adv Skin Wound Care. 2003;16: 214. Gordon DB, Dahl JL, Miaskowski C, et al. American pain society recommendations for improving the quality of acute and cancer pain management: American Pain Society Quality of Care Task Force. Arch Intern Med. 2005;165:1574–80. Jensen MP, Karoly P. Self-report scales and procedures for assessing pain in adults. In: Turk DC, Melzack R (eds). Handbook of Pain Assessment. 2nd ed. New York, NY: The Guilford Press, 2001. pp. 15–34. Butler PV. Linear analogue self-assessment and procrustean measurement: a critical review of visual analogue scaling in pain assessment. J Clin Psychol Med Settings. 1997;4: 111–29. Cook AJ, Roberts DA, Henderson MD, Van Winkle LC Chastain DC, Hamill-Ruth RJ. Electronic pain questionnaires: a randomized, crossover comparison with paper questionnaires for chronic pain assessment. Pain. 2004;110:310–7. Seymour RA. The use of pain scales in assessing the efficacy of analgesics in post-operative dental pain. Eur J Clin Pharmacol. 1982;23:441–4. Jensen MP, Karoly P, Braver S. The measurement of clinical pain intensity: a comparison of six methods. Pain. 1986;27: 117–26. Kremer E, Atkinson JH, Ignelzi RJ. Measurement of pain: patient preference does not confound pain measurement. Pain. 1981;10:241–8. Gramling SE, Elliott TR. Efficient pain assessment in clinical settings. Behav Res Ther. 1992;30:71–3. Teske K, Daut RL, Cleeland CS. Relationships between nurses’ observations and patients’ self-reports of pain. Pain. 1983;16: 289–96. Ahles TA, Ruckdeschel JC, Blanchard EB. Cancer-related pain—II. Assessment with visual analogue scales. J Psychosom Res. 1984;28:121–4.
24. Gatson-Johansson F. Measurement of pain: the psychometric properties of the Pain-O-Meter, a simple, inexpensive pain assessment tool that could change health care practices. J Pain Symptom Manage. 1996;12:172–81. 25. Breivik EK, Bjornsson GA, Skovlund E. A comparison of pain rating scales by sampling from clinical trial data. Clin J Pain. 2000;16:22–8. 26. Bolton JE, Wilkinson RC. Responsiveness of pain scales: a comparison of three pain intensity measures in chiropractic patients. J Manipulative Physiol Ther. 1998;21:1–7. 27. Lundeberg T, Lund I, Dahlin L, et al. Reliability and responsiveness of three different pain assessments. J Rehabil Med. 2001;33:279–83. 28. McCormack HM, Horne DJ, Sheather S. Clinical applications of visual analogue scales: a critical review. Psychol Med. 1988;18: 1007–19. 29. Junker U, Freynhagen R, Langler K, et al. Paper versus electronic rating scales for pain assessment: a prospective, randomised, cross-over validation study with 200 chronic pain patients. Curr Med Res Opin. 2008;24:1797–806. 30. Gerson GR, Gerson JN. Assessment of pain using a computer. Anaesthesia. 1980;35:815–7. 31. Stratton RJ, Stubbs RJ, Hughes D, King N, Blundell JE, Elia M. Comparison of the traditional paper visual analogue scale questionnaire with an Apple Newton electronic appetite rating system (EARS) in free living subjects feeding ad libitum. Eur J Clin Nutr. 1998;52:737–41. 32. Lehman LA, Sindhu BS, Shechtman O, Romero S, Velozo CA. A comparison of the ability of two upper extremity assessments to measure change in function. J Hand Ther. 2010;23: 31–9, quiz 40. 33. Stratford PW, Binkley FM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–23. 34. Lehman LA, Velozo CA. Ability to detect change in patient function: responsiveness designs and methods of calculation. J Hand Ther. 2010;23:361–70. 35. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27:S178–89. 36. Scrimshaw SV, Maher C. Responsiveness of visual analogue and McGill pain scale measures. J Manipulative Physiol Ther. 2001;24:501–4. 37. Gloth FM 3rd, Scheve AA, Stober CV, Chow S, Prosser J. The Functional Pain Scale: reliability, validity, and responsiveness in an elderly population. J Am Med Dir Assoc. 2001;2: 110–4. 38. Melzack R, Katz J. The McGill pain questionnaire: appraisal and current status. In: Turk DC, Melzack R (eds). Handbook of Pain Assessment. 2nd ed. New York, NY: The Guilford Press, 2001. pp. 35–52. 39. Slider Web FX. Slider. 1.02 ed. Gothenburg, Sweden: Slider WebFX, 2006. Available at: http://webfx.eae.net/dhtml/ slider/slider.html. 40. Fess EE. Grip strength. In: Casanova JS (ed). Clinical Assessment Recommendations. 2nd ed. Chicago, IL: The American Society of Hand Therapists, 1992. pp. 41–5. 41. Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice. 2nd ed. Norwalk, CT: Appleton & Lange, 2000. 42. Revill SI, Robinson JO, Rosen M, Hogg MI. The reliability of a linear analogue for evaluating pain. Anaesthesia. 1976;31: 1191–8. 43. Shechtman O, Gestewitz L, Kimble C. Reliability and validity of the DynEx dynamometer. J Hand Ther. 2005;18:339–47.
OctobereDecember 2011 363
JHT Read for Credit Quiz: Article #206
Record your answers on the Return Answer Form found on the tear-out coupon at the back of this issue or to complete online and use a credit card, go to JHTReadforCredit.com. There is only one best answer for each question. #1. Which of the following is NOT TRUE regarding responsiveness? a. responsiveness of an assessment means its ability to detect clinically meaningful change over time b. a common statistical method used to calculate responsiveness is effect size (ES) c. similar effect size (ES) coefficients between two pain scales indicate no difference in the ability of these scales to measure change in pain d. smaller effect size (ES) coefficients indicate greater responsiveness #2. In this study, test-retest reliability was determined by calculating a. Pearson moment correlation coefficients b. ANOVA c. Intraclass correlation coefficients (ICCs) d. Kappas
364
JOURNAL OF HAND THERAPY
#3. A key result was that the digital VAS model was a. responsive, reliable, and valid b. responsive but not valid because it was not reliable c. reliable and valid but not responsive d. too difficult to administer to render it clinically useful #4. Pain was assessed a. while a noxious stimulus (pin prick) was applied b. pre and post gripping c. at the conclusion of treatment d. at the beginning of treatment #5. The authors concluded that the 3 pain scales that were compared were essentially all very compatible a. true b. false When submitting to the HTCC for re-certification, please batch your JHT RFC certificates in groups of 3 or more to get full credit.