- Email: [email protected]
Responsiveness of the Liverpool Elbow Score in elbow arthroplasty
J Shoulder Elbow Surg (2013) 22, 312-317
www.elsevier.com/locate/ymse
Responsiveness of the Liverpool Elbow Score in elbow arthroplasty Karthik Vishwanathan, MBBS, MS (Orth), DNB (Orth), MRCS, MSc (Orth), MCh (Orth)a, Omid Alizadehkhaiyat, MD, PhDb, Graham J. Kemp, DM, FRCPath, FHEA, CSci, FSBc, Simon P. Frostick, MA, DM, FRCS (Eng), FRCS (Edin)d,* a
Countess of Chester Hospital, Chester, UK Musculoskeletal Science Research Group, Institute of Translational Medicine, University of Liverpool, Liverpool, UK c Department of Musculoskeletal Biology II, Institute of Ageing and Chronic Disease, University of Liverpool, Liverpool, UK d Musculoskeletal Science Research Group, Institute of Translational Medicine, University of Liverpool, Liverpool, UK b
Background: Responsiveness and floor and ceiling effect are important parameters for evaluating the sensitivity of an outcome instrument in detecting the changes in the clinical condition of patients after an intervention as well as evaluating the content validity of the instrument. The aim of this prospective observational study was to assess these parameters for the Liverpool Elbow Score (LES) in total elbow replacement (TER). Methods: The study included 121 cemented TER cases with linked elbow prosthesis (Discovery Elbow, Biomet Orthopaedics, Swindon, UK) for various conditions, including inflammatory arthritis, noninflammatory arthritis, trauma, and loosening. The proportion of patients with the lowest score (0 points; floor effect) and maximum score (10 points; ceiling effect) was checked preoperatively and 1 year postoperatively. Distribution-based methods (effect size [ES], standardized response mean [SRM], Guyatt responsiveness ratio [GRR]) and anchor-based methods (receiver operating characteristic [ROC] curve and Spearman correlation coefficient) were used to assess responsiveness. Patient satisfaction after TER was used as an external anchor. Results: Patients were a mean age of 63 years (range, 20-86 years). Large ES (1.64), SRM (1.25), and GRR (1.69) were found during the follow-up period. Area under the ROC curve was 0.71 (95% confidence interval, 0.56-0.87; P ¼ .03). There was significant positive correlation (Spearman correlation coefficient, 0.35; P ¼ .004) between changes in LES and satisfaction level. LES showed no floor and ceiling effect preoperatively and at 1 year postoperatively. Conclusion: LES is a responsive measure and has no floor and ceiling effect. This encourages its use as an outcome instrument for TER. Level of evidence: Basic Science Study, Development or Validation of Outcome Instruments. Ó 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. Keywords: Liverpool Elbow Score; total elbow replacement; responsiveness; floor effect; ceiling effect; outcome measure
This study was approved by Sefton Research Ethics Committee, REC Number: 08/H1001/109; Trust Study Number: 3735 *Reprint requests: Prof. Simon P. Frostick, Musculoskeletal Science Research Group, Institute of Translational Medicine, Faculty
of Health and Life Sciences, University of Liverpool, Liverpool L69 3GA, UK. E-mail address: [email protected] (S.P. Frostick).
1058-2746/$ - see front matter Ó 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. http://dx.doi.org/10.1016/j.jse.2012.09.003
Responsiveness of the Liverpool Elbow Score The Liverpool Elbow Score (LES) is an elbow-specific outcome score completed by both the clinician and patient. Since the original demonstration that LES is a valid, reliable, and responsive outcome tool in a number of elbow conditions,35 it has been used to assess the outcome of total elbow replacement (TER) in rheumatoid arthritis,2,6 posttraumatic arthritis,1,2 distal humeral fracture,19,33 and olecranon fractures.28 Responsiveness has not been previously described specifically in the context of TER. Responsiveness describes the ability of a tool to detect clinically relevant changes over time after an intervention.41 It can be evaluated using anchor-based or distribution-based methods.30 Evidence of responsiveness is context-specific and applicable only to the studied population and applied intervention.16,40 An outcome tool in a situation where an appreciable fraction, conventionally >15%, of patients gain the highest or lowest possible scores (the so-called ceiling and floor effects, respectively23,27,40,42) will likely have limited content validity and responsiveness.27,40 LES has not been previously evaluated for ceiling and floor effects. There has been no published evaluation of the responsiveness and ceiling and floor effect of LES in TER, and this was the aim of the present study.
Materials and methods Participants The prospective observational study included 121 consecutive adult patients (aged >18 years) who underwent TER using the Discovery Elbow System (Biomet Orthopaedics, Biomet Orthopaedics, Wiltshire, UK) between January 2003 and April 2010 in the Upper Limb Unit, Royal Liverpool and Broadgreen University Hospital Trust. Underlying conditions included rheumatoid arthritis, osteoarthritis (traumatic and nontraumatic), distal humeral fracture, and prosthesis loosening. During this period, the LES was completed as part of routine care in elbow arthroplasty patients.
Outcome assessment Clinical and functional outcome of TER was assessed using LES preoperatively and postoperatively at 3 and 6 months, 1 year, and then annually. Patients completed the patient-answered questionnaire part of LES (PAQ-LES) and were examined clinically by independent clinical fellows (CAS-LES). The 1-year postoperative LES assessment was used to evaluate responsiveness and ceiling/ floor effects (in 1 patient where this was not available, the nearest available score was included). PAQ-LES comprises the domains of pain, functional ability to perform activities of daily living, and functional ability to participate in sporting and recreational activities. CAS-LES assesses range of motion, muscle strength, and ulnar nerve function. LES values span from 0 (worst outcome) to 10 (best outcome). To assess patient satisfaction after TER, a 4-point Likert scale (‘‘very satisfied,’’ ‘‘satisfied,’’ ‘‘somewhat satisfied,’’ and ‘‘unsatisfied’’) was used. To calculate the receiver operating characteristic (ROC) curve and Guyatt responsiveness ratio (GRR; see below), this
313 was converted to a binary scale by combining ‘‘very satisfied’’ and ‘‘satisfied’’ patients as ‘‘definitely satisfied (improved)’’ and ‘‘somewhat satisfied’’ and ‘‘unsatisfied’’ patients as ‘‘not definitely satisfied (not improved).’’
Responsiveness In the absence of a gold standard,9,43 the responsiveness of LES was analyzed using distribution-based methods (effect size [ES], standardized response mean [SRM], and GRR) and anchor-based methods (ROC analysis and correlation coefficient). ES is the ratio of the mean score change to the standard deviation (SD) of the initial score.20 SRM is the ratio of the mean score change to the SD of the score change.22 GRR is the ratio of the mean score change in patients who were ‘‘definitely satisfied (improved)’’ to the SD of the score change in patients who were ‘‘not definitely satisfied (not improved).’’14 Values of ES, SRM and GRR were interpreted using Cohen’s criteria, whereby values of more than 0.8, 0.5, and 0.2 indicate a large, moderate, and small effect, respectively.7,15,31,34 We hypothesized that change in LES would show a moderate but significant correlation to patient satisfaction (‘‘unsatisfied,’’ ‘‘somewhat satisfied,’’ ‘‘satisfied,’’ and ‘‘very satisfied’’). This was tested by calculating the Spearman correlation coefficient, and the P value was used to assess its statistical significance. Further interpretation used Cohen’s criteria, whereby >0.10, 0.30, and 0.50 represent a small, moderate, and large correlation, respectively. For calculating the area under the ROC curve (AUC), patients were grouped as ‘‘definitely satisfied (improved)’’ and ‘‘not definitely satisfied (not improved).’’. ROC plots sensitivity (y-axis) against 1-specificity (x-axis) for all possible cutoff points (ie, the true positive rate against the false positive rate). AUC spans the range of 0.5 (no discrimination better than chance) to 1.0 (perfect discrimination).10,36 The sensitivity, specificity, and 95% confidence interval of the AUC and P values were calculated. In common practice, an outcome measure is considered to be responsive if the GRR >1.9616 or AUC >0.70,16,20,40 or ES or SRM >0.80.29
Floor and ceiling effect Floor and ceiling effects occur when a large fraction of respondents (conventionally, 15%)16,40 achieve the lowest and highest possible score, respectively. These fractions (ie, patients with LES ¼ 0 and LES ¼ 10, respectively) were checked preoperatively and then at 6 months, 1 year, and 2 years postoperatively.
Statistical tests Continuous data are presented as mean (SD) and range. Categoric data are presented as number and percentages. SPSS 17.0 software (SPSS Inc, Chicago, IL, USA) was used for statistical analysis.
Results Patient characteristics The patients were a mean age of 63 years (range, 2086 years); 78 (65%) were women, and 43 (36%) were men.
314 Type of TER was available for 112 of 121 patients: 87 (78%) had primary TER and 25 had revision TER (22%). At presentation, 57 patients (56%) had unilateral involvement and 45 (37%) had bilateral elbow involvement. The dominant elbow was affected in 52 patients (50%) and the nondominant side in 53 (50%). The indications for TER were rheumatoid arthritis in 44, loosening in 25, osteoarthritis in 21, post-traumatic arthritis in 10, fracture of humerus in 7, and hemophiliac arthritis in 3.
LES and patient satisfaction Mean preoperative LES improved from 3.5 (SD, 1.6; range, 0.2-8.3) to 6.4 (SD, 1.8; range, 1.9-9.3). Mean follow-up was 19 months (range, 6 months-7 years). At the postoperative LES evaluation, 2 patients (3%) were not satisfied, 9 (12%) were somewhat satisfied, 34 (44%) were satisfied, and 33 (42%) were very satisfied with the results of TER. Preoperative LES was available in 105 of 121 patients (87%) and the postoperative LES in 85 (70%). For ES estimation, details of LES change were available in 75 patients and preoperative LES in 109. For SRM calculation, data were available in 75 patients. Patient satisfaction data were available in 78 of 121 patients (65%). For GRR, Cohen’s criteria estimation, and ROC analysis, complete data on patient satisfaction and LES change were available in 68 patients.
Responsiveness and floor/ceiling effect The overall ES and SRM were both >0.8 (Table I). Table I and Figure 1 show that as satisfaction level improved, the mean change in LES, ES, and SRM also increased, and vice versa. The mean change in LES in the ‘‘not definitely satisfied’’ and ‘‘definitely satisfied’’ groups was 1.4 (SD, 1.7) and 2.9 (SD, 2.1), respectively. The estimated GRR was 1.69. ES, SRM, and GRR demonstrated large effect size by Cohen’s criteria. There was significant positive correlation (Spearman correlation coefficient, 0.35; P ¼ .004) between LES change and satisfaction grades, which is in line with our hypothesis. The AUC was 0.71 (95% confidence interval, 0.56-0.87), significantly different from 0.5 (P ¼ .03; Figure 2). The sensitivity was 58% and specificity was 82%. AUC was >0.7, indicating that LES is adequately discriminative between satisfied and not definitely satisfied patients. LES showed no significant floor or ceiling effect preoperatively and postoperatively, because none of the patients scored the lowest or highest possible score (Table II).
Discussion This prospective study is the first to evaluate responsiveness and floor/ceiling effects of LES in TER. It gains in robustness by applying anchor-based and distribution-based methods and in external validity, with specific applicability to TER by being based on primary and revision
K. Vishwanathan et al. Table I Mean change in Liverpool Elbow Score, effect size, and standardized response mean in the entire cohort and in groups with different satisfaction levels Satisfaction level
Change in LES
ES
SRM
1.64 0.10 1.44 1.49 1.78
1.25 0.14 0.99 1.28 1.51
Mean (SD) Overall Unsatisfied Somewhat satisfied Satisfied Very satisfied
2.6 0.1 1.7 2.4 3.5
(2.11) (0.35) (1.70) (1.86) (2.29)
ES, effect size; LES, Liverpool Elbow Score; SRM, standardized response mean.
arthroplasties in different underlying conditions. By published criteria,29,40 LES shows satisfactory responsiveness in TER by both kinds of method (ES, SRM, and GRR were >0.8; AUC was >0.70), and it was not affected by ceiling/floor effects. In line with our hypothesis, there is moderate correlation between change in LES and the chosen external anchor (patient satisfaction), and ROC AUC was significantly >0.70. Because the hypothesis was predefined and sample size was >50 patients, this is Level 1 evidence for responsiveness.42 The original validation study described responsiveness of the LES in 18 patients with different elbow conditions 6 months postoperatively after various interventions.35 The study tested the responsiveness by comparing preoperative and postoperative LES and correlating it with change in region-specific (Disabilities of the Arm, Shoulder and Hand [DASH]) and generic outcome measures (Nottingham Health Profile [NHP] and Short Form 12-Item Health Survey [SF-12]), finding significant correlation with DASH (r ¼ 0.45) and NHP (r ¼ 0.42) but nonsignificant correlation with SF-12. Since this original description, new methods have become available for optimal assessment of responsiveness,15 and we therefore wished to readdress this question. Another study described correlation of LES with Mayo Elbow Performance Score (MEPS) in 44 patients with rheumatoid and post-traumatic arthritis who had TER (GSB III prosthesis, Sulzer Medica, Winterthur, Switzerland),2 finding good correlation between the LES and MEPS final follow-up scores (correlation coefficient, 0.84; P < .001). Neither study reported preoperative and postoperative LES. A recent study reported only postoperative LES in 54 rheumatoid arthritis and post-traumatic arthritis patients with TER (GSB III prosthesis).1 The present study is the first to report on both preoperative and postoperative LES in elbow arthroplasty. Ceiling and floor effects have not been previously assessed for any of elbow outcome scores. Compared with spine and shoulder, there are relatively few studies evaluating responsiveness of elbow outcome instruments. LES had higher ES compared with ES of the Oswestry Disability Index (range, 0.37-0.84).4,11,25 SRM and AUC of LES was within the range of values observed
Responsiveness of the Liverpool Elbow Score
Figure 1
Figure 2 analysis.
315
Mean change in Liverpool Elbow Score (LES) and its 95% confidence interval (error bars) in various satisfaction level groups.
Receiver operating characteristic (ROC) curve
for SRM (range, 0.3-1.9)5,18,21,32,37,39 and AUC (range, 0.67-0.92) of the Oswestry Disability Index.3,5,11–13,25,26 The Oswestry Disability Index had higher correlation coefficient (0.69) compared with LES.25 Our study could be compared to the responsiveness study on Oxford Elbow Score (OES) and DASH, consisting of 74 patients,6 in terms of large sample size. Another study on DASH17 had a small sample size of 35 patients, whereas the study comparing Mayo Elbow Performance Scale (MEPS), the Hospital for Special Surgery (HSS) elbow assessment scale (modified version of HSS elbow assessment scale [HSS-2]) and Elbow Functional Assessment (EFA) scale had only 25 patients.8 Our study population was a homogenous group of patients who underwent TER, whereas other studies have consisted of populations
undergoing various elbow surgeries, details of which were not specified,6 and a heterogeneous group consisting of TER, synovectomy, and radial head replacement in patients having rheumatoid arthritis.8,17 LES had higher ES than OES (range, 0.79-1.18 for various subscales)6 and DASH (range, 0.5-0.76).6,17 It was not possible to compare ES of HSS, HSS-2, EFA, and MEPS because the study failed to mention overall ES and instead mentioned ES separately for improved and nonchanged cohorts. SRM for OES has not been described. SRM of LES is higher than SRM of the DASH score (0.6) for elbow disorders.17 It was not possible to compare SRM of HSS, HSS-2, EFA, and MEPS because the study8 failed to mention overall SRM and instead mentioned SRM separately for improved and nonchanged cohorts. GRR in our study was lower than the threshold value of 1.96, suggesting that LES might have limited responsiveness; however; none of the studies on elbow scores reported GRR; hence, it is difficult to compare our result with other elbow scores. Because GRR depends on the external anchor chosen, it is possible that patient satisfaction might not be an ideal external anchor. The correlation coefficient of LES (0.35) was comparable to that of DASH (range, 0.35-0.53)6 and slightly lower than the correlation coefficient of OES (range, 0.370.58 for various subscales using different external anchors).6 Correlation coefficient has not been described for MEPS, HSS, HSS-2, and MEPS.8 The AUC of LES (0.71) was slightly better than the AUC of MEPS (0.70)8 and was comparable with the AUC of DASH (range, 0.71-0.76).6 The AUC of LES was lower than the AUC of OES (range, 0.72-0.90 for various
316
K. Vishwanathan et al.
Table II
The range of values for Liverpool Elbow Score at various evaluation points Evaluation point No.) LES (No.)y Preoperative Postoperative 6 months 1 year 2 years
Floor effect (%)
Ceiling effect (%)
8.3 (1)
0
0
9.4 (1) 9.5 (1) 9.6 (1)
0 0 0
0 0 0
Lower limit
Upper limit
105
0.2 (2)
65 66 44
1.0 (1) 2.5 (1) 1.9 (1)
LES, Liverpool Elbow Score. ) Total number of patients available for follow-up at various times. y Number of patients showing lowest and highest values of LES at various times.
subscales using 2 different external anchors),6 HSS (0.83),8 HSS-2 (0.86),8 and EFA (0.91).8 Comparison with other studies has limited value because the AUC of OES and DASH was estimated using external anchors, such as elbow pain postoperatively and elbow function postoperatively,6 while the AUC of HSS, HSS-2, MEPS, and EFA was calculated using global perceived effect of operative intervention as an external anchor.8 None of the studies on elbow scores used patient satisfaction as external anchor. Of course, the choice of external criteria for change is debatable for any method of responsiveness estimation. Whereas most studies use patient-assessed global assessment of change as an external criterion, we used patient satisfaction. The satisfaction score used is well established, but its validity and reliability are still to be determined. No claim is being made that the LES is a gold standard, and more comparisons of elbow scales against other external anchors may be needed. Patient satisfaction after surgical intervention is influenced by various factors, such as age,38 marital status,7 expectations before and after operative intervention,7,24 and severity of condition preoperatively,24 and hence, patient satisfaction might not be the ideal choice for an external anchor. This could possibly account for the AUC of LES being just above 0.70 and the GRR of LES being below 1.96. Values of measures of responsiveness, such as the AUC, GRR, and correlation coefficient, tend to vary with the choice of external anchor. This was demonstrated in the study by Dawson et al,6 wherein the AUC and correlation coefficient values of OES and DASH varied with use of different external anchors such as elbow pain postoperatively, elbow function postoperatively, and patient satisfaction. We consider the choice of a single question in a Likert scale pattern concerning patient satisfaction as the only external anchor to be a limitation to the design of this prospective study. Another limitation of our study was the use of a single external anchor. Using different external anchors might possibly reveal different values of GRR, the correlation coefficient, and AUC, because the value of these measures tends to depend on the external anchor used. Future studies could evaluate the influence of various
external anchors on values of the AUC, GRR, and correlation coefficient of LES. We do not claim superiority of LES over other elbow outcome tools because we did not perform such a comparison here. The relatively large amount of missing data could threaten the internal validity of the study, but the sample size of >50 patients and the availability of data from 75 patients for SRM calculation and 68 patients for GRR, correlation coefficient, and ROC analysis should have limited the effect on the overall results.
Conclusion Distribution-based and anchor-based methods have shown that LES is a responsive measure and is sensitive to changes in the clinical condition of the patient after TER. LES has no floor and ceiling effect postoperatively, and this encourages use of LES as an outcome evaluation for TER. Hence, the LES is a useful tool to assess the outcome of total elbow arthroplasty.
Disclaimer S.P.F. is a consultant for Biomet Inc, but Biomet had no influence on the methods and results of this study. The authors, their immediate families, and any research foundations with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.
References 1. Amirfeyz R, Blewitt N. Mid-term outcome of GSB-III total elbow arthroplasty in patients with rheumatoid arthritis and patients with post-traumatic arthritis. Arch Orthop Trauma Surg 2009;129:1505-10. http://dx.doi.org/10.1007/s00402-009-0876-y
Responsiveness of the Liverpool Elbow Score 2. Ashmore AM, Gozzard C, Blewitt N. Use of the Liverpool Elbow Score as a postal questionnaire for the assessment of outcome after total elbow arthroplasty. J Shoulder Elbow Surg 2007;16:55-8. http:// dx.doi.org/10.1016/j.jse.2006.08.008 3. Beurskens AJ, de Vet HC, K€oke AJ. Responsiveness of functional status in low back pain: a comparison of different instruments. Pain 1996;65: 71-6. 4. Coelho RA, Siqueira FB, Ferreira PH, Ferreira ML. Responsiveness of the Brazilian-Portuguese version of the Oswestry Disability Index in subjects with low back pain. Eur Spine J 2008;17:1101-6. http://dx. doi.org/10.1007/s00586-008-0690-1 5. Davidson M, Keating JL. A comparison of five low back disability questionnaires: reliability and responsiveness. Phys Ther 2002;82:8-24. 6. Dawson J, Doll H, Boller I, Fitzpatrick R, Little C, Rees J, Carr A. Comparative responsiveness and minimal change for the Oxford Elbow Score following surgery. Qual Life Res 2008;17:1257-67. http://dx.doi.org/10.1007/s11136-008-9409-3 7. Dawson J, Doll H, Boller I, Fitzpatrick R, Little C, Rees J, et al. Factors associated with satisfaction in patients undergoing elbow surgery: a prospective study. J Shoulder Elbow Surg 2010;19:635-44. http://dx.doi.org/10.1016/j.jse.2010.02.003 8. De Boer YA, Hazes JMW, Winia PCA, Brand R, Rozing PM. Comparative responsiveness of four elbow scoring instruments in patients with rheumatoid arthritis. J Rheumatol 2001;28:2616-23. 9. de Vet HC, Terwee CB, Bouter LM. Current challenges in clinimetrics. J Clin Epidemiol 2003;56:1137-41. http://dx.doi.org/10.1016/ j.jclinepi.2003.08.012 10. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Controlled Clin Trial 1991;12:142-58. 11. Farasyn A, Meeusen R. Validity of the new Backache Index (BAI) in patients with low back pain. Spine J 2006;6:565-71. http://dx.doi.org/ 10.1016/j.spinee.2006.01.021 12. Frost H, Lamb SE, Stewart-Brown S. Responsiveness of a patient specific outcome measure compared with the Oswestry Disability Index v2.1 and Roland and Morris Disability Questionnaire for patients with subacute and chronic low back pain. Spine 2008;33: 2450-7. http://dx.doi.org/10.1097/BRS.0b013e31818916fd 13. Grotle M, Brox JI, Vøllestad NK. Concurrent comparison of responsiveness in pain and functional status measurements used for patients with low back pain. Spine 2004;29:492-501. 14. Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. CMAJ 1986;134:889-95. 15. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000;53:459-68. 16. Irrgang JJ, Lubowitz JH. Measuring arthroscopic outcome. Arthroscopy 2008;24:718-22. http://dx.doi.org/10.1016/j.arthro.2007.10.007 17. Ishikawa H, Murasawa A, Nakazono K, Abe A, Otani H, Netsu T, et al. The patient-based outcome of upper-extremity surgeries using the DASH questionnaire and the effect of disease activity of the patients with rheumatoid arthritis. Clin Rheumatol 2008;27:967-73. 18. Jette DU, Jette AM. Physical therapy and health outcomes in patients with spinal impairments. Phys Ther 1996;76:930-41. 19. Kalogrianitis S, Sinopidis C, El Meligy M, Rawal A, Frostick SP. Unlinked elbow arthroplasty as primary treatment for fractures of the distal humerus. J Shoulder Elbow Surg 2008;17:287-92. http://dx.doi. org/10.1016/j.jse.2007.06.011 20. Kazis L, Anderson J, Meenan R. Effect sizes for interpreting changes in health status. Med Care 1989;277(3 Suppl):S178-89. 21. Lauridsen HH, Hartvigsen J, Manniche C, Korsholm L, Grunnet-Nilsson N. Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskelet Disord 2006;7:1-15. http://dx.doi.org/ 10.1186/1471-2474-7-82 22. Liang M, Fossel A, Larson M. Comparison of five health-status instruments for orthopaedic evaluation. Med Care 1990;28:632-42.
317 23. Lohr KN, Aaronson NK, Alonso J, Burnam MA, Patrick DL, Perrin EB, et al. Evaluating quality of life and health status instruments: development of scientific review criteria. Clin Ther 1996;18:979-92. 24. Mancuso CA, Salvati EA, Johanson NA, Peterson MG, Charlson ME. Patients’ expectations and satisfaction with total hip arthroplasty. J Arthroplasty 1997;12:387-96. 25. Mannion AF, Junge A, Grob D, Dvorak J, Fairbank JC. Development of a German version of the Oswestry Disability Index. Part 2: sensitivity to change after spinal surgery. Eur Spine J 2006;15:66-73. http:// dx.doi.org/10.1007/s00586-004-0816-z 26. Maughan EF, Lewis JS. Outcome measures in chronic low back pain. Eur Spine J 2010;19:1484-94. http://dx.doi.org/10.1007/s00586-0101353-6 27. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 1995;4:293-307. 28. Munoz-Mahamud E, Fernandez-Valencia JA, Riba J. Plate osteosynthesis for severe olecranon fractures. J Orthop Surg 2010;18:80-4. 29. Naal FD, Impellizzeri FM, Rippstein PF. Which are the most frequently used outcome instruments in studies on total ankle arthroplasty? Clin Orthop Relat Res 2010;468:815-26. http://dx.doi. org/10.1007/s11999-009-1036-y 30. Norman GR, Gwadry Sridhar F, Guyatt G, Walter SD. Relation of distribution- and anchor based approaches in interpretation of changes in health related quality of life. Med Care 2001;39:1039-47. 31. Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;50:869-79. 32. Patrick DL, Deyo RA, Atlas SJ, Singer DE, Chapin A, Keller RB. Assessing health-related quality of life in patients with sciatica. Spine 1995;20:1899-909. 33. Reising K, Hauschild O, Strohm PC, Suedkamp NP. Stabilisation of articular fractures of the distal humerus: Early experience with a novel perpendicular plate system. Injury 2009;40:611-7. http://dx.doi.org/10. 1016/j.injury.2008.12.018 34. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102-9. http://dx. doi.org/10.1016/j.jclinepi.2007.03.012 35. Sathyamoorthy P, Kemp GJ, Rawal A, Rayner V, Frostick SP. Development and validation of an elbow score. Rheumatology 2004; 43:1434-40. http://dx.doi.org/10.1093/rheumatology/keh367 36. Streiner DL, Cairney J. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatry 2007;52:121-8. 37. Suarez-Almazor ME, Kendall C, Johnson JA, Skeith K, Vincent D. Use of health status measures in patients with low back pain in clinical settings: comparison of specific, generic and preference-based instruments. Rheumatology 2000;39:783-90. http://dx.doi.org/10. 1093/rheumatology/39.7.783 38. Tashjian RZ, Bradley MP, Tocci S, Rey J, Henn RF, Green A. Factors influencing patient satisfaction after rotator cuff repair. J Shoulder Elbow Surg 2007;16:752-8. http://dx.doi.org/10.1016/j.jse.2007.02.136 39. Taylor SJ, Taylor AE, Foy MA, Fogg AJ. Responsiveness of common outcome measures for patients with low back pain. Spine 1999;24:1805-12. 40. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. Clin Epidemiol 2007;60:3442. http://dx.doi.org/10.1016/j.jclinepi.2006.03.012 41. Testa MA, Simonson DC. Assessment of quality-of-life outcomes. N Engl J Med 1996;334:835-40. 42. Van der Leeden M, Steultjens MP, Terwee CB, Rosenbaum D, Turner D, Woodburn J, et al. A systematic review of instruments measuring foot function, foot pain, and foot-related disability in patients with rheumatoid arthritis. Arthritis Rheum 2008;59:1257-69. http://dx.doi.org/10.1002/art.24016 43. Wright J, Young N. A comparison of different indices of responsiveness. J Clin Epidemiol 1997;50:239-46.
www.elsevier.com/locate/ymse
Responsiveness of the Liverpool Elbow Score in elbow arthroplasty Karthik Vishwanathan, MBBS, MS (Orth), DNB (Orth), MRCS, MSc (Orth), MCh (Orth)a, Omid Alizadehkhaiyat, MD, PhDb, Graham J. Kemp, DM, FRCPath, FHEA, CSci, FSBc, Simon P. Frostick, MA, DM, FRCS (Eng), FRCS (Edin)d,* a
Countess of Chester Hospital, Chester, UK Musculoskeletal Science Research Group, Institute of Translational Medicine, University of Liverpool, Liverpool, UK c Department of Musculoskeletal Biology II, Institute of Ageing and Chronic Disease, University of Liverpool, Liverpool, UK d Musculoskeletal Science Research Group, Institute of Translational Medicine, University of Liverpool, Liverpool, UK b
Background: Responsiveness and floor and ceiling effect are important parameters for evaluating the sensitivity of an outcome instrument in detecting the changes in the clinical condition of patients after an intervention as well as evaluating the content validity of the instrument. The aim of this prospective observational study was to assess these parameters for the Liverpool Elbow Score (LES) in total elbow replacement (TER). Methods: The study included 121 cemented TER cases with linked elbow prosthesis (Discovery Elbow, Biomet Orthopaedics, Swindon, UK) for various conditions, including inflammatory arthritis, noninflammatory arthritis, trauma, and loosening. The proportion of patients with the lowest score (0 points; floor effect) and maximum score (10 points; ceiling effect) was checked preoperatively and 1 year postoperatively. Distribution-based methods (effect size [ES], standardized response mean [SRM], Guyatt responsiveness ratio [GRR]) and anchor-based methods (receiver operating characteristic [ROC] curve and Spearman correlation coefficient) were used to assess responsiveness. Patient satisfaction after TER was used as an external anchor. Results: Patients were a mean age of 63 years (range, 20-86 years). Large ES (1.64), SRM (1.25), and GRR (1.69) were found during the follow-up period. Area under the ROC curve was 0.71 (95% confidence interval, 0.56-0.87; P ¼ .03). There was significant positive correlation (Spearman correlation coefficient, 0.35; P ¼ .004) between changes in LES and satisfaction level. LES showed no floor and ceiling effect preoperatively and at 1 year postoperatively. Conclusion: LES is a responsive measure and has no floor and ceiling effect. This encourages its use as an outcome instrument for TER. Level of evidence: Basic Science Study, Development or Validation of Outcome Instruments. Ó 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. Keywords: Liverpool Elbow Score; total elbow replacement; responsiveness; floor effect; ceiling effect; outcome measure
This study was approved by Sefton Research Ethics Committee, REC Number: 08/H1001/109; Trust Study Number: 3735 *Reprint requests: Prof. Simon P. Frostick, Musculoskeletal Science Research Group, Institute of Translational Medicine, Faculty
of Health and Life Sciences, University of Liverpool, Liverpool L69 3GA, UK. E-mail address: [email protected] (S.P. Frostick).
1058-2746/$ - see front matter Ó 2013 Journal of Shoulder and Elbow Surgery Board of Trustees. http://dx.doi.org/10.1016/j.jse.2012.09.003
Responsiveness of the Liverpool Elbow Score The Liverpool Elbow Score (LES) is an elbow-specific outcome score completed by both the clinician and patient. Since the original demonstration that LES is a valid, reliable, and responsive outcome tool in a number of elbow conditions,35 it has been used to assess the outcome of total elbow replacement (TER) in rheumatoid arthritis,2,6 posttraumatic arthritis,1,2 distal humeral fracture,19,33 and olecranon fractures.28 Responsiveness has not been previously described specifically in the context of TER. Responsiveness describes the ability of a tool to detect clinically relevant changes over time after an intervention.41 It can be evaluated using anchor-based or distribution-based methods.30 Evidence of responsiveness is context-specific and applicable only to the studied population and applied intervention.16,40 An outcome tool in a situation where an appreciable fraction, conventionally >15%, of patients gain the highest or lowest possible scores (the so-called ceiling and floor effects, respectively23,27,40,42) will likely have limited content validity and responsiveness.27,40 LES has not been previously evaluated for ceiling and floor effects. There has been no published evaluation of the responsiveness and ceiling and floor effect of LES in TER, and this was the aim of the present study.
Materials and methods Participants The prospective observational study included 121 consecutive adult patients (aged >18 years) who underwent TER using the Discovery Elbow System (Biomet Orthopaedics, Biomet Orthopaedics, Wiltshire, UK) between January 2003 and April 2010 in the Upper Limb Unit, Royal Liverpool and Broadgreen University Hospital Trust. Underlying conditions included rheumatoid arthritis, osteoarthritis (traumatic and nontraumatic), distal humeral fracture, and prosthesis loosening. During this period, the LES was completed as part of routine care in elbow arthroplasty patients.
Outcome assessment Clinical and functional outcome of TER was assessed using LES preoperatively and postoperatively at 3 and 6 months, 1 year, and then annually. Patients completed the patient-answered questionnaire part of LES (PAQ-LES) and were examined clinically by independent clinical fellows (CAS-LES). The 1-year postoperative LES assessment was used to evaluate responsiveness and ceiling/ floor effects (in 1 patient where this was not available, the nearest available score was included). PAQ-LES comprises the domains of pain, functional ability to perform activities of daily living, and functional ability to participate in sporting and recreational activities. CAS-LES assesses range of motion, muscle strength, and ulnar nerve function. LES values span from 0 (worst outcome) to 10 (best outcome). To assess patient satisfaction after TER, a 4-point Likert scale (‘‘very satisfied,’’ ‘‘satisfied,’’ ‘‘somewhat satisfied,’’ and ‘‘unsatisfied’’) was used. To calculate the receiver operating characteristic (ROC) curve and Guyatt responsiveness ratio (GRR; see below), this
313 was converted to a binary scale by combining ‘‘very satisfied’’ and ‘‘satisfied’’ patients as ‘‘definitely satisfied (improved)’’ and ‘‘somewhat satisfied’’ and ‘‘unsatisfied’’ patients as ‘‘not definitely satisfied (not improved).’’
Responsiveness In the absence of a gold standard,9,43 the responsiveness of LES was analyzed using distribution-based methods (effect size [ES], standardized response mean [SRM], and GRR) and anchor-based methods (ROC analysis and correlation coefficient). ES is the ratio of the mean score change to the standard deviation (SD) of the initial score.20 SRM is the ratio of the mean score change to the SD of the score change.22 GRR is the ratio of the mean score change in patients who were ‘‘definitely satisfied (improved)’’ to the SD of the score change in patients who were ‘‘not definitely satisfied (not improved).’’14 Values of ES, SRM and GRR were interpreted using Cohen’s criteria, whereby values of more than 0.8, 0.5, and 0.2 indicate a large, moderate, and small effect, respectively.7,15,31,34 We hypothesized that change in LES would show a moderate but significant correlation to patient satisfaction (‘‘unsatisfied,’’ ‘‘somewhat satisfied,’’ ‘‘satisfied,’’ and ‘‘very satisfied’’). This was tested by calculating the Spearman correlation coefficient, and the P value was used to assess its statistical significance. Further interpretation used Cohen’s criteria, whereby >0.10, 0.30, and 0.50 represent a small, moderate, and large correlation, respectively. For calculating the area under the ROC curve (AUC), patients were grouped as ‘‘definitely satisfied (improved)’’ and ‘‘not definitely satisfied (not improved).’’. ROC plots sensitivity (y-axis) against 1-specificity (x-axis) for all possible cutoff points (ie, the true positive rate against the false positive rate). AUC spans the range of 0.5 (no discrimination better than chance) to 1.0 (perfect discrimination).10,36 The sensitivity, specificity, and 95% confidence interval of the AUC and P values were calculated. In common practice, an outcome measure is considered to be responsive if the GRR >1.9616 or AUC >0.70,16,20,40 or ES or SRM >0.80.29
Floor and ceiling effect Floor and ceiling effects occur when a large fraction of respondents (conventionally, 15%)16,40 achieve the lowest and highest possible score, respectively. These fractions (ie, patients with LES ¼ 0 and LES ¼ 10, respectively) were checked preoperatively and then at 6 months, 1 year, and 2 years postoperatively.
Statistical tests Continuous data are presented as mean (SD) and range. Categoric data are presented as number and percentages. SPSS 17.0 software (SPSS Inc, Chicago, IL, USA) was used for statistical analysis.
Results Patient characteristics The patients were a mean age of 63 years (range, 2086 years); 78 (65%) were women, and 43 (36%) were men.
314 Type of TER was available for 112 of 121 patients: 87 (78%) had primary TER and 25 had revision TER (22%). At presentation, 57 patients (56%) had unilateral involvement and 45 (37%) had bilateral elbow involvement. The dominant elbow was affected in 52 patients (50%) and the nondominant side in 53 (50%). The indications for TER were rheumatoid arthritis in 44, loosening in 25, osteoarthritis in 21, post-traumatic arthritis in 10, fracture of humerus in 7, and hemophiliac arthritis in 3.
LES and patient satisfaction Mean preoperative LES improved from 3.5 (SD, 1.6; range, 0.2-8.3) to 6.4 (SD, 1.8; range, 1.9-9.3). Mean follow-up was 19 months (range, 6 months-7 years). At the postoperative LES evaluation, 2 patients (3%) were not satisfied, 9 (12%) were somewhat satisfied, 34 (44%) were satisfied, and 33 (42%) were very satisfied with the results of TER. Preoperative LES was available in 105 of 121 patients (87%) and the postoperative LES in 85 (70%). For ES estimation, details of LES change were available in 75 patients and preoperative LES in 109. For SRM calculation, data were available in 75 patients. Patient satisfaction data were available in 78 of 121 patients (65%). For GRR, Cohen’s criteria estimation, and ROC analysis, complete data on patient satisfaction and LES change were available in 68 patients.
Responsiveness and floor/ceiling effect The overall ES and SRM were both >0.8 (Table I). Table I and Figure 1 show that as satisfaction level improved, the mean change in LES, ES, and SRM also increased, and vice versa. The mean change in LES in the ‘‘not definitely satisfied’’ and ‘‘definitely satisfied’’ groups was 1.4 (SD, 1.7) and 2.9 (SD, 2.1), respectively. The estimated GRR was 1.69. ES, SRM, and GRR demonstrated large effect size by Cohen’s criteria. There was significant positive correlation (Spearman correlation coefficient, 0.35; P ¼ .004) between LES change and satisfaction grades, which is in line with our hypothesis. The AUC was 0.71 (95% confidence interval, 0.56-0.87), significantly different from 0.5 (P ¼ .03; Figure 2). The sensitivity was 58% and specificity was 82%. AUC was >0.7, indicating that LES is adequately discriminative between satisfied and not definitely satisfied patients. LES showed no significant floor or ceiling effect preoperatively and postoperatively, because none of the patients scored the lowest or highest possible score (Table II).
Discussion This prospective study is the first to evaluate responsiveness and floor/ceiling effects of LES in TER. It gains in robustness by applying anchor-based and distribution-based methods and in external validity, with specific applicability to TER by being based on primary and revision
K. Vishwanathan et al. Table I Mean change in Liverpool Elbow Score, effect size, and standardized response mean in the entire cohort and in groups with different satisfaction levels Satisfaction level
Change in LES
ES
SRM
1.64 0.10 1.44 1.49 1.78
1.25 0.14 0.99 1.28 1.51
Mean (SD) Overall Unsatisfied Somewhat satisfied Satisfied Very satisfied
2.6 0.1 1.7 2.4 3.5
(2.11) (0.35) (1.70) (1.86) (2.29)
ES, effect size; LES, Liverpool Elbow Score; SRM, standardized response mean.
arthroplasties in different underlying conditions. By published criteria,29,40 LES shows satisfactory responsiveness in TER by both kinds of method (ES, SRM, and GRR were >0.8; AUC was >0.70), and it was not affected by ceiling/floor effects. In line with our hypothesis, there is moderate correlation between change in LES and the chosen external anchor (patient satisfaction), and ROC AUC was significantly >0.70. Because the hypothesis was predefined and sample size was >50 patients, this is Level 1 evidence for responsiveness.42 The original validation study described responsiveness of the LES in 18 patients with different elbow conditions 6 months postoperatively after various interventions.35 The study tested the responsiveness by comparing preoperative and postoperative LES and correlating it with change in region-specific (Disabilities of the Arm, Shoulder and Hand [DASH]) and generic outcome measures (Nottingham Health Profile [NHP] and Short Form 12-Item Health Survey [SF-12]), finding significant correlation with DASH (r ¼ 0.45) and NHP (r ¼ 0.42) but nonsignificant correlation with SF-12. Since this original description, new methods have become available for optimal assessment of responsiveness,15 and we therefore wished to readdress this question. Another study described correlation of LES with Mayo Elbow Performance Score (MEPS) in 44 patients with rheumatoid and post-traumatic arthritis who had TER (GSB III prosthesis, Sulzer Medica, Winterthur, Switzerland),2 finding good correlation between the LES and MEPS final follow-up scores (correlation coefficient, 0.84; P < .001). Neither study reported preoperative and postoperative LES. A recent study reported only postoperative LES in 54 rheumatoid arthritis and post-traumatic arthritis patients with TER (GSB III prosthesis).1 The present study is the first to report on both preoperative and postoperative LES in elbow arthroplasty. Ceiling and floor effects have not been previously assessed for any of elbow outcome scores. Compared with spine and shoulder, there are relatively few studies evaluating responsiveness of elbow outcome instruments. LES had higher ES compared with ES of the Oswestry Disability Index (range, 0.37-0.84).4,11,25 SRM and AUC of LES was within the range of values observed
Responsiveness of the Liverpool Elbow Score
Figure 1
Figure 2 analysis.
315
Mean change in Liverpool Elbow Score (LES) and its 95% confidence interval (error bars) in various satisfaction level groups.
Receiver operating characteristic (ROC) curve
for SRM (range, 0.3-1.9)5,18,21,32,37,39 and AUC (range, 0.67-0.92) of the Oswestry Disability Index.3,5,11–13,25,26 The Oswestry Disability Index had higher correlation coefficient (0.69) compared with LES.25 Our study could be compared to the responsiveness study on Oxford Elbow Score (OES) and DASH, consisting of 74 patients,6 in terms of large sample size. Another study on DASH17 had a small sample size of 35 patients, whereas the study comparing Mayo Elbow Performance Scale (MEPS), the Hospital for Special Surgery (HSS) elbow assessment scale (modified version of HSS elbow assessment scale [HSS-2]) and Elbow Functional Assessment (EFA) scale had only 25 patients.8 Our study population was a homogenous group of patients who underwent TER, whereas other studies have consisted of populations
undergoing various elbow surgeries, details of which were not specified,6 and a heterogeneous group consisting of TER, synovectomy, and radial head replacement in patients having rheumatoid arthritis.8,17 LES had higher ES than OES (range, 0.79-1.18 for various subscales)6 and DASH (range, 0.5-0.76).6,17 It was not possible to compare ES of HSS, HSS-2, EFA, and MEPS because the study failed to mention overall ES and instead mentioned ES separately for improved and nonchanged cohorts. SRM for OES has not been described. SRM of LES is higher than SRM of the DASH score (0.6) for elbow disorders.17 It was not possible to compare SRM of HSS, HSS-2, EFA, and MEPS because the study8 failed to mention overall SRM and instead mentioned SRM separately for improved and nonchanged cohorts. GRR in our study was lower than the threshold value of 1.96, suggesting that LES might have limited responsiveness; however; none of the studies on elbow scores reported GRR; hence, it is difficult to compare our result with other elbow scores. Because GRR depends on the external anchor chosen, it is possible that patient satisfaction might not be an ideal external anchor. The correlation coefficient of LES (0.35) was comparable to that of DASH (range, 0.35-0.53)6 and slightly lower than the correlation coefficient of OES (range, 0.370.58 for various subscales using different external anchors).6 Correlation coefficient has not been described for MEPS, HSS, HSS-2, and MEPS.8 The AUC of LES (0.71) was slightly better than the AUC of MEPS (0.70)8 and was comparable with the AUC of DASH (range, 0.71-0.76).6 The AUC of LES was lower than the AUC of OES (range, 0.72-0.90 for various
316
K. Vishwanathan et al.
Table II
The range of values for Liverpool Elbow Score at various evaluation points Evaluation point No.) LES (No.)y Preoperative Postoperative 6 months 1 year 2 years
Floor effect (%)
Ceiling effect (%)
8.3 (1)
0
0
9.4 (1) 9.5 (1) 9.6 (1)
0 0 0
0 0 0
Lower limit
Upper limit
105
0.2 (2)
65 66 44
1.0 (1) 2.5 (1) 1.9 (1)
LES, Liverpool Elbow Score. ) Total number of patients available for follow-up at various times. y Number of patients showing lowest and highest values of LES at various times.
subscales using 2 different external anchors),6 HSS (0.83),8 HSS-2 (0.86),8 and EFA (0.91).8 Comparison with other studies has limited value because the AUC of OES and DASH was estimated using external anchors, such as elbow pain postoperatively and elbow function postoperatively,6 while the AUC of HSS, HSS-2, MEPS, and EFA was calculated using global perceived effect of operative intervention as an external anchor.8 None of the studies on elbow scores used patient satisfaction as external anchor. Of course, the choice of external criteria for change is debatable for any method of responsiveness estimation. Whereas most studies use patient-assessed global assessment of change as an external criterion, we used patient satisfaction. The satisfaction score used is well established, but its validity and reliability are still to be determined. No claim is being made that the LES is a gold standard, and more comparisons of elbow scales against other external anchors may be needed. Patient satisfaction after surgical intervention is influenced by various factors, such as age,38 marital status,7 expectations before and after operative intervention,7,24 and severity of condition preoperatively,24 and hence, patient satisfaction might not be the ideal choice for an external anchor. This could possibly account for the AUC of LES being just above 0.70 and the GRR of LES being below 1.96. Values of measures of responsiveness, such as the AUC, GRR, and correlation coefficient, tend to vary with the choice of external anchor. This was demonstrated in the study by Dawson et al,6 wherein the AUC and correlation coefficient values of OES and DASH varied with use of different external anchors such as elbow pain postoperatively, elbow function postoperatively, and patient satisfaction. We consider the choice of a single question in a Likert scale pattern concerning patient satisfaction as the only external anchor to be a limitation to the design of this prospective study. Another limitation of our study was the use of a single external anchor. Using different external anchors might possibly reveal different values of GRR, the correlation coefficient, and AUC, because the value of these measures tends to depend on the external anchor used. Future studies could evaluate the influence of various
external anchors on values of the AUC, GRR, and correlation coefficient of LES. We do not claim superiority of LES over other elbow outcome tools because we did not perform such a comparison here. The relatively large amount of missing data could threaten the internal validity of the study, but the sample size of >50 patients and the availability of data from 75 patients for SRM calculation and 68 patients for GRR, correlation coefficient, and ROC analysis should have limited the effect on the overall results.
Conclusion Distribution-based and anchor-based methods have shown that LES is a responsive measure and is sensitive to changes in the clinical condition of the patient after TER. LES has no floor and ceiling effect postoperatively, and this encourages use of LES as an outcome evaluation for TER. Hence, the LES is a useful tool to assess the outcome of total elbow arthroplasty.
Disclaimer S.P.F. is a consultant for Biomet Inc, but Biomet had no influence on the methods and results of this study. The authors, their immediate families, and any research foundations with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.
References 1. Amirfeyz R, Blewitt N. Mid-term outcome of GSB-III total elbow arthroplasty in patients with rheumatoid arthritis and patients with post-traumatic arthritis. Arch Orthop Trauma Surg 2009;129:1505-10. http://dx.doi.org/10.1007/s00402-009-0876-y
Responsiveness of the Liverpool Elbow Score 2. Ashmore AM, Gozzard C, Blewitt N. Use of the Liverpool Elbow Score as a postal questionnaire for the assessment of outcome after total elbow arthroplasty. J Shoulder Elbow Surg 2007;16:55-8. http:// dx.doi.org/10.1016/j.jse.2006.08.008 3. Beurskens AJ, de Vet HC, K€oke AJ. Responsiveness of functional status in low back pain: a comparison of different instruments. Pain 1996;65: 71-6. 4. Coelho RA, Siqueira FB, Ferreira PH, Ferreira ML. Responsiveness of the Brazilian-Portuguese version of the Oswestry Disability Index in subjects with low back pain. Eur Spine J 2008;17:1101-6. http://dx. doi.org/10.1007/s00586-008-0690-1 5. Davidson M, Keating JL. A comparison of five low back disability questionnaires: reliability and responsiveness. Phys Ther 2002;82:8-24. 6. Dawson J, Doll H, Boller I, Fitzpatrick R, Little C, Rees J, Carr A. Comparative responsiveness and minimal change for the Oxford Elbow Score following surgery. Qual Life Res 2008;17:1257-67. http://dx.doi.org/10.1007/s11136-008-9409-3 7. Dawson J, Doll H, Boller I, Fitzpatrick R, Little C, Rees J, et al. Factors associated with satisfaction in patients undergoing elbow surgery: a prospective study. J Shoulder Elbow Surg 2010;19:635-44. http://dx.doi.org/10.1016/j.jse.2010.02.003 8. De Boer YA, Hazes JMW, Winia PCA, Brand R, Rozing PM. Comparative responsiveness of four elbow scoring instruments in patients with rheumatoid arthritis. J Rheumatol 2001;28:2616-23. 9. de Vet HC, Terwee CB, Bouter LM. Current challenges in clinimetrics. J Clin Epidemiol 2003;56:1137-41. http://dx.doi.org/10.1016/ j.jclinepi.2003.08.012 10. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Controlled Clin Trial 1991;12:142-58. 11. Farasyn A, Meeusen R. Validity of the new Backache Index (BAI) in patients with low back pain. Spine J 2006;6:565-71. http://dx.doi.org/ 10.1016/j.spinee.2006.01.021 12. Frost H, Lamb SE, Stewart-Brown S. Responsiveness of a patient specific outcome measure compared with the Oswestry Disability Index v2.1 and Roland and Morris Disability Questionnaire for patients with subacute and chronic low back pain. Spine 2008;33: 2450-7. http://dx.doi.org/10.1097/BRS.0b013e31818916fd 13. Grotle M, Brox JI, Vøllestad NK. Concurrent comparison of responsiveness in pain and functional status measurements used for patients with low back pain. Spine 2004;29:492-501. 14. Guyatt GH, Bombardier C, Tugwell PX. Measuring disease-specific quality of life in clinical trials. CMAJ 1986;134:889-95. 15. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol 2000;53:459-68. 16. Irrgang JJ, Lubowitz JH. Measuring arthroscopic outcome. Arthroscopy 2008;24:718-22. http://dx.doi.org/10.1016/j.arthro.2007.10.007 17. Ishikawa H, Murasawa A, Nakazono K, Abe A, Otani H, Netsu T, et al. The patient-based outcome of upper-extremity surgeries using the DASH questionnaire and the effect of disease activity of the patients with rheumatoid arthritis. Clin Rheumatol 2008;27:967-73. 18. Jette DU, Jette AM. Physical therapy and health outcomes in patients with spinal impairments. Phys Ther 1996;76:930-41. 19. Kalogrianitis S, Sinopidis C, El Meligy M, Rawal A, Frostick SP. Unlinked elbow arthroplasty as primary treatment for fractures of the distal humerus. J Shoulder Elbow Surg 2008;17:287-92. http://dx.doi. org/10.1016/j.jse.2007.06.011 20. Kazis L, Anderson J, Meenan R. Effect sizes for interpreting changes in health status. Med Care 1989;277(3 Suppl):S178-89. 21. Lauridsen HH, Hartvigsen J, Manniche C, Korsholm L, Grunnet-Nilsson N. Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskelet Disord 2006;7:1-15. http://dx.doi.org/ 10.1186/1471-2474-7-82 22. Liang M, Fossel A, Larson M. Comparison of five health-status instruments for orthopaedic evaluation. Med Care 1990;28:632-42.
317 23. Lohr KN, Aaronson NK, Alonso J, Burnam MA, Patrick DL, Perrin EB, et al. Evaluating quality of life and health status instruments: development of scientific review criteria. Clin Ther 1996;18:979-92. 24. Mancuso CA, Salvati EA, Johanson NA, Peterson MG, Charlson ME. Patients’ expectations and satisfaction with total hip arthroplasty. J Arthroplasty 1997;12:387-96. 25. Mannion AF, Junge A, Grob D, Dvorak J, Fairbank JC. Development of a German version of the Oswestry Disability Index. Part 2: sensitivity to change after spinal surgery. Eur Spine J 2006;15:66-73. http:// dx.doi.org/10.1007/s00586-004-0816-z 26. Maughan EF, Lewis JS. Outcome measures in chronic low back pain. Eur Spine J 2010;19:1484-94. http://dx.doi.org/10.1007/s00586-0101353-6 27. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 1995;4:293-307. 28. Munoz-Mahamud E, Fernandez-Valencia JA, Riba J. Plate osteosynthesis for severe olecranon fractures. J Orthop Surg 2010;18:80-4. 29. Naal FD, Impellizzeri FM, Rippstein PF. Which are the most frequently used outcome instruments in studies on total ankle arthroplasty? Clin Orthop Relat Res 2010;468:815-26. http://dx.doi. org/10.1007/s11999-009-1036-y 30. Norman GR, Gwadry Sridhar F, Guyatt G, Walter SD. Relation of distribution- and anchor based approaches in interpretation of changes in health related quality of life. Med Care 2001;39:1039-47. 31. Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;50:869-79. 32. Patrick DL, Deyo RA, Atlas SJ, Singer DE, Chapin A, Keller RB. Assessing health-related quality of life in patients with sciatica. Spine 1995;20:1899-909. 33. Reising K, Hauschild O, Strohm PC, Suedkamp NP. Stabilisation of articular fractures of the distal humerus: Early experience with a novel perpendicular plate system. Injury 2009;40:611-7. http://dx.doi.org/10. 1016/j.injury.2008.12.018 34. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102-9. http://dx. doi.org/10.1016/j.jclinepi.2007.03.012 35. Sathyamoorthy P, Kemp GJ, Rawal A, Rayner V, Frostick SP. Development and validation of an elbow score. Rheumatology 2004; 43:1434-40. http://dx.doi.org/10.1093/rheumatology/keh367 36. Streiner DL, Cairney J. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatry 2007;52:121-8. 37. Suarez-Almazor ME, Kendall C, Johnson JA, Skeith K, Vincent D. Use of health status measures in patients with low back pain in clinical settings: comparison of specific, generic and preference-based instruments. Rheumatology 2000;39:783-90. http://dx.doi.org/10. 1093/rheumatology/39.7.783 38. Tashjian RZ, Bradley MP, Tocci S, Rey J, Henn RF, Green A. Factors influencing patient satisfaction after rotator cuff repair. J Shoulder Elbow Surg 2007;16:752-8. http://dx.doi.org/10.1016/j.jse.2007.02.136 39. Taylor SJ, Taylor AE, Foy MA, Fogg AJ. Responsiveness of common outcome measures for patients with low back pain. Spine 1999;24:1805-12. 40. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. Clin Epidemiol 2007;60:3442. http://dx.doi.org/10.1016/j.jclinepi.2006.03.012 41. Testa MA, Simonson DC. Assessment of quality-of-life outcomes. N Engl J Med 1996;334:835-40. 42. Van der Leeden M, Steultjens MP, Terwee CB, Rosenbaum D, Turner D, Woodburn J, et al. A systematic review of instruments measuring foot function, foot pain, and foot-related disability in patients with rheumatoid arthritis. Arthritis Rheum 2008;59:1257-69. http://dx.doi.org/10.1002/art.24016 43. Wright J, Young N. A comparison of different indices of responsiveness. J Clin Epidemiol 1997;50:239-46.