- Email: [email protected]
A comparison of prospective and retrospective assessment of functional outcome after rotator cuff repair
A comparison of prospective and retrospective assessment of functional outcome after rotator cuff repair Robert Z. Tashjian, MD,a Michael P. Bradley, MD,b Stephen Tocci, MD,c Ralph F. Henn, MD,c Jesus Rey, MD,b and Andrew Green, MD,b Salt Lake City, UT, Providence, RI, and New York, NY
Prospective outcome studies are generally considered to be better than retrospective studies. The purpose of this study was to assess correlations between prospective and retrospective outcome assessment after rotator cuff repair. One-hundred and twelve patients (118 shoulders) with chronic rotator cuff tears were evaluated at a mean of 54 months (34-85) after rotator cuff repair, using several outcome measures including a retrospective assessment of improvement. The retrospective assessment of post-operative pain, function, and quality of life had fair correlations with the prospectively determined improvement (R ¼ .23-.25, P < .01). Postoperative patient satisfaction was more highly correlated with all retrospective evaluations than with the prospective improvement in all functional outcome measures. Retrospective and prospective evaluations of the outcome of rotator cuff repair are different. Patient satisfaction has a greater correlation with retrospective outcomes. Retrospective evaluation may aid in supplementing prospective evaluations, as it may better reflect a patient’s perception of the success after surgery. (J Shoulder Elbow Surg 2008;17:853-859.)
D
etermining the outcome of orthopaedic surgical procedures, such as rotator cuff repair, is essential to define the impact of disease and to determine the optimal treatment. Rotator cuff disorders are common, and in the future, it is possible that there will be a dramatic increase in symptomatic individuals. Therefore, it is important that the outcome assessment is performed with the most relevant and appropriate tools and methodology. From the Department of Orthopaedics,a University of Utah, Salt Lake City, UT, Department of Orthopedic Surgery,b Brown Medical School, Providence, and Department of Orthopaedics,c Hospital For Special Surgery, New York, NY. Reprint requests: Robert Z. Tashjian, MD, University of Utah Orthopaedic Center, 590 Wakara Way, Salt Lake City, UT 84108 (E-mail: [email protected]). Copyright ª 2008 by Journal of Shoulder and Elbow Surgery Board of Trustees. 1058-2746/2008/$34.00 doi:10.1016/j.jse.2008.04.003
Retrospective and prospective analyses are the 2 principle methods of health change measurement. Prospective methods evaluate serial change in outcome parameters from baseline to the endpoint of assessment. Retrospective studies utilize an assessment of improvement from a prior health status to the current health state, which is determined at the time of study without the use of earlier or pre-treatment information. Retrospective measurements are often labeled transition questions.6 Transition questions are often phrased as, ‘‘Are you feeling better or worse, and if so, what is the extent of the change?’’ Although these 2 types of measurements of change are commonly used in orthopaedic studies, there is little true understanding of how the information obtained is different or clinically relevant. Many authors assume that serial, or prospective, measurement of outcome is more accurate than a patient’s retrospective perceptions of changes from treatment.8 To some extent, the question under consideration in a specific study may determine whether a prospective method is ideal. Studies that compare the efficacy of different treatments, pharmaceuticals, and procedures appear to require prospective analysis. In contrast, there may be some studies that do not require prospective methods. In fact, it may be more relevant or appropriate to use retrospective analysis. The outcome of orthopaedic treatments usually relate to quality of life, in contrast to traditional concepts of medical health. If a patient’s perception, or self-assessment, of the result of treatment is the optimal outcome measure, then perhaps a retrospective assessment of the outcome is the most relevant endpoint. Despite these potential benefits of a retrospective study, there are some inherent limitations or biases of a retrospective assessment, including a lack of randomization and controls during outcome assessment. The purpose of this investigation was to evaluate the correlation between prospective and retrospective assessment of the outcome of rotator cuff repair and their correlation with final postoperative patient satisfaction. Our goal is to determine how well these measures reflect one another and which outcome assessment measure (retrospective or prospective) better reflects patient satisfaction which we consider as the benchmark measure of the outcome of treatment.
853
854
Tashjian et al
J Shoulder Elbow Surg November/December 2008
Our hypothesis is that retrospective measures better reflect satisfaction and should be included as part of prospective outcomes assessment.
Table I Comparison of responders and non-responders
MATERIALS AND METHODS
Average age# Number of open repairsy Number of complete arthroscopic repairsy Number of mini-open repairsy Average number of comorbidities# Average number of previous non-shoulder surgical procedures# Average tear size (cm)# Average duration of preoperative symptoms (months)#
This study was approved by our hospital investigational review board. One-hundred and twelve patients (118 shoulders) who underwent rotator cuff repair were recruited. All had a chronic rotator cuff tear, defined as having symptoms for greater than three months. The indications for surgery included failure of nonoperative treatment with a physical therapy program, and, in some cases, a corticosteroid injection. Patients with glenohumeral arthritis, adhesive capsulitis, and history of a workers’ compensation claim were excluded. Full thickness rotator cuff tears were confirmed at the time of surgery. The mean length of follow-up after repair was 54 months (range, 34-85). The mean age at the time of surgery was 59 years (range, 33-86 years). Sixty-five shoulders (55%) were in men and 53 shoulders (45%) in women. The mean duration (and standard deviation) of the symptoms prior to the preoperative evaluation was 15 6 20.17 months. The mean anterior-posterior tear size was 2.3 6 1.18 cm. Tears were classified as small (< 1 cm), medium (1 – 3 cm), large (3 – 5 cm), or massive (> 5 cm). There were 21 small, 49 medium, 35 large, and 13 massive rotator cuff tears. Twenty-five shoulders (21%) had an open repair, 62 (53%) had a mini-open repair, and 31 (26%) had a complete arthroscopic repair. The preoperative evaluation included demographic information, a detailed medical history, a physical examination, and completion of a questionnaire with a series of sections evaluating shoulder pain, functional abilities, and general health status.14 Tear size, presence of associated shoulder pathology, and details of the surgical technique were obtained from operative reports. The pre-operative assessment data and information was stored in a prospectively maintained database. The questionnaire included the Disability of the Arm, Shoulder, and Hand (DASH) questionnaire.1 The DASH was designed to assess symptoms and functional status, with an emphasis on physical function, in patients with upper extremity musculoskeletal conditions. The DASH specifically questions pain, weakness, stiffness, ability to perform daily activities (eg, dressing, eating, sleeping), occupational function, family care, and self-image.9 It utilizes a Likert scale, which presents a set of attitude statements and then asks the respondent whether they agree or disagree in differing degrees of certainty using a numerical scaling system.7 The DASH has been validated for the assessment of shoulder disorders.4,13 We used the DASH as a region specific tool to measure functional outcome. All patients completed the DASH questionnaire. The 2nd section of the patient self-assessment questionnaire addressed general medical problems, included in the The Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS). The MODEMS is an instrument for collecting musculoskeletal outcome data using several questionnaires,2 and includes questions about comorbidities, work status, and education level. The 3rd section included the Simple Shoulder Test (SST). We used the SST as a shoulder/region specific tool to
P Responders Non-responders value 55 25 31
59 25 48
.004* .615 .104
63
59
.138
2.08
1.99
.681
2.14
2.07
.765
2.33 14.54
2.32 16.9
.739 .485
P values for Chi-squarey and t tests#. *represents significance (P < .05).
measure functional outcome. The results of the SST are recorded as the total number of positive responses. The 4th section included Visual Analog Scales (VAS) for shoulder pain, an overall rating of shoulder function, and an overall quality of life assessment. The patients were asked to place a mark on a 10-cm line that was marked on either end from ‘‘none’’ to ‘‘disabling’’ for pain, ‘‘comfortable’’ to ‘‘can’t use it’’ for function, and from ‘‘little or no problem’’ to ‘‘very bad’’ for quality of life. After the patients completed each VAS assessment, the mark was measured to the nearest millimeter and recorded as the score in centimeters. Final follow-up evaluations were performed at greater than 2 years postoperatively. We initially attempted to contact each patient who underwent rotator cuff repair from 1998 to 2001 by telephone, informing them of our interest to include them in the study. Two-hundred and fifty-two patients were included in the initial phone list. Each patient was sent a follow-up questionnaire. Attempts to re-contact all of the patients who did not respond to the initial mailing were made with at least 2 additional mailings. The final cohort of patients who were included was those who responded. The responders and non-responders were compared using Pearson Chi-squares (surgical technique) and 2-sample t tests (age, number of co-morbidities, number of previous surgical procedures (non-shoulder), tear size, duration of symptoms). There were no significant differences between responders and non-responders in terms of surgical technique, number of co-morbidities, tear size, or duration of symptoms (P > .05). Responders (55 years old) were slightly younger than non-responders (59 years old) (P ¼ .004) (Table I). The follow-up questionnaire included the follow-up module for the MODEMS (including the DASH and various demographic information), the SST, and VAS (pain, function, quality of life). A final section of the follow-up questionnaire assessed patient satisfaction and retrospective improvement in outcomes (Figure 1). Patients were asked to complete a visual analog scale to rate their level of satisfaction. We term
Tashjian et al
J Shoulder Elbow Surg Volume 17, Number 6
855
Table II Correlations between retrospective outcomes and final DASH, SST, and VAS scores Retro Function VAS VAS VAS question pain function quality of life
Figure 1 Visual Analog Scale (VAS) Satisfaction Score.
#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11
.6422
SST
DASH
.5479 .6069 .5437 .5883
.6269 .5737
.5059 .5838 .7236 .7029 .6916 .5821 .6800 .5211 .6258 .6104 .6581
Spearman correlation coefficients (rs) for correlations between the retrospective outcomes (Retro Function questions 1-11) and the absolute outcome scores at follow-up for DASH, SST and VAS (pain, function and quality of life). All correlations are significant with all P values < .0001.
Figure 2 Retrospective questionnaire.
this the VAS Satisfaction Score. The patients were asked to place a mark on a 10-cm line that was marked on either end from ‘‘not satisfied at all’’ to ‘‘very satisfied.’’ After completion of the VAS assessment, the mark was measured to the nearest millimeter and recorded as the score in centimeters. The results of the prospective outcome evaluation were obtained by determining the differences between the follow-up assessment and the pre-treatment assessment. Retrospective outcome assessment was evaluated using a questionnaire with eleven 7-point Likert-format questions. The questions were derived from the context of several of the prospective questionnaires utilized in this study evaluating pain, function, and quality of life. The questions asked, ‘‘How would your rate your ______ now compared to before your surgery?’’, with the blanks filled in with pain, shoulder function, quality of life, ability to sleep comfortably, ability to perform your usual job, ability to exercise or play sport activities, ability to perform every day housework or yardwork
activities, shoulder symptoms (stiffness, swelling, numbness, weakness, and instability), ability to do activities above your head, ability to carry objects at your side, and shoulder motion (Figure 2). We termed each of the questions Retro Function (1-11). Patients were asked to agree or disagree in different degrees of certainty from ‘‘very much worse’’ to ‘‘very much better,’’ with the former given a value of 1 and the latter a value of 7. The outcome parameter scores obtained at the final evaluation were also used as a form of retrospective outcome assessment. The validity of the Retro Function questions was assessed by determining the correlation between the responses to these questions with the outcomes obtained at the followup evaluation (Table II). There was statistically significant, moderate correlation between the Retro Function questions and the related outcomes. There were also statistically significant, moderate correlations between the VAS scores for pain, function, and quality of life and the first 3 Retro Function questions. Consequently, we concluded that the Retro Function questions are a valid tool for retrospective assessment of the outcome of rotator cuff repair
Statistical Analysis Paired t tests were used to compare baseline and postoperative outcomes (DASH, SST, VAS pain, function, and quality of life). Spearman correlations were utilized to compare retrospective function (Retro Function 1-11) with the improvement from baseline of the DASH, Simple Shoulder Test, and VAS pain, function, and quality of life. Spearman correlations were also used to compare Retro Function (1-11) and the improvement from baseline of the DASH, SST, and VAS (pain, function, and quality of life) with final postoperative patient satisfaction (VAS Satisfaction Score). Spearman correlations were found to have little or no relationship with Rs values between 0 and .25, a fair relationship with Rs values between .25 and .5, a moderate to good relationship with Rs values between .5 and .75, and a very good or excellent relationship with Rs values greater than .75. P values < .05 were considered statistically significant.
856
Tashjian et al
J Shoulder Elbow Surg November/December 2008
Table III Baseline, postoperative, and improvement from baseline of DASH, SST, and VAS scores Outcome score
Baseline score
Postoperative score
Improvement
P value
DASH SST VAS pain VAS function VAS quality of life
41.79 6 17.1 4.00 6 2.8 6.13 6 2.1 6.27 6 2.1 6.00 6 2.3
15.42 6 18.4 9.92 6 2.6 1.60 6 2.4 1.64 6 2.2 1.70 6 2.5
26.37 6 19.6 5.71 6 2.7 4.27 6 3.0 4.44 6 3.0 4.06 6 3.2
< .01* < .01* < .01* < .01* < .01*
Mean and standard deviations for baseline, postoperative and improvement from baseline DASH, SST, and VAS (pain, function, and quality of life) scores. P values represent t tests comparing baseline and postoperative scores. *represents significance (P < .05).
Table IV Retrospective outcome measures Retro Function question
Mean score ± standard deviation
#1 – shoulder pain #2 – shoulder function #3 – quality of life #4 – sleep comfortably #5 – perform usual job #6 – exercise/play sports #7 – housework/yardwork #8 – shoulder symptoms #9 – activities overhead #10 – activities at waist #11 – shoulder motion
6.33 6 1.1 6.25 6 1.1 6.03 6 1.3 6.02 6 1.2 5.93 61.4 5.87 6 1.4 5.98 61.3 6.10 61.2 5.98 6 1.3 6.02 61.2 6.11 6 1.3
Table V Correlations between patient satisfaction and improvement from baseline of the DASH, SST, and VAS scores Outcome parameter
Mean and standard deviations for retrospective outcomes (Retro Function questions 1-11). Range of scores is from 1 (lowest possible score) to 7 (highest possible score).
RESULTS Patient satisfaction with the result of their treatment was generally high. The mean VAS Satisfaction Score was 8.76 6 2.4. There was statistically significant improvement in all of the outcome parameters, including VAS for pain, function, and quality of life, the SST, and the DASH (see Table III). The mean retrospective scores all demonstrated that the patients thought that their result was greater than ‘‘better’’ (see Table IV). Improvement in shoulder pain had the highest retrospective score, while improvement in the ability to exercise or play sport activities had the lowest score. Postoperative patient satisfaction (VAS Satisfaction Score) was significantly correlated with improvement in VAS shoulder function (P ¼ .005), VAS quality of life (P ¼ .019), and the SST (P ¼ .036) (Table V). However, the Spearman correlation coefficients (Rs) ranged from .192 to .256, indicative of a fair correlation at best. There were no significant linear associations between postoperative satisfaction and the improvement in VAS pain and the DASH scores (P > .05). There were statistically significant correlations between the postoperative patient satisfaction (VAS Satisfaction Score) with all 11 of the retrospective
VAS pain VAS function VAS quality of life DASH SST
Correlation coefficient (Rs)
P value
.178 .256 .217 .151 .192
.055 .005* .019* .104 .036*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing patient satisfaction (VAS satisfaction score) and improvement from baseline of shoulder function (SST, DASH, VAS function), quality of life (VAS quality of life), and pain (VAS pain). *represents significance (P < .05)
functional questions (see Table VI). The Spearman correlation coefficients (Rs) ranged from .406 to .577. The strongest correlation was between satisfaction and question #1 (shoulder pain), while the weakest correlation was with question #5 (ability to perform usual job). The Spearman correlation coefficients comparing satisfaction and the retrospective outcomes were substantially greater than the corresponding correlations comparing satisfaction with the prospective outcomes (Table VI vs. Table V). There were also statistically significant correlations between postoperative patient satisfaction (VAS satisfaction) and the absolute scores of the outcome measures (VAS SST, DASH) at follow-up (Table VII). There were statistically significant correlations between the improvement in VAS scores for pain, function, and quality of life and the first 3 retrospective questions: pain (Rs ¼ .226; P ¼.01), shoulder function (Rs ¼ .248; P ¼ .006), and quality of life (Rs ¼ .248; P ¼ .0063). Improvement in the SST was significantly correlated with all 7 retrospective functional questions to which it was compared (Rs ¼ .204-.281) (P ¼ .004-.026) (Table VIII). Because the SST only specifically relates to Retro Function questions 1, 2, 4, 5, and 9-11, correlations were only performed between these questions and the SST. These retrospective questions attempt to capture all components of the SST. The correlations
Tashjian et al
J Shoulder Elbow Surg Volume 17, Number 6
857
Table VI Correlations between patient satisfaction and retrospective outcomes
Table VIII Correlations between retrospective outcomes and improvement from baseline of the SST.
Retro Function question Correlation coefficient (Rs) P value
Retro Function question Correlation coefficient (Rs) P value
#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11
#1 #2 #4 #5 #9 #10 #11
.577 .543 .496 .413 .406 .478 .466 .567 .548 .483 .489
< .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing patient satisfaction (VAS Satisfaction Score) and retrospective outcomes after rotator cuff repair (Retro Function questions 1-11). *represents significance (P < .05).
Table VII Correlations between patient satisfaction and final postoperative DASH, SST, and VAS scores Outcome parameter VAS pain VAS function VAS quality of life DASH SST
Correlation coefficient (Rs)
P value
.604 .584 .621 .434 .400
< .001* < .001* < .001* < .001* < .001*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing patient satisfaction (VAS Satisfaction Score) with final postoperative shoulder function (SST, DASH, VAS function), quality of life (VAS quality of life), pain (VAS pain), and general health status (SF-36). *represents significance (P < .05)
between the retrospective function and prospective SST (range of Rs: .204-.281;Table VIII) were lower than the correlations between the retrospective function and postoperative satisfaction (range of Rs: .406-.577; Table VI). Improvement in the DASH was significantly correlated with all 8 retrospective functional questions to which it was compared (P ¼ .001-.005) (Table IX). Because the DASH only specifically relates to Retro Function questions 1-2 and 6-11, correlations were only performed between these questions and the DASH. Again, correlations between the retrospective function and prospective DASH (range of Rs: .257-.346;Table IX) were lower than the correlations between the retrospective outcome and postoperative satisfaction (range of Rs: .466-.577; Table VI). DISCUSSION In this study, we found that patient satisfaction after rotator cuff repair correlated with both prospective
.230 .233 .204 .281 .218 .263 .212
.011* .011* .026* .002* .017* .004* .020*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing the improvement from baseline of the Simple Shoulder Test (SST) and retrospective outcomes (Retro Function questions 1, 2, 4, 5, and 9-11). *represents significance (P < .05).
Table IX Correlations between retrospective outcomes and improvement from baseline of the DASH Retro Function question Correlation coefficient (Rs) P value #1 #2 #6 #7 #8 #9 #10 #11
.264 .259 .346 .313 .306 .269 .292 .257
.004* .004* < .001* .001* .001* .003* .001* .005*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing the improvement from baseline of the DASH and retrospective outcomes (Retro Function questions 1, 2, and 6-11). *represents significance (P < .05).
and retrospective outcome measures. However, the retrospective measures had much better correlation with patient satisfaction than the prospective measures. This suggests that retrospective self-assessed patient outcome is a better method for assessing outcomes that relate to patient satisfaction with the results of rotator cuff repair. Similarly, we also found that retrospective outcome assessment has only fair correlation with prospective assessments at best. These findings demonstrate that the results of retrospective and prospective outcome assessment are not the same, and that improvement does not correlate as well with the current status of patient satisfaction after rotator cuff repair. The emphasis on evidenced-based medicine has led to the classification of published clinical studies according to the level of evidence. Within this schema, prospective studies are afforded greater strength or value than retrospective studies. Despite this preference, the comparative validity or clinical relevance of prospective and retrospective studies has not been well studied. There are only a few studies that have
858
Tashjian et al
addressed the controversy between prospective and retrospective methodologies.5,12 Unfortunately, prospective data can be difficult to acquire and use in daily practice. Retrospective study is very appealing because it is easier to perform. Nevertheless, several authors have expressed concern about the use of retrospective instruments. Recall bias and guessing, when recall is deficient, have been cited as pitfalls of transition questions.8 There is a potential tendency for subjects to recall greater change than actually occurred, especially after surgical intervention.3,11 Several authors have challenged the use of retrospective measures claiming that they reflect follow-up status rather than a true change over time.10,12 Schmitt et al studied 211 patients with upper extremity musculoskeletal problems who were treated with physical therapy.12 They evaluated them at baseline and at 3 months follow-up with a number of outcomes measures. They found moderate correlations between prospective and retrospective outcomes measures (correlation coefficients between .57 and .66). However, because the correlations between the retrospective global ratings of change did not correlate well with the baseline scores, they concluded that retrospective measures were a less valid measure of change over time. Consequently, their conclusions suggest that retrospective measures are not valid outcome assessment tools. In contrast, we believe that it is more appropriate to conclude that retrospective and prospective outcome assessments have differing purposes. Fischer et al studied 202 patients who had chronic hip or knee arthritis.5 The outcomes of 3 different interventions (patient education, medical treatment, or surgery) were assessed. They found that when the change in outcome was small (education), prospective assessment (serial measures) correlated poorly with retrospective assessment. The correlations between retrospective and prospective outcome were stronger for treatments that produced a greater change (medication, surgery). They also demonstrated that retrospective measures were more sensitive to change than serial measures, and that they correlated more strongly with the patients’ satisfaction. Our results are not fully consistent with these findings, as we found that the correlations between the retrospective and prospective outcome assessment of rotator cuff repair were statistically significant, but not particularly strong. Nevertheless, we also found that the correlation between patient satisfaction and the retrospective assessment was stronger than the correlation between satisfaction and the prospective assessment of change in outcome. Despite apparently similar findings, that retrospective assessments correlate poorly with prospective or serial assessment, Fischer and Schmitt expressed con-
J Shoulder Elbow Surg November/December 2008
flicting conclusions.5,12 Schmitt concluded that retrospective assessment should not be used in outcome assessment because of the poor correlation. In contrast, Fischer concluded that retrospective assessments should be included in outcome evaluations, because they provide important information that is not well reflected by prospective outcome assessment. Consequently, Fischer recommended that prospective serial measures be supplemented with retrospective assessments. Our study of rotator cuff repair similarly demonstrates a moderate correlation between patient satisfaction and retrospective assessment and only a weak correlation between retrospective and prospective assessment. Because orthopaedic surgery deals primarily with quality of life issues and not life and death issues, final patient satisfaction, perhaps regardless of the determinants, should be a primary goal of treatment. Patient satisfaction may be affected by nonanatomic factors other than the surgical procedure, including the patient-physician relationship. While patient satisfaction may not be as well-defined an outcome measure as tendon healing, motion, or strength, it is still a very relevant and important outcome measure in orthopaedics. Our study enabled us to recognize that serial improvements in functional outcomes may not be the only or best method for predicting patient satisfaction with the outcome of rotator cuff repair. This study demonstrates that patient satisfaction and serial improvement measure 2 very different aspects of outcome. Consequently, retrospective assessment may aid in supplementing prospective assessments, as it appears to provide a better reflection of a patient’s perception of the success of rotator cuff repair, even though it does not correlate as well with the prospectively determined improvement in outcome. This study has limitations that may have biased our interpretation of the results. The range of outcome scores and patient satisfaction were relatively narrow. Most patients had a high satisfaction with outcome, and the outcomes were generally good. The results of the analysis might differ if the patient cohort included some with more varied outcomes.Another limitation is the high percentage of subjects lost to follow-up. In addition, we made assumptions about the assessment of patient satisfaction and retrospective functional assessment. We assume that use of a visual analog scale to determine satisfaction, while commonly utilized, is the best method to determine final patient satisfaction. Similarly, we have attempted to create retrospective questions that capture many aspects of the prospectively derived outcome data, although there may be better methods of questioning regarding retrospective outcomes. Without more extensive study, we cannot be sure that our methods are valid for the population that we studied.
J Shoulder Elbow Surg Volume 17, Number 6
We found that patient self-assessed outcome status after rotator cuff repair has better correlation with patient satisfaction, compared to the correlation between patient satisfaction and outcome change. The discrepancies between the correlations of satisfaction with prospective and retrospective assessment, which we identified, strongly suggest that the determinants of patient self-assessed outcomes after rotator cuff repair include factors and biases other than the actual change in self-assessed functional status. The relative importance of patient satisfaction and retrospective assessment of change compared to prospective assessment of change needs to be further elucidated.
REFERENCES
1. American Academy of Orthopaedic Surgeons: Disabilities of the Arm, Shoulder, and Hand. Rosemont, Illinois, American Academy of Orthopaedic Surgeons, 1995. 2. American Academy of Orthopaedic Surgeons: Musculoskeletal Outcomes Data Evaluation and Management System. Rosemont, Illinois, American Academy of Orthopaedic Surgeons, 1997. 3. Aseltine RH, Carlson KJ, Fowler FJ, Barry MJ. Comparing prospective and retrospective measures of treatment outcomes. Med Care 1995;33:AS67-76.
Tashjian et al
859
4. Beaton D, Richards RR. Assessing the reliability and responsiveness of 5 shoulder questionnaires. J Shoulder Elbow Surg 1998;7: 565-72. 5. Fischer D, Stewart AL, Bloch DA, Lorig K, Laurent D, Holman H. Capturing the patient’s view of change as a clinical outcome measure. JAMA 1999;282:1157-62. 6. Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol 2002;55:900-8. 7. Guyatt GH, Townsend M, Berman LB, Keller JL. A comparison of Likert and visual analogue scales for measuring change in function. J Chron Dis 1987;40:1129-33. 8. Herrmann D. Reporting current, past, and changed health status: what we know about distortion. Med Care 1995;33(Suppl): AS89-94. 9. Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: The DASH (Disabilities of the arm, shoulder, and hand). The Upper Extremity Collaborative Group. Am J Ind Med 1996;29:602-8. 10. Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;50:869-79. 11. Ross M. Relation of implicit theories to the construction of personal histories. Psychol Rev 1989;96:341-57. 12. Schmitt J, DiFabio RP. The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil 2005; 86:2270-6. 13. Soohoo NF, McDonald AP, Seiler JG, McGillivary GR. Evaluation of the construct validity of the DASH questionnaire by correlation to the SF-36. J Hand Surg Am 2002;27:537-41. 14. Blinded for Review.
Prospective outcome studies are generally considered to be better than retrospective studies. The purpose of this study was to assess correlations between prospective and retrospective outcome assessment after rotator cuff repair. One-hundred and twelve patients (118 shoulders) with chronic rotator cuff tears were evaluated at a mean of 54 months (34-85) after rotator cuff repair, using several outcome measures including a retrospective assessment of improvement. The retrospective assessment of post-operative pain, function, and quality of life had fair correlations with the prospectively determined improvement (R ¼ .23-.25, P < .01). Postoperative patient satisfaction was more highly correlated with all retrospective evaluations than with the prospective improvement in all functional outcome measures. Retrospective and prospective evaluations of the outcome of rotator cuff repair are different. Patient satisfaction has a greater correlation with retrospective outcomes. Retrospective evaluation may aid in supplementing prospective evaluations, as it may better reflect a patient’s perception of the success after surgery. (J Shoulder Elbow Surg 2008;17:853-859.)
D
etermining the outcome of orthopaedic surgical procedures, such as rotator cuff repair, is essential to define the impact of disease and to determine the optimal treatment. Rotator cuff disorders are common, and in the future, it is possible that there will be a dramatic increase in symptomatic individuals. Therefore, it is important that the outcome assessment is performed with the most relevant and appropriate tools and methodology. From the Department of Orthopaedics,a University of Utah, Salt Lake City, UT, Department of Orthopedic Surgery,b Brown Medical School, Providence, and Department of Orthopaedics,c Hospital For Special Surgery, New York, NY. Reprint requests: Robert Z. Tashjian, MD, University of Utah Orthopaedic Center, 590 Wakara Way, Salt Lake City, UT 84108 (E-mail: [email protected]). Copyright ª 2008 by Journal of Shoulder and Elbow Surgery Board of Trustees. 1058-2746/2008/$34.00 doi:10.1016/j.jse.2008.04.003
Retrospective and prospective analyses are the 2 principle methods of health change measurement. Prospective methods evaluate serial change in outcome parameters from baseline to the endpoint of assessment. Retrospective studies utilize an assessment of improvement from a prior health status to the current health state, which is determined at the time of study without the use of earlier or pre-treatment information. Retrospective measurements are often labeled transition questions.6 Transition questions are often phrased as, ‘‘Are you feeling better or worse, and if so, what is the extent of the change?’’ Although these 2 types of measurements of change are commonly used in orthopaedic studies, there is little true understanding of how the information obtained is different or clinically relevant. Many authors assume that serial, or prospective, measurement of outcome is more accurate than a patient’s retrospective perceptions of changes from treatment.8 To some extent, the question under consideration in a specific study may determine whether a prospective method is ideal. Studies that compare the efficacy of different treatments, pharmaceuticals, and procedures appear to require prospective analysis. In contrast, there may be some studies that do not require prospective methods. In fact, it may be more relevant or appropriate to use retrospective analysis. The outcome of orthopaedic treatments usually relate to quality of life, in contrast to traditional concepts of medical health. If a patient’s perception, or self-assessment, of the result of treatment is the optimal outcome measure, then perhaps a retrospective assessment of the outcome is the most relevant endpoint. Despite these potential benefits of a retrospective study, there are some inherent limitations or biases of a retrospective assessment, including a lack of randomization and controls during outcome assessment. The purpose of this investigation was to evaluate the correlation between prospective and retrospective assessment of the outcome of rotator cuff repair and their correlation with final postoperative patient satisfaction. Our goal is to determine how well these measures reflect one another and which outcome assessment measure (retrospective or prospective) better reflects patient satisfaction which we consider as the benchmark measure of the outcome of treatment.
853
854
Tashjian et al
J Shoulder Elbow Surg November/December 2008
Our hypothesis is that retrospective measures better reflect satisfaction and should be included as part of prospective outcomes assessment.
Table I Comparison of responders and non-responders
MATERIALS AND METHODS
Average age# Number of open repairsy Number of complete arthroscopic repairsy Number of mini-open repairsy Average number of comorbidities# Average number of previous non-shoulder surgical procedures# Average tear size (cm)# Average duration of preoperative symptoms (months)#
This study was approved by our hospital investigational review board. One-hundred and twelve patients (118 shoulders) who underwent rotator cuff repair were recruited. All had a chronic rotator cuff tear, defined as having symptoms for greater than three months. The indications for surgery included failure of nonoperative treatment with a physical therapy program, and, in some cases, a corticosteroid injection. Patients with glenohumeral arthritis, adhesive capsulitis, and history of a workers’ compensation claim were excluded. Full thickness rotator cuff tears were confirmed at the time of surgery. The mean length of follow-up after repair was 54 months (range, 34-85). The mean age at the time of surgery was 59 years (range, 33-86 years). Sixty-five shoulders (55%) were in men and 53 shoulders (45%) in women. The mean duration (and standard deviation) of the symptoms prior to the preoperative evaluation was 15 6 20.17 months. The mean anterior-posterior tear size was 2.3 6 1.18 cm. Tears were classified as small (< 1 cm), medium (1 – 3 cm), large (3 – 5 cm), or massive (> 5 cm). There were 21 small, 49 medium, 35 large, and 13 massive rotator cuff tears. Twenty-five shoulders (21%) had an open repair, 62 (53%) had a mini-open repair, and 31 (26%) had a complete arthroscopic repair. The preoperative evaluation included demographic information, a detailed medical history, a physical examination, and completion of a questionnaire with a series of sections evaluating shoulder pain, functional abilities, and general health status.14 Tear size, presence of associated shoulder pathology, and details of the surgical technique were obtained from operative reports. The pre-operative assessment data and information was stored in a prospectively maintained database. The questionnaire included the Disability of the Arm, Shoulder, and Hand (DASH) questionnaire.1 The DASH was designed to assess symptoms and functional status, with an emphasis on physical function, in patients with upper extremity musculoskeletal conditions. The DASH specifically questions pain, weakness, stiffness, ability to perform daily activities (eg, dressing, eating, sleeping), occupational function, family care, and self-image.9 It utilizes a Likert scale, which presents a set of attitude statements and then asks the respondent whether they agree or disagree in differing degrees of certainty using a numerical scaling system.7 The DASH has been validated for the assessment of shoulder disorders.4,13 We used the DASH as a region specific tool to measure functional outcome. All patients completed the DASH questionnaire. The 2nd section of the patient self-assessment questionnaire addressed general medical problems, included in the The Musculoskeletal Outcomes Data Evaluation and Management System (MODEMS). The MODEMS is an instrument for collecting musculoskeletal outcome data using several questionnaires,2 and includes questions about comorbidities, work status, and education level. The 3rd section included the Simple Shoulder Test (SST). We used the SST as a shoulder/region specific tool to
P Responders Non-responders value 55 25 31
59 25 48
.004* .615 .104
63
59
.138
2.08
1.99
.681
2.14
2.07
.765
2.33 14.54
2.32 16.9
.739 .485
P values for Chi-squarey and t tests#. *represents significance (P < .05).
measure functional outcome. The results of the SST are recorded as the total number of positive responses. The 4th section included Visual Analog Scales (VAS) for shoulder pain, an overall rating of shoulder function, and an overall quality of life assessment. The patients were asked to place a mark on a 10-cm line that was marked on either end from ‘‘none’’ to ‘‘disabling’’ for pain, ‘‘comfortable’’ to ‘‘can’t use it’’ for function, and from ‘‘little or no problem’’ to ‘‘very bad’’ for quality of life. After the patients completed each VAS assessment, the mark was measured to the nearest millimeter and recorded as the score in centimeters. Final follow-up evaluations were performed at greater than 2 years postoperatively. We initially attempted to contact each patient who underwent rotator cuff repair from 1998 to 2001 by telephone, informing them of our interest to include them in the study. Two-hundred and fifty-two patients were included in the initial phone list. Each patient was sent a follow-up questionnaire. Attempts to re-contact all of the patients who did not respond to the initial mailing were made with at least 2 additional mailings. The final cohort of patients who were included was those who responded. The responders and non-responders were compared using Pearson Chi-squares (surgical technique) and 2-sample t tests (age, number of co-morbidities, number of previous surgical procedures (non-shoulder), tear size, duration of symptoms). There were no significant differences between responders and non-responders in terms of surgical technique, number of co-morbidities, tear size, or duration of symptoms (P > .05). Responders (55 years old) were slightly younger than non-responders (59 years old) (P ¼ .004) (Table I). The follow-up questionnaire included the follow-up module for the MODEMS (including the DASH and various demographic information), the SST, and VAS (pain, function, quality of life). A final section of the follow-up questionnaire assessed patient satisfaction and retrospective improvement in outcomes (Figure 1). Patients were asked to complete a visual analog scale to rate their level of satisfaction. We term
Tashjian et al
J Shoulder Elbow Surg Volume 17, Number 6
855
Table II Correlations between retrospective outcomes and final DASH, SST, and VAS scores Retro Function VAS VAS VAS question pain function quality of life
Figure 1 Visual Analog Scale (VAS) Satisfaction Score.
#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11
.6422
SST
DASH
.5479 .6069 .5437 .5883
.6269 .5737
.5059 .5838 .7236 .7029 .6916 .5821 .6800 .5211 .6258 .6104 .6581
Spearman correlation coefficients (rs) for correlations between the retrospective outcomes (Retro Function questions 1-11) and the absolute outcome scores at follow-up for DASH, SST and VAS (pain, function and quality of life). All correlations are significant with all P values < .0001.
Figure 2 Retrospective questionnaire.
this the VAS Satisfaction Score. The patients were asked to place a mark on a 10-cm line that was marked on either end from ‘‘not satisfied at all’’ to ‘‘very satisfied.’’ After completion of the VAS assessment, the mark was measured to the nearest millimeter and recorded as the score in centimeters. The results of the prospective outcome evaluation were obtained by determining the differences between the follow-up assessment and the pre-treatment assessment. Retrospective outcome assessment was evaluated using a questionnaire with eleven 7-point Likert-format questions. The questions were derived from the context of several of the prospective questionnaires utilized in this study evaluating pain, function, and quality of life. The questions asked, ‘‘How would your rate your ______ now compared to before your surgery?’’, with the blanks filled in with pain, shoulder function, quality of life, ability to sleep comfortably, ability to perform your usual job, ability to exercise or play sport activities, ability to perform every day housework or yardwork
activities, shoulder symptoms (stiffness, swelling, numbness, weakness, and instability), ability to do activities above your head, ability to carry objects at your side, and shoulder motion (Figure 2). We termed each of the questions Retro Function (1-11). Patients were asked to agree or disagree in different degrees of certainty from ‘‘very much worse’’ to ‘‘very much better,’’ with the former given a value of 1 and the latter a value of 7. The outcome parameter scores obtained at the final evaluation were also used as a form of retrospective outcome assessment. The validity of the Retro Function questions was assessed by determining the correlation between the responses to these questions with the outcomes obtained at the followup evaluation (Table II). There was statistically significant, moderate correlation between the Retro Function questions and the related outcomes. There were also statistically significant, moderate correlations between the VAS scores for pain, function, and quality of life and the first 3 Retro Function questions. Consequently, we concluded that the Retro Function questions are a valid tool for retrospective assessment of the outcome of rotator cuff repair
Statistical Analysis Paired t tests were used to compare baseline and postoperative outcomes (DASH, SST, VAS pain, function, and quality of life). Spearman correlations were utilized to compare retrospective function (Retro Function 1-11) with the improvement from baseline of the DASH, Simple Shoulder Test, and VAS pain, function, and quality of life. Spearman correlations were also used to compare Retro Function (1-11) and the improvement from baseline of the DASH, SST, and VAS (pain, function, and quality of life) with final postoperative patient satisfaction (VAS Satisfaction Score). Spearman correlations were found to have little or no relationship with Rs values between 0 and .25, a fair relationship with Rs values between .25 and .5, a moderate to good relationship with Rs values between .5 and .75, and a very good or excellent relationship with Rs values greater than .75. P values < .05 were considered statistically significant.
856
Tashjian et al
J Shoulder Elbow Surg November/December 2008
Table III Baseline, postoperative, and improvement from baseline of DASH, SST, and VAS scores Outcome score
Baseline score
Postoperative score
Improvement
P value
DASH SST VAS pain VAS function VAS quality of life
41.79 6 17.1 4.00 6 2.8 6.13 6 2.1 6.27 6 2.1 6.00 6 2.3
15.42 6 18.4 9.92 6 2.6 1.60 6 2.4 1.64 6 2.2 1.70 6 2.5
26.37 6 19.6 5.71 6 2.7 4.27 6 3.0 4.44 6 3.0 4.06 6 3.2
< .01* < .01* < .01* < .01* < .01*
Mean and standard deviations for baseline, postoperative and improvement from baseline DASH, SST, and VAS (pain, function, and quality of life) scores. P values represent t tests comparing baseline and postoperative scores. *represents significance (P < .05).
Table IV Retrospective outcome measures Retro Function question
Mean score ± standard deviation
#1 – shoulder pain #2 – shoulder function #3 – quality of life #4 – sleep comfortably #5 – perform usual job #6 – exercise/play sports #7 – housework/yardwork #8 – shoulder symptoms #9 – activities overhead #10 – activities at waist #11 – shoulder motion
6.33 6 1.1 6.25 6 1.1 6.03 6 1.3 6.02 6 1.2 5.93 61.4 5.87 6 1.4 5.98 61.3 6.10 61.2 5.98 6 1.3 6.02 61.2 6.11 6 1.3
Table V Correlations between patient satisfaction and improvement from baseline of the DASH, SST, and VAS scores Outcome parameter
Mean and standard deviations for retrospective outcomes (Retro Function questions 1-11). Range of scores is from 1 (lowest possible score) to 7 (highest possible score).
RESULTS Patient satisfaction with the result of their treatment was generally high. The mean VAS Satisfaction Score was 8.76 6 2.4. There was statistically significant improvement in all of the outcome parameters, including VAS for pain, function, and quality of life, the SST, and the DASH (see Table III). The mean retrospective scores all demonstrated that the patients thought that their result was greater than ‘‘better’’ (see Table IV). Improvement in shoulder pain had the highest retrospective score, while improvement in the ability to exercise or play sport activities had the lowest score. Postoperative patient satisfaction (VAS Satisfaction Score) was significantly correlated with improvement in VAS shoulder function (P ¼ .005), VAS quality of life (P ¼ .019), and the SST (P ¼ .036) (Table V). However, the Spearman correlation coefficients (Rs) ranged from .192 to .256, indicative of a fair correlation at best. There were no significant linear associations between postoperative satisfaction and the improvement in VAS pain and the DASH scores (P > .05). There were statistically significant correlations between the postoperative patient satisfaction (VAS Satisfaction Score) with all 11 of the retrospective
VAS pain VAS function VAS quality of life DASH SST
Correlation coefficient (Rs)
P value
.178 .256 .217 .151 .192
.055 .005* .019* .104 .036*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing patient satisfaction (VAS satisfaction score) and improvement from baseline of shoulder function (SST, DASH, VAS function), quality of life (VAS quality of life), and pain (VAS pain). *represents significance (P < .05)
functional questions (see Table VI). The Spearman correlation coefficients (Rs) ranged from .406 to .577. The strongest correlation was between satisfaction and question #1 (shoulder pain), while the weakest correlation was with question #5 (ability to perform usual job). The Spearman correlation coefficients comparing satisfaction and the retrospective outcomes were substantially greater than the corresponding correlations comparing satisfaction with the prospective outcomes (Table VI vs. Table V). There were also statistically significant correlations between postoperative patient satisfaction (VAS satisfaction) and the absolute scores of the outcome measures (VAS SST, DASH) at follow-up (Table VII). There were statistically significant correlations between the improvement in VAS scores for pain, function, and quality of life and the first 3 retrospective questions: pain (Rs ¼ .226; P ¼.01), shoulder function (Rs ¼ .248; P ¼ .006), and quality of life (Rs ¼ .248; P ¼ .0063). Improvement in the SST was significantly correlated with all 7 retrospective functional questions to which it was compared (Rs ¼ .204-.281) (P ¼ .004-.026) (Table VIII). Because the SST only specifically relates to Retro Function questions 1, 2, 4, 5, and 9-11, correlations were only performed between these questions and the SST. These retrospective questions attempt to capture all components of the SST. The correlations
Tashjian et al
J Shoulder Elbow Surg Volume 17, Number 6
857
Table VI Correlations between patient satisfaction and retrospective outcomes
Table VIII Correlations between retrospective outcomes and improvement from baseline of the SST.
Retro Function question Correlation coefficient (Rs) P value
Retro Function question Correlation coefficient (Rs) P value
#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11
#1 #2 #4 #5 #9 #10 #11
.577 .543 .496 .413 .406 .478 .466 .567 .548 .483 .489
< .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001* < .001*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing patient satisfaction (VAS Satisfaction Score) and retrospective outcomes after rotator cuff repair (Retro Function questions 1-11). *represents significance (P < .05).
Table VII Correlations between patient satisfaction and final postoperative DASH, SST, and VAS scores Outcome parameter VAS pain VAS function VAS quality of life DASH SST
Correlation coefficient (Rs)
P value
.604 .584 .621 .434 .400
< .001* < .001* < .001* < .001* < .001*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing patient satisfaction (VAS Satisfaction Score) with final postoperative shoulder function (SST, DASH, VAS function), quality of life (VAS quality of life), pain (VAS pain), and general health status (SF-36). *represents significance (P < .05)
between the retrospective function and prospective SST (range of Rs: .204-.281;Table VIII) were lower than the correlations between the retrospective function and postoperative satisfaction (range of Rs: .406-.577; Table VI). Improvement in the DASH was significantly correlated with all 8 retrospective functional questions to which it was compared (P ¼ .001-.005) (Table IX). Because the DASH only specifically relates to Retro Function questions 1-2 and 6-11, correlations were only performed between these questions and the DASH. Again, correlations between the retrospective function and prospective DASH (range of Rs: .257-.346;Table IX) were lower than the correlations between the retrospective outcome and postoperative satisfaction (range of Rs: .466-.577; Table VI). DISCUSSION In this study, we found that patient satisfaction after rotator cuff repair correlated with both prospective
.230 .233 .204 .281 .218 .263 .212
.011* .011* .026* .002* .017* .004* .020*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing the improvement from baseline of the Simple Shoulder Test (SST) and retrospective outcomes (Retro Function questions 1, 2, 4, 5, and 9-11). *represents significance (P < .05).
Table IX Correlations between retrospective outcomes and improvement from baseline of the DASH Retro Function question Correlation coefficient (Rs) P value #1 #2 #6 #7 #8 #9 #10 #11
.264 .259 .346 .313 .306 .269 .292 .257
.004* .004* < .001* .001* .001* .003* .001* .005*
Spearman correlation coefficients (Rs) and P values for Spearman correlations comparing the improvement from baseline of the DASH and retrospective outcomes (Retro Function questions 1, 2, and 6-11). *represents significance (P < .05).
and retrospective outcome measures. However, the retrospective measures had much better correlation with patient satisfaction than the prospective measures. This suggests that retrospective self-assessed patient outcome is a better method for assessing outcomes that relate to patient satisfaction with the results of rotator cuff repair. Similarly, we also found that retrospective outcome assessment has only fair correlation with prospective assessments at best. These findings demonstrate that the results of retrospective and prospective outcome assessment are not the same, and that improvement does not correlate as well with the current status of patient satisfaction after rotator cuff repair. The emphasis on evidenced-based medicine has led to the classification of published clinical studies according to the level of evidence. Within this schema, prospective studies are afforded greater strength or value than retrospective studies. Despite this preference, the comparative validity or clinical relevance of prospective and retrospective studies has not been well studied. There are only a few studies that have
858
Tashjian et al
addressed the controversy between prospective and retrospective methodologies.5,12 Unfortunately, prospective data can be difficult to acquire and use in daily practice. Retrospective study is very appealing because it is easier to perform. Nevertheless, several authors have expressed concern about the use of retrospective instruments. Recall bias and guessing, when recall is deficient, have been cited as pitfalls of transition questions.8 There is a potential tendency for subjects to recall greater change than actually occurred, especially after surgical intervention.3,11 Several authors have challenged the use of retrospective measures claiming that they reflect follow-up status rather than a true change over time.10,12 Schmitt et al studied 211 patients with upper extremity musculoskeletal problems who were treated with physical therapy.12 They evaluated them at baseline and at 3 months follow-up with a number of outcomes measures. They found moderate correlations between prospective and retrospective outcomes measures (correlation coefficients between .57 and .66). However, because the correlations between the retrospective global ratings of change did not correlate well with the baseline scores, they concluded that retrospective measures were a less valid measure of change over time. Consequently, their conclusions suggest that retrospective measures are not valid outcome assessment tools. In contrast, we believe that it is more appropriate to conclude that retrospective and prospective outcome assessments have differing purposes. Fischer et al studied 202 patients who had chronic hip or knee arthritis.5 The outcomes of 3 different interventions (patient education, medical treatment, or surgery) were assessed. They found that when the change in outcome was small (education), prospective assessment (serial measures) correlated poorly with retrospective assessment. The correlations between retrospective and prospective outcome were stronger for treatments that produced a greater change (medication, surgery). They also demonstrated that retrospective measures were more sensitive to change than serial measures, and that they correlated more strongly with the patients’ satisfaction. Our results are not fully consistent with these findings, as we found that the correlations between the retrospective and prospective outcome assessment of rotator cuff repair were statistically significant, but not particularly strong. Nevertheless, we also found that the correlation between patient satisfaction and the retrospective assessment was stronger than the correlation between satisfaction and the prospective assessment of change in outcome. Despite apparently similar findings, that retrospective assessments correlate poorly with prospective or serial assessment, Fischer and Schmitt expressed con-
J Shoulder Elbow Surg November/December 2008
flicting conclusions.5,12 Schmitt concluded that retrospective assessment should not be used in outcome assessment because of the poor correlation. In contrast, Fischer concluded that retrospective assessments should be included in outcome evaluations, because they provide important information that is not well reflected by prospective outcome assessment. Consequently, Fischer recommended that prospective serial measures be supplemented with retrospective assessments. Our study of rotator cuff repair similarly demonstrates a moderate correlation between patient satisfaction and retrospective assessment and only a weak correlation between retrospective and prospective assessment. Because orthopaedic surgery deals primarily with quality of life issues and not life and death issues, final patient satisfaction, perhaps regardless of the determinants, should be a primary goal of treatment. Patient satisfaction may be affected by nonanatomic factors other than the surgical procedure, including the patient-physician relationship. While patient satisfaction may not be as well-defined an outcome measure as tendon healing, motion, or strength, it is still a very relevant and important outcome measure in orthopaedics. Our study enabled us to recognize that serial improvements in functional outcomes may not be the only or best method for predicting patient satisfaction with the outcome of rotator cuff repair. This study demonstrates that patient satisfaction and serial improvement measure 2 very different aspects of outcome. Consequently, retrospective assessment may aid in supplementing prospective assessments, as it appears to provide a better reflection of a patient’s perception of the success of rotator cuff repair, even though it does not correlate as well with the prospectively determined improvement in outcome. This study has limitations that may have biased our interpretation of the results. The range of outcome scores and patient satisfaction were relatively narrow. Most patients had a high satisfaction with outcome, and the outcomes were generally good. The results of the analysis might differ if the patient cohort included some with more varied outcomes.Another limitation is the high percentage of subjects lost to follow-up. In addition, we made assumptions about the assessment of patient satisfaction and retrospective functional assessment. We assume that use of a visual analog scale to determine satisfaction, while commonly utilized, is the best method to determine final patient satisfaction. Similarly, we have attempted to create retrospective questions that capture many aspects of the prospectively derived outcome data, although there may be better methods of questioning regarding retrospective outcomes. Without more extensive study, we cannot be sure that our methods are valid for the population that we studied.
J Shoulder Elbow Surg Volume 17, Number 6
We found that patient self-assessed outcome status after rotator cuff repair has better correlation with patient satisfaction, compared to the correlation between patient satisfaction and outcome change. The discrepancies between the correlations of satisfaction with prospective and retrospective assessment, which we identified, strongly suggest that the determinants of patient self-assessed outcomes after rotator cuff repair include factors and biases other than the actual change in self-assessed functional status. The relative importance of patient satisfaction and retrospective assessment of change compared to prospective assessment of change needs to be further elucidated.
REFERENCES
1. American Academy of Orthopaedic Surgeons: Disabilities of the Arm, Shoulder, and Hand. Rosemont, Illinois, American Academy of Orthopaedic Surgeons, 1995. 2. American Academy of Orthopaedic Surgeons: Musculoskeletal Outcomes Data Evaluation and Management System. Rosemont, Illinois, American Academy of Orthopaedic Surgeons, 1997. 3. Aseltine RH, Carlson KJ, Fowler FJ, Barry MJ. Comparing prospective and retrospective measures of treatment outcomes. Med Care 1995;33:AS67-76.
Tashjian et al
859
4. Beaton D, Richards RR. Assessing the reliability and responsiveness of 5 shoulder questionnaires. J Shoulder Elbow Surg 1998;7: 565-72. 5. Fischer D, Stewart AL, Bloch DA, Lorig K, Laurent D, Holman H. Capturing the patient’s view of change as a clinical outcome measure. JAMA 1999;282:1157-62. 6. Guyatt GH, Norman GR, Juniper EF, Griffith LE. A critical look at transition ratings. J Clin Epidemiol 2002;55:900-8. 7. Guyatt GH, Townsend M, Berman LB, Keller JL. A comparison of Likert and visual analogue scales for measuring change in function. J Chron Dis 1987;40:1129-33. 8. Herrmann D. Reporting current, past, and changed health status: what we know about distortion. Med Care 1995;33(Suppl): AS89-94. 9. Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: The DASH (Disabilities of the arm, shoulder, and hand). The Upper Extremity Collaborative Group. Am J Ind Med 1996;29:602-8. 10. Norman GR, Stratford P, Regehr G. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997;50:869-79. 11. Ross M. Relation of implicit theories to the construction of personal histories. Psychol Rev 1989;96:341-57. 12. Schmitt J, DiFabio RP. The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil 2005; 86:2270-6. 13. Soohoo NF, McDonald AP, Seiler JG, McGillivary GR. Evaluation of the construct validity of the DASH questionnaire by correlation to the SF-36. J Hand Surg Am 2002;27:537-41. 14. Blinded for Review.