Use of goal attainment scaling in measuring clinically important change in cognitive rehabilitation patients

] Clin Epidemiol Vol. 50, No 5, pp. 581-588, Copyright 0 1997 Elsevier Science Inc.

0895.4356/97/$17.00 PII SO895-4356(97)00014-O

1997

ELSEVIER

Use of Goal Attainment Scaling in Measuring Clinically Important Change in Cognitive Rehabilitation Patients Kenneth ‘DIVISION

Rockwood,“*

Brenda Joyce,2 and Paul Stolee’13

OF GERIATRIC MEDICINE AND 2D~~~~~~~ OF PHYSICAL MEDICINE AND REHABILITATION, DALHOUSIE UNIVERSITY, HALIFAX, NOVA SCOTIA, CANADA; AND )SOUTHWESTERN ONTARIO REGIONAL GERIATRIC PROGRAM, LONDON, ONTARIO, CANADA

ABSTRACT.

Measuring the effectiveness of cognitive rehabilitation programs poses both conceptual and practical challenges. We compared several standardized outcome measures with goal attainment scaling (GAS) to assess their sensitivity to changes in health status in patients undergoing cognitive rehabilitation. GAS is a measurement approach that accommodates multiple individual patient goals, and has a scoring system which allows for comparisons between patients. Forty-four patients were evaluated. GAS yielded a mean 4.4 goals per patient. The mean gain in the GAS score was compared with the change in the Rappaport Disability Rating Scale, the Kohlman Evaluation of Daily Living Skills, the Milwaukee Evaluation of Daily Living, the KleinBell elimination scale and mobility scale, the Instrumental Activities of Daily Living Scale, and the Spitzer Quality of Life Index. Using a relative efficiency statistic, GAS proved more responsive than any other measure. The effect size statistic also demonstrated greater responsiveness to change with GAS compared with standard measures. GAS shows promise as a responsive measure in cognitive rehabilitation. This study replicates a similar study of GAS in frail elderly patients, suggesting that individualized measures may have broad merit in evaluating rehabilitation programs. J CLIN EPIDEMIOL 50;5:581-588, 1997. 0 1997 Elsevier Science Inc.

KEY WORDS.

Cognitive

rehabilitation,

change

measures,

goal attainment

scaling,

functional

capacity,

relative

efficiency

Measuring the effectiveness of cognitive rehabilitation programs poses both conceptual and practical problems. An important challenge arises from the heterogeneity of the patients, reflected in both the variety of the injuries, and the broad range of patients who can be affected. [l-5] Partly in consequence, a single outcome (such as returning to work) will not be applicable to all patients (e.g., those who did not work prior to their injury). For some patients, the ability to move about independently in a wheelchair will be a triumph; for others the same outcome would represent a failure. Coping with such wide variability in what constitutes a successful outcome has proven difficult for standardized measures, in which all patients are judged according to the same criteria. This problem is similar to that faced in the evaluation of specialized geriatric interventions, which also deal with heterogenous populations and many goal areas. Recently, we studied the measurement properties of an individualized

assessment instrument known as goal attainment scaling [6] (GAS) as a means of evaluating outcomes in elderly patients [7,8]. As pointed out by Laths [9] in an accompanying editorial, with GAS each patient, in effect, receives his or her own outcome measure. Partly in consequence, GAS proved to be more responsive to change than any of the standard measures which we studied. The editorial by Laths raised an important question: could our results be replicated in other settings? In an earlier article we examined the reliability and validity of GAS in patients taking part in an inpatient cognitive rehabilitation program [IO]. In this article, we report the use of GAS as a measure of clinically important change in cognitive rehabilitation. Specifically, we assess the responsiveness (sensitivity to change) of GAS in comparison with standard measures now used to evaluate patient progress in cognitive rehabilitation programs.

METHODS ‘Address for correspondence: Kenneth Rockwood, Centre for Health Care for rhe Elderly, Queen Elizabeth I1 Health Sciences Centre, 5955 Jubilee Rd., Halifax, Nova Scotia, Canada, B3H 2El. Accepted for publication on 30 January 1997.

A prospective descriptive study of GAS was conducted in consecutive patients admitted in 1993-1994 to a specialized cognitive rehabilitation inpatient ward at the Nova Scotia

582

Rehabilitation Centre, a 104-bed tertiary rehabilitation hospital in Halifax, Nova Scotia. On admission and at discharge, standard assessments were completed on all patients. These assessments include the Rappaport Disability Rating Scale, (RDRS) [l l] widely endorsed as a measure which can be used to evaluate patient outcome following cognitive rehabilitation [12-141. We also included two other scales which had been used to evaluate cognitive rehabilitation programs [14,15]: the Klein-Bell Activities of Daily Living Scale, [15] (which was scored and reported in two components, the elimination score-ADLELIM-and the mobility score-ADLMOB) and the Lawton-Brody Instrumental Activities of Daily Living index (IADL) [16]. As alternate measures of function we chose the Milwaukee Evaluation of Daily Living Skills, (MEDLS) [17] the Kohlman Evaluation of Living Skills [18], (KELS), both of which had been validated in patients with cognitive impairment. These measures were completed by individual members of the clinical team, and reviewed at the team conference. In addition, the multidisciplinary rehabilitation team developed and scored goal attainment follow-up guides, and established a consensus rating for the Spitzer Quality of Life Index (QLI) [19], which, in our earlier study, had been the most responsive of the standard measures [8]. Goal attainment scaling is an individualized outcome measurement technique developed for use in the mental health field [6]. It has also been tested for use in assessment of traumatic brain injured individuals [10,20]. In brief, a sixstep process is followed (see Table 1). After the patient (in this case, a 22-year-old female) is assessed by the members of the multidisciplinary team, goal areas are selected (step one). In our experience, selection of the goal areas is typically not controversial, with good agreement between patients, families, and the team, although an indication of what can reasonably be attained within these goal areas is often, as discussed below, an important focus for education of patients and their families. In this case, the three goals included addressing problems in memory, dressing, and feeding. Consider the goal area of dressing, identified here as an area for intervention. The current level of function is noted; for example, “able to dress upper extremities with maximal cuing and set-up.” In this case, given that a worse plausible outcome existed (“no initiation of dressing,” the patient had been in this state early after the head injury) her level of function was scored as “- 1.” (If no clinically worse outcome is plausible, the admitting level can be scored as -2, as was done in the case of the patient’s memory problem. While complete anterograde amnesia would of course be a worse memory problem than that defined by the -2 level, the admitting level of memory problem was so severe that discharge to the community depended on a better performance. On this basis, the memory attainment at baseline was scored as -2.) Next (step two), the goal area is

K. Rockwood et al.

weighted, although this step is optional (in effect, the weights are then all set to equal 1). In the current example, the pervasive influence of memory impairment resulted in that goal being weighted as “2,” that is, twice as clinically important as dressing or feeding. For step three a follow-up time period is selected; in the current example, a typical follow-up time would be 4 weeks. In step four, the goal (level “0”) is set; for example “Able to dress upper and lower extremities with minimal cuing and set up.” In this patient’s case, it was felt that even if this could be achieved, she would still require assistance with her ankle-foot orthosis. This was therefore explained to the patient and her family, and made explicit in her goal. Setting precise goals often involves such education of patients and families, and negotiation with the team between what is desirable and what is feasible. Step five comprises completion of other scale levels, using possible outcomes including much less than expected ( - 2); somewhat less than expected (-l), somewhat more than expected (+ 1) and much better than expected (+2). To complete our example, much less than expected (-2) was no initiation of dressing; (+1) was “Able to dress upper and lower extremities, including ankle-foot orthosis, with minimal cuing”; (+2) was “Able to dress independently.” When the matrix of possible attainment levels is completed, the resulting form is referred to as the goal attainment follow-up guide. Step six is follow-up, at which time the GAS score is calculated, according to the following formula: Goal Attainment

Score = 50 +

10x (wtd 4.7c w; + .3(X wJ2

where w, is the weight assigned to the ith goal,and x, the numerical value (-2 to +2) of the attainment level of the ith goal. In effect, the composite goal attainment score (the sum of attainment levels for each goal area multiplied by their relative weights) is transformed into a standardized variable, with mean = 50 and SD = 10. Given that treatment results should exceed and fall short of expectations in relatively equal proportions, over a sufficiently large number of patients, there should be an approximately normal distribution of scores about 50 [21]. Our experience on geriatric assessment and geriatric rehabilitation services has closely approximated the theoretically expected results [7,8]. The denominator or variance term in the equation is calculated using a typical average intercorrelation, p, of the goal scales. Kiresuk and Sherman [6] suggest 0.3 as a typical intercorrelation (and 0.7 = 1 -p). In the current example, calculating the score based on these three goal areas only, the admission score is equal to 30. At discharge, with two goals met and one goal exceeded, the score is equal to 53.3. Earlier work has shown that GAS is usually moderately well correlated (r - 0.60) with a standard functional mea-

Goal Attainment

Scaling

583

TABLE 1. A sample goal attainment and dressing in a 22-year-old woman

scaling follow-up admitted after

Memory

Much less than expected (-2) Somewhat

less than expected (- 1)

guide: goal a traumatic

Unable (or refuses) to use memory book for daily scheduling J Uses memory book, but consistently requires cuing from another person Uses memory book with external cues (e.g., “beeping” watch) *

Somewhat better than expected (+ 1)

Able to spontaneously use memory book to follow a daily schedule

Much better than expected (+2)

Able to use memory book to set and change daily schedule, and for prospective planning

marks and asterisks

signify:

J = admission;

to address injury

(wt:2)

Program goal (0)

Note-check

setting brain

impaired

Goal

areas

Dressing

(wt:

memory,

1)

and

self-care

Feeding

No initiation of dressing; unable to dress independently Able to dress upper extremities with maximal cuing and set-up from another person J Able to dress upper and lower extremities with minimal cuing and set-up. Requires assistance with ankle-foot orthosis * Able to dress upper and lower extremities, including ankle-foot orthosis, with minimal cuing Able to dress independently

in feeding

(wt:

1)

Patient aspirates, with pneumonia, or choking Inconsistent use of strategies to compensate for disordered swallowing J Patient consumes pureed diet with consistent use of compensatory strategies Mechanical soft diet is consumed with consistent use of compensatory strategies * Full diet, safely consumed

* = discharge,

sure (in geriatric rehabilitation studies [7,8] the Barthel Index [22], and in the cognitive rehabilitation study [IO], the RDRS). To achieve a Pearson correlation of 0.60 between GAS and a standard functional measure, specifying the null hypothesis as r = 0.30, assuming alpha = 0.05 and beta = 0.90, and, using the method of Kraemer and Thiemann [23], we calculated the sample size requirement to be 36 subjects. To allow for drop-out due to death and refusal, 44 patients were evaluated. To assess the reliability of GAS in this setting, completed GAS attainment levels were rated independently by the attending physician and by a staff nurse. The evaluation of the clinical importance of the changes captured by the measures was tested by correlating the results with the clinical global impression (CGI) of the attending physician based on her experience with patients in this program. The CGI was scored from 1 to 7, with 1 indicating a very unsuccessful clinical outcome, 4 indicating that a usual level of success had obtained, and 7 indicating a highly successful clinical outcome. The CGI rater had a detailed knowledge of the patient, and participated in the development of the followup guide, but was nevertheless blind to the actual GAS score when scoring the CGI. Data were analysed using the Statistix software package [24], supplemented by hand calculations. To assess responsiveness, the Relative Efficiency [25] and Effects Size [26,27] statistics were calculated. For calculation of RE, the RDRS is used as a standard. The relative efficiency statistic was

described by Liang et al. [25] for use in comparing health status changes in patients with arthritis, and is calculated as: RE = (thstand2. A score of 1 .OO = the same efficiency as the standard; > 1 .OO is more efficient, and cl.00 is less efficient than the standard. ES is calculated as: ES = post-treatment mean - pre-treatment pre-treatment standard deviation so that a larger number

represents

mean

a larger effect size.

RESULTS

The mean age of the 44 subjects was 29.2 years (range 2261); 25 were women. The average length of stay of patients was 90 days. The most common diagnoses were traumatic brain injury (28 cases), subarachnoid hemorrhage (9 cases) and postinfectious encephalitis (4 cases). All patients were medically stable, but showed at least moderate functional impairment on admission, with the most common impairments being in memory, organizational and problem-solving skills, language, and mobility. On average, 4.4 goals per patient were set using GAS. Of the 195 goals set, 130 (67%)

584

K. Rockwood et al.

TABLE 2. Mean charge”

-C SD scores of selected assessment instruments, Value indicating best factor

Measure Spitzer quality of life index Rappaport disability rating scale Klein-Bell activities of daily living scale, mobility Klein-Bell activities of daily living scale, elimination Kohlman evaluation of living scales Milwaukee evaluation of daily living scale Instrumental activities of daily living score Goal attainment scaling ‘In

GAS

the

best/worst

score

was calculated

correlation

Measure

IADL

KELS MEDLS

0.6308 0.5225 0.3455 -0.1264 -0.0582 0.2824 0.2894

QLI

RDRS ADLELIM ADLMOB GAS Abbreviations: living; QLI = Klein-Bell

IADL = Quality activity

matrix

and at dis-

Admission

Discharge

10

0

4.9 ? 1.4

6.8 t 1.5

0

30

6.8 t 4.6

4.6 t 3.4

68

0

49.8 k 24.4

59.0 t 17.9

47

0

37.5 2 15.4

42.3 t 9.0

18

0

4.9 t 5.5

8.1 2 5.9

80

0

43.4 f 20.1

55.2 ? 15.5

14 82

0 18

6.9 ? 4.4 29.5 ? 4.7

9.0 f 3.6 53.6 -+ 8.3

as the

were set initially at the - 1 level, and the remaining 65 at the -2 level. Table 2 reports the mean scores t SD and percent change for each of the measures described above. (As the RDRS uses higher values to indicate greater independence, whereas the other scales use higher values to indicate greater incapacity, we have indicated the maximum values of complete dependence and complete independence.) We used as the range for GAS, the scores for eight (the highest number of goals for any patient) maximally positive to eight maximally negative goals as the denominator in calculating the GAS percentage change. GAS showed the smallest standard deviations for both admission and discharge scores. Table 3 reports Pearson correlations between the mean changes in each measure. Most change scores are weakly to moderately correlated with each other. (The positive and

TABLE 3. Pearson

Value indicating worst factor

on admission

score

for +2/-2

on each

of eight

goals,

equally

weighted

negative correlations reflect whether higher scores indicate independence or incapacity.) Table 4 reports the relative efficiency (RE) and effect size (ES) of each of the measures which were assessed. By both tests, GAS, and to a lesser extent the Spitzer QL Index, are the most responsive measures. GAS also showed excellent inter-rater reliability; the ICC values were 0.95 for admission scoring, 0.95 for discharge scoring, and 0.93 for the change scores. The CGI scores were roughly normally distributed about 5 (range 3-7; skewness -0.26). Both the change scores and the scores at discharge were correlated with the CGI as an estimate of the clinical importance of these changes. For all measures except GAS, correlations with the CGI were at best modest (Pearson r < 0.45). The Pearson correlation coefficient between the CGI and the GAS change score

for change scores

KELS 0.4272 0.3151 0.0079 -0.2751 0.1112

0.2687

MEDLS

0.2672 -0.3851 0.2375 0.4249 0.5176

QLI

RDR

-0.3442 0.0141 0.2139 0.2638

-0.6102 -0.4771 -0.4722

= Instrumental activities of daily living; KELS = Kohlman evaluation of living skills; of life index; RDRS = Rappaport disability rating scale; ADLELIM = Klein-Bell activity of daily living scale, mobility; GAS = goal attainment scaling.

MEDLS of daily

ADLELIM

ADLMOB

0.5038 0.1765

0.4001

= Milwaukee evaluation living scale, elimination;

of daily ADLMOB

Goal

Attainment

585

Scaling

TABLE

4.

Relative

efficiency

and effect size of assessment instruments

Measure Spitzer quality of life index Rappaport disability rating scale Klein-Bell activities of daily living scale, mobility Klein-Bell activities of daily living scale, elimination Kohlman evaluation of living scales Milwaukee evaluation of daily living scale Instrumental activities of daily living score Goal attainment scaling

was 0.73, and between charge was 0.63.

the CGI and the GAS

score at dis-

DISCUSSION We found GAS to be responsive to clinically important change in patients in a cognitive rehabilitation program. GAS has been described in such programs both by our group [lo] and by Malec et al. [20] at the Mayo Clinic. Malec emphasized many advantages of GAS, including its value in monitoring patient progress, structuring team conferences, planning decision making, and communicating to families. These advantages were also obtained in our setting, suggesting that GAS can have a useful role in cognitive rehabilitation. As calculated by the methods of Kazis et al. [27] and Liang et al. [25], the measurement of responsiveness compares change with a variance estimate. Measures with low variance are therefore more responsive, and we thus considered the possibility that the apparent responsiveness of GAS actually reflects a floor effect. In the extreme, all starting goal areas could be set at - 2. In such a case, the variance (assuming equal weighting) would only arise from the number of goal areas set for each patient, and there would be no possibility for patients to deteriorate. In the current series however, two thirds of the goals were set at - 1. It is nevertheless the case that the initial variance is small. We tested whether the increased responsiveness simply reflected low variance at baseline by calculating effect size using the standard deviation of the change scores as the denominator. The results were similar, in that while the responsiveness of GAS was lower (the effect size fell from 5.11 to 2.57) other measures also showed smaller effect sizes, so that the ordering of GAS as the most responsive measure was not changed. The other effect sizes were calculated to be: RDRS = 0.92; ADLELIM = 0.48, ADLMOB = 0.66; IADL = 0.61; DELS = 0.81; MEDLS = 1.09; Spitzer QL Index = 1.29.

Another safeguard against floor effects is that GAS, in contrast to standardized measures, builds in specification of a range of plausible outcomes for each patient and for each

Relative efficiencv

Effect size

1.95 1.00 0.51 0.28 0.78 1.41 0.43 7.80

1.40 0.48 0.38 0.30 0.59 0.59 0.46 5.11

goal area. It may remain the case, however, that measurable change can occur even beyond a -2 (much worse than expected) outcome. Consider, for example, our patient, in whom under “memory,” -2 was defined as “unable (or refuses) to use memory book for daily scheduling.” Clearly, there are worse states than this (e.g., complete anterograde amnesia). From the point of view of the team and the family, however, all the worse states were equally bad; without progress to at least - 1, the patient would be unable to return home. In such a case, from a clinical standpoint, accurate measurement below this level is not required to inform judgments about the usefulness of the intervention. Some of the correlations between GAS and the change scores of the standard measures differ somewhat compared with the first report [lo] (e.g., GAS-MEDLS 0.52 in this report cfO.50; GAS-KELS 0.27 here cfO.004; GAS-QL 0.35 vs. 0.26). Of note, the GAS-RDRS [5] change score was 0.62 in the earlier report, on which the current sample size calculation was based, but only 0.47 here. In general, increasing the sample size will produce a more stable estimate of the change score correlations, as evidenced by a more narrow range in the current report (0.18-0.52) than in the first report (0.003-0.61). Over time, a stable estimate of the change score correlations is important, as it provides one means of monitoring the construct validity of GAS. Our plan is therefore to continue to examine trends in these correlations in future studies. These results also replicate our earlier study with GAS in geriatric rehabilitation, and point to the possibility that an individualized approach to patient evaluation may have broad merit. As Laths [9] has pointed out, an attractive feature of GAS is that it seems to be a replication of clinical reasoning and process. Two components are particularly important in this regard. The first is that goals can be individualized to patient needs, so that the same outcome (e.g., discharge in a wheelchair) can be judged as a success or failure depending on the patient’s circumstances. The second is that important aspects of the rehabilitation process that are not captured by measures of function can be evaluated using GAS. These include goals such as relief of pain and diminution of caregiver stress, as well as safe performance of func-

586

tion. In effect, the GAS follow-up guide provides not only an indication of the treatment plan, but also an indication of a plausible range of outcomes of treatment, with a priori specification of what constitutes worse than and better than expected outcomes. In this way, GAS can help to sharpen clinical reasoning. For these reasons, we believe that GAS meets the “sensibility” criterion proposed by Feinstein

[W. A practical advantage of the use of GAS is that it helps patients and their families to better understand the possible outcomes of care, and their roles in achieving these outcomes. As noted above, goal setting often involves education of patients and families, who typically have little understanding of what might be achieved by the cognitive rehabilitation process. Negotiation between the patients, their families, and the team about what is desirable and what is feasible can become an important part of the therapeutic intervention. For example, if a patient and his family seek a return to the premorbid role of father and chief earner, frustration at the inability to achieve this level of function may mean that lesser, but still important goals (such as independence in personal care and the ability to help with simple household chores) are not attained. This focus on a description of what patients can actually do at the end of the period of inpatient rehabilitation is why we chose to evaluate measures of function and health status rather than measures of psychological test performance. Individualized measures are not without problems, however. They are not necessarily free of arbitrariness. For example, we cannot know whether, had the patients in this study been assessed by another multidisciplinary team, that team would have produced the same inventory of goals. Such a determination was beyond the scope of the current study. This issue was addressed for physician assessments in an earlier study [7] in which we found that the GAS followup guides could be constructed reliably between two different physicians. Individualized measures require a willingness to invest time and effort in the measures at the outset. Not all health care providers will be comfortable with the explicit nature of the goal setting, which individualized measures entail. In addition, there is no guarantee (indeed, no expectation) that two patients who have each met their goals will have the same level of function at discharge. This obvious point often gives rise to the worry that teams will set goals that are too easy. There are several safeguards against this however. The first is inherent in the process: teams with consistently high scores are likely setting goals that are too easy to attain. The second is that the explicit nature of the goals allows for ready peer review, both by other team members when the goals are set, and through independent clinical audit. Related to this, discussion of goals with patients and families is another important check. Third, GAS need not supplant the use of standard scales, which can be used, over time, as bench marks-albeit crude ones. Changes in the

K. Rockwood et al.

bench mark scoring can then be readily investigated by conducting clinical audits of GAS follow-up guides for patients scoring below the bench mark changes. Finally, in their discussion of this potential problem, Kiresuk and Lund [21] note that the combination of independent goal setting by an intake team and independent goal scoring by a discharge team would provide an alternate means of making sure that the goals are not too easy. The latter was not a realistic possibility in our setting, where we have relied on peer review, supplemental use of standard scales, and the inherent properties of the formula (i.e., that consistently very high or very low scores may reflect poor goal setting) as routine safeguards. In experimental evaluations, blinding would obviate much of such bias. In this study, after the first 13 patients, the team set all goal area weights at 1. In part, this was because they found it was too time consuming to achieve consensus on whether one task was more important than another. As this step was optional, and as the team felt that it was difficult to discount a goal area without at least the implication of discounting the importance of intervention in that area, we did not insist on weighting. A post-hoc comparison of scores from this study with differential weighting, and with all weights set to 1, revealed no clinically important or statistically significant differences in the GAS scores, or in responsiveness scores. The numbers involved in these calculations were small however, and we are exploring the issue of weighting further in a separate study. A potential problem of GAS that is less easily dealt with is the confusion sometimes observed between what might be termed “process” goals, in comparison with “outcome” goals. For GAS to fulfill its potential as an outcome measure (and not just as a means to record a treatment plan) it is important that the team focus on the outcomes of care, and not simply on the processes by which these outcomes are achieved. For example, consider a patient in whom the communication goal is to “gesture function of pictured object.” The process by which this goal was achieved included an assessment by the speech therapist, instruction of the team by the therapist about consistency and reinforcement of therapy, and daily therapy sessions. There is sometimes a tendency therefore for the goal to be written as “speech therapy to assess, and perform daily therapy with patient and instruct team about reinforcement.” As necessary as this was to achieving the desired outcome, it is indistinguishable from equally intensive efforts which fail. It is therefore important that the goals be written in terms that are observable as the outcomes of care. We recognize, however, that the distinction can be a fine one. The “memory book” goal of Table 1, for example, is at first glance, arguably as much a process as it is an outcome. From the team’s standpoint, however, it is an important outcome. Achievement of this goal means that the patient has acquired a tool which can be put to many uses. In this instance, describing one outcome (mastery of the memory book) is more effi-

Goal

Attainment

Scaling

cient than evaluating each of the many applications in which its use lessens the patient’s disabilities. Individualized outcome measures used in program evaluation offer the important advantage that the team’s performance can be evaluated by efforts which are consistently in the interests of the patient. Too often standard goals can result in unsatisfactory outcomes for many patients. A common example is discharge to supervised care. Programs evaluated

on

their

ability

to discharge

patients

home

set

up dilemmas for those patients who will require more supervision: the team will be sanctioned for working in the patients’ best interests, and rewarded for working against those interests. The finding that, under a system which rewarded early discharge, a large number of patients were discharged while medically unstable [29] provides compelling evidence that outcome measures influence care. Individualized measures may provide a routine means of evaluating the multiple interventions for patients with many needs. Our work in two settings has shown that GAS is a useful means of evaluating specialized multidisciplinary interventions, and there are several recent references pointing to the merit of GAS in a variety of rehabilitation settings [20,30-321. Whether this is a property of GAS, or whether it can

be generalized

as the DISCAN Performance inquiry.

to other

method Measure

For

now,

GAS

individualized

measures,

[33] or the Canadian [34],

is an important

appears

such

Occupational area

to us to present

for

future

a practical

supplement to standard scales which, as Feinstein [35] has pointed out, often “omit the particular symptoms that are the focal concern to be addressed.” Moreover, GAS is an example of an instrument which uses the views of patients to supplement instruments developed by experts [36], and provides a means to address the abandonment by researchers of the use of clinical description in assessing changes in patient status [37]. Further work on individualized measures can help strengthen the link between the processes and outcomes of care.

References Levine H. Cognitive rehabilitation: Unproved but promising. Arch Neurol 1990; 47: 223-224. Hachinski V. Cognitive rehabilitation. Arch Neurol 1990; 47: 224. Berrol S. Issues in cognitive rehabilitation. Arch Neurol 1990; 27: 219-220. Vople B, McDowell FH. The efficiency of cognitive rehabilitation in patients with traumatic brain injury. Arch Neurol 1990; 47: 220-222. 5. Ben-Yishay Y, Diller L. Cognitive rehabilitation in traumatic brain injury: Update and issues. Arch Phys Med Rehabil 1993; 74: 204-213. 6. Kiresuk TJ, Sherman RE. Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Comm Mental Health J 1968; 4: 443-453. K, Fox RA, Streiner DL. The use of goal 7. Stolee P, Rockwood attainment scaling in a geriatric care setting. J Am Geriatr Sot 1992; 40: 574-578.

8. Rockwood K, Stolee I’, Fox RA. Use of goal attainment scaling in measuring clinically important change in the frail elderly. J Clin Epidemiol 1993; 46: 1113-l 118. 9. Laths MS. The more thines change. J Clin Epidemiol 1993; 46: 1091-1092. ” 10. Joyce BM, Rockwood K, Mate-Kole C. Use of goal attainment scaling in brain injury in a rehabilitation hospital. Am J Phys Med Rehabil 1994; 73: 10-14. 11. Rappaport M, Hall KM, Hopkins K, Belleza T, Cope DN. Disability rating scale for severe head trauma: Coma to community. Arch Phys Med Rehabil 1982; 63: 118-123. 12. Rao N, Kilgore KM. Predicting return to work in traumatic brain injury using assessment scales. Arch Phys Med Rehabil 1992; 73: 911-916. 13. High WM, Boake C, Lehmkuhl LD. Critical analysis of studies evaluating the effectiveness of rehabilitation after traumatic brain injury. J Head Trauma Rehabil 1995; 10: 14-26. 14. Hall KM, Johnston MV. Outcome evaluation in traumatic brain injury rehabilitation. Measurement tools for a nationwide data system. Arch Phys Med Rehabil 1994; 75: SC-lO18. 15. Klein RM, Bell BJ. Self-care skills. Behaviour measurement with patients with Klein-Bell ADL scale. Arch Phys Med Rehabil 1982; 63: 335-338. 16. Lawton MP, Brody EM. Assessment of older people: Selfmaintaining and instrumental activities of daily living. Gerontologist 1969; 9: 179-186. 17. Leonardelli CA. The Milwaukee Evaluation of Daily Living Skills: Evaluation in Long-term Psychiatric Care. Thorofare, NJ: Slack, Inc.; 1988. 18. McGourty LK. Kohlman evaluation of living skills (KELS). In: Hemphill B, Ed. Mental Health Assessment in Occupational Therapy. Thorofare, NJ: Slack, Inc.; 1988. 19. Spitzer WO, Dobson AJ, Hall J, et al. Measuring the quality of life of cancer patients. A concise QL-Index for use by physicians. J Chron Dis 1981; 34: 585-597. 20. Malec JF, Smigielski JS, DePompolo RW. Goal attainment scaling and outcome measurement in postacute brain injury rehabilitation. Arch Phys Med Rehabil 1991; 72: 138-143. 21. Kiresuk TJ, Lund SH. Goal attainment scaling. In: Attkisson CC, Hargreaves WA, Sorensen JE, Eds. Evaluation of Human Service Programs. New York: Academic Press; 1978. 22. Granger CV, Albrecht GL, Hamilton BB. Outcome of comprehensive medical rehabilitation: Measurement of PULSES Profile and the Barthel Index. Arch Phys Med Rehabil 1979; 60: 145-154. 23 Kraemer HC, Thiemann S. How Many Subjects? Statistical Power Analysis in Research. Newbury Park, CA: Sage; 1987: Chap. 3. 24. Analytical Software, Inc. Statistix, version 4.1. St. Paul, MN: Analytic Software, Inc.; 1994. 25. Liang MH, Larson MG, Cullen KE, et al. Comparative measurement efficiency and sensitivity of five health status instruments for arthritis research. Arthritis Rheum 1985; 28: 542547. 26. Anderson JJ, Firschein HE, Meenan RF. Sensitivity of a health status measure to short-term clinical changes in arthritis. Arthritis Rheum 1989; 32: 844-849. 27. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care 1989; 27: S178-189. 28. Feinstein AR. Clinimetrics. New Haven, CT: Yale University Press; 1987. 29. Rogers WH, Draper D, Kahn KL, Keeler EB, Rubenstein LV, Kosecoff J, Brook RH. Quality of care before and after implementation of the DRG-based prospective payment system. A summary of effects. JAMA 1990; 264: 1989-1994.

K. Rockwood et al.

30. Palisano RJ, Haley SM, Brown DA. Goal attainment scaling as a measure of change in infants with motor delays. Physical Therapy

1992;

72: 432-437.

31. Ottenbacher KJ, Cusick A. Goal attainment scaling as a method of clinical service evaluation. Am J Occupational Therapy

1990;

44: 519-525.

32. Rockwood K, Stolee P, Howard K, Mallery L. Use of goal attainment scaling to measure treatment effects in an antidementia drug trial. Neuroepidemiol 1996; 15: 330-338. 33. Singh AC, Bilsbury C. Measurement of subjective variables: The Discan method. Acta Scandinavica Psychiatrica Suppl 1989; 347 (Suppl. 79): l-38.

34. Law M, Baptiste S, McCall MA, et al. The Canadian occupational performance measure: An outcome measure for occupational therapy. Can J Occupational Therapy 1990; 57: 8291.

35. Feinstein AR. Clinical judgement revisited: The distraction of quantitative models. Ann Int Med 1994; 120: 799-805. 36. Gill TM, Feinstein AR. A critical appraisal of the quality of quality of life measurements. JAMA 1994; 272: 619-626. 37. Feinstein AR. Clinical Judgement. Huntington, NY; Robert E. Kreiger; 1967.