- Email: [email protected]
Functional Independence Domains in Patients Receiving Rehabilitation in Skilled Nursing Facilities: Evaluation of Psychometric Properties
1089
ORIGINAL ARTICLE
Functional Independence Domains in Patients Receiving Rehabilitation in Skilled Nursing Facilities: Evaluation of Psychometric Properties Diane U. Jette, PT, DSc, Reg L. Warren, PhD, Christopher Wirtalla, BS ABSTRACT. Jette DU, Warren RL, Wirtalla C. Functional independence domains in patients receiving rehabilitation in skilled nursing facilities: evaluation of psychometric properties. Arch Phys Med Rehabil 2005;86:1089-94. Objective: To provide evidence for the validity of using FIM items to derive 4 domains of functional independence (mobility, activities of daily living [ADLs], sphincter management, executive function) in patients receiving rehabilitation in skilled nursing facilities (SNFs). Design: A retrospective analysis of secondary data using data from a privately owned administrative dataset. Setting: Seventy SNFs under contract with SeniorMetrix Inc health plan clients. Participants: Patients (N⫽7536) with a variety of conditions, covered by Medicare⫹Choice plans, and admitted to an SNF in 2002. Interventions: Not applicable. Main Outcome Measures: Item score distributions, corrected item-total correlations, factor correlations, internal consistency, and stage ceiling and floor effects for each hypothesized functional independence domain. Results: With the exception of 2 items, the items within a domain had similar standard deviations and distributions of items were not highly skewed. Four factors accounted for 73.4% of the variance in functional independence. Corrected item-total correlations ranged from .58 to .80 for the ADL domain, from .23 to .71 for the mobility domain, and from .78 to .88 for the executive function domain. The correlation between bowel and bladder management items was .84. Cronbach ␣ coefficients ranged from .76 for the mobility domain to .96 for the executive function domain. At admission, there were substantial floor effects for the sphincter management (34.4%) and mobility domains (43.1%) and ceiling effects for the executive function domain (26.7%). Conclusions: With a few exceptions, the items proposed for each functional independence domain met the criteria for supporting the validity of the domains. Key Words: Geriatric assessment; Rehabilitation; Skilled nursing facility. © 2005 by American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation
From the Physical Therapy Department, Simmons College, Boston, MA (Jette); and SeniorMetrix Inc, Nashville, TN (Warren, Wirtalla). At the time this study was initiated, Jette was affiliated with Boston University, Boston, MA. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated. Correspondence to Diane U. Jette, PT, DSc, Physical Therapy Department, Simmons College, 300 The Fenway, Boston, MA 02115, e-mail: [email protected]. Reprints are not available from the author. 0003-9993/05/8606-9289$30.00/0 doi:10.1016/j.apmr.2004.11.018
HE FIM INSTRUMENT IS used as a standard tool for T collecting data about patients’ functional performance on their entry into and discharge from an inpatient rehabilitation setting. The FIM is a measure of function in 13 motor areas and 5 cognitive areas. It contains 18 items, each rated on a 7-level scale, with 1 equating with total dependence and 7 equating to total independence. FIM scores are most often derived as a continuous total score (range, 18 –126) or as 2 continuous subscale scores for motor and cognitive abilities. Work by Stineman et al1 in 1997 suggested that the FIM could be viewed as a “multilayered, multidimensional” measure of function. They used factor analysis to derive dimensions that were more finely graded than the broad motor and cognitive dimensions. Although the dimensions they described differed slightly depending on the patients’ medical condition, 4 common domains of function emerged for 13 of the 20 medical conditions they examined. These domains included activities of daily living (ADLs) (FIM items for eating, grooming, bathing, dressing upper body, dressing lower body, toileting), sphincter management (FIM items for bladder management, bowel management), mobility (FIM items for bed, chair, and wheelchair transfer; toilet transfer; tub or shower transfer; walking or wheelchair mobility), and executive function (FIM items for comprehension, expression, social interaction, problem-solving, memory). In subsequent work, Stineman et al2,3 described a system that used each of the 4 domains of function to define profiles or stages of function. Each domain is comprised of 7 stages. Each stage represents a profile of scores within that functional domain based on FIM item scores. The stages suggest the average amount of assistance needed by the patient and the amount of effort required by the patient within each domain. For example, stage 1 in any domain equates with the patient being able to provide less than 25% of the effort required to accomplish the tasks in the domain; stage 4 indicates a minimal amount of assistance from another person and an effort of 75% on the part of the patient, and stage 7 indicates full independence. A patient must receive a designated minimum score on each item in the domain in order to be classified at a particular stage. For example, stage 4 in the mobility domain, which equates with needing a minimal amount of assistance, requires scores of at least 5 in chair and toilet transfers, at least 4 in tub transfer and walking or wheelchair mobility, and 1 in stairs. The functional domains described by Stineman1 were derived by using a sample of patients receiving care in inpatient rehabilitation facilities (IRF). Because a measurement instrument’s psychometric properties can vary depending on the sample of patients to which it is applied, the question arises as to the appropriateness of applying the 4 functional independence domains and stages to patients receiving rehabilitation in the skilled nursing facility (SNF) setting. SNFs are often chosen as the rehabilitation setting for patients who are moderately disabled yet need somewhat less intensive or comprehensive rehabilitation than that provided in IRFs.4 Patients receiving rehabilitation in SNFs are older and have lower cognitive and Arch Phys Med Rehabil Vol 86, June 2005
1090
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette
physical function on admission.5 Further, although factor analysis was applied to derive the domains, and internal consistency was tested,6 additional testing to determine the validity of the 4 domains was not reported. The purpose of the present study, therefore, was to examine the psychometric properties of the 4 domains of the FIM as described by Stineman.1 Specifically, we examined the validity of using the FIM items to derive 4 domains of function in patients receiving rehabilitation in SNFs. METHODS Design This study is a secondary data analysis using data from an administrative dataset compiled and owned by SeniorMetrix Inc. The company is privately held and has contracts with health plans to assist them to improve the delivery of rehabilitation services in SNFs for patients covered under Medicare⫹Choice reimbursement plans. Medicare⫹Choice is a plan that allows Medicare beneficiaries to choose among health maintenance organizations (HMOs), provider-sponsored organizations, and other fee-for-service plans that provide care under contract to Medicare. Under that program, HMOs contract with Medicare to provide the full range of Medicare benefits in return for monthly per-person payment rates. As of June 2002, of nearly 40 million Americans enrolled in Medicare, 13% had chosen to be in a Medicare⫹Choice managed care plan.7 More than 500,000 of the Medicare⫹Choice population are under care by providers contracting with SeniorMetrix health plan clients. This study was approved by the Institutional Review Board of Boston University. Sample The sample was derived from 9357 patients covered by Medicare⫹Choice plans who were admitted for short-term rehabilitation for the first time in 1 of 70 SNFs that were associated with SeniorMetrix in 2002. All facilities were in urban areas, and all but 3 were located in nursing home based, free-standing SNFs. Thirty-eight sites were in the northern Pacific region of the country, 17 were located in the southern Pacific region, 11 were in the mountain region, and 4 were in the mid-Atlantic region. We excluded 1064 patients who died during their SNF stay or who were admitted to an acute care facility from the SNF because, in our opinion, these patients likely had acute medical conditions that would affect their ability to attain functional improvements. We also excluded cases with negative values for time from impairment onset to SNF admission and cases with episode lengths of more than 100 days. This methodology resulted in a sample of 7526 patients. See table 1 for patient characteristics. Procedure Data were collected for all patients regarding demographics, medical complexity, length of stay, time from onset of primary condition to SNF admission, setting to which the patient was discharged, and functional status at SNF admission and at discharge. Functional status data included FIM scores on admission and discharge. Clinicians were trained and credentialed by SeniorMetrix in administering the FIM. The FIM instrument has good interrater reliability (intraclass correlation coefficient, ⬎.90).8 The validity of the FIM has been supported by its ability to discriminate among patients on the basis of age and comorbid status and among patients with different levels of Arch Phys Med Rehabil Vol 86, June 2005
Table 1: Patient Characteristics Characteristic
Value
Age, mean (95% CI) Male gender Ethnicity White, non-Hispanic African American Asian Native American Hispanic Other Impairment Stroke Neurologic, not stroke Orthopedic Cardiovascular/pulmonary Other Medical complexity score No other disease Inactive or irrelevant disease Active disease but not limiting Active and limiting disease Active and severely limiting disease Moribund Living in community before onset Discharged to community Days from onset to SNF admission, median (interquartile range) Length of stay, median (interquartile range)
76.3 (76.1–76.6) 2827 (37.6) 6344 (84.3) 469 (6.2) 233 (3.1) 3 (⬍1.0) 390 (5.2) 87 (1.2) 993 (13.2) 288 (3.8) 2711 (36.0) 1099 (14.6) 2435 (32.4) 95 (1.3) 197 (2.6) 1168 (15.5) 3850 (51.2) 2035 (27.0) 181 (2.4) 7277 (96.7) 6102 (81.1) 5 (5) 10 (13)
Abbreviation: CI, confidence interval. NOTE. Values are n (%) unless otherwise noted.
spinal cord injury and amputation level. Discharge destination is also related to discharge FIM score.9 Data Analysis To examine evidence for construct validity of the 4 FIM functional domains, we performed several types of analyses using the admission FIM scores. Item-level analysis. First, we examined the mean, standard deviation (SD), and skewness for each item within a stage as well as the frequency of item responses. We described the items in this way to determine if the items within each stage might contribute to the same degree to the determination of stage, that is, had similar means, SDs, and response frequencies. Factor analysis. We used principle components analysis with varimax rotation to determine how items would cluster in a 4-factor solution. Items were considered to belong to 1 factor if they were fairly well correlated within a factor (ⱖ.40) and less well correlated with other factors (⬍.40). Corrected item-total correlations. We looked for evidence that items in each domain as suggested by Stineman et al1-3 measured a common underlying construct by examining corrected item-total correlations (Spearman ). These analyses determined the correlations between the score for each item and the total of item scores for all other remaining items in the domain. For example, within the ADL domain, we determined the correlation between the score for the grooming item and the total of scores for eating, dressing the upper body, dressing the lower body, toileting, and bathing items. A minimum of r equal to .40 was deemed necessary to consider the items appropriate to combine into 1
1091
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette Table 2: Item-Level Analysis Stage/Item Score
Response Frequency (%)
Domain/Item
Mean ⫾ SD
Skew
1
2
3
4
5
6
7
ADL Eating Grooming Bathing Upper-body dressing Lower-body dressing Toileting Sphincter management Bowel management Bladder management Mobility Chair transfer Toilet transfer Tub transfer Walking or wheelchair mobility Stairs Executive function Comprehension Expression Social interaction Problem solving Memory
2.63⫾1.17 5.34⫾1.69 4.34⫾1.53 2.44⫾1.21 3.79⫾1.53 2.48⫾1.28 2.78⫾1.51 3.59⫾2.37 4.06⫾2.30 3.68⫾2.39 1.68⫾0.72 3.06⫾1.28 2.82⫾1.42 1.75⫾1.20 1.62⫾1.23 1.07⫾0.50 4.76⫾1.95 5.55⫾1.66 5.67⫾1.71 5.76⫾1.66 4.83⫾1.91 5.05⫾1.93
0.51 ⫺1.05 ⫺0.34 0.74 ⫺0.02 0.93 0.62 0.22 ⫺0.07 0.18 1.29 0.09 0.19 1.43 2.27 7.83 ⫺0.47 ⫺1.01 ⫺1.18 ⫺1.27 ⫺0.42 ⫺0.61
17.8 6.0 5.5 25.6 8.5 23.6 25.0 34.4 23.2 32.8 43.1 15.0 26.9 65.5 70.5 97.0 7.9 2.4 3.1 2.5 6.1 5.7
30.8 3.0 8.6 30.4 13.8 36.3 23.6 9.5 11.2 10.0 48.2 17.3 14.4 10.0 17.0 1.2 8.7 4.8 4.8 4.6 8.8 8.4
29.3 3.5 10.5 25.2 16.3 18.8 18.6 6.9 8.0 6.8 6.5 27.0 21.0 11.5 1.4 0.3 11.2 6.9 5.7 5.4 10.9 9.0
16.5 9.1 23.2 13.8 27.5 14.5 20.2 9.9 10.5 9.6 1.7 31.3 28.5 10.3 4.9 0.6 12.6 9.9 8.5 7.9 15.3 13.0
4.3 29.5 35.1 3.4 24.3 4.5 7.9 9.9 10.9 9.7 0.2 6.9 6.8 2.1 4.1 0.6 17.4 15.5 13.6 12.6 18.4 15.2
1.0 14.3 7.5 0.9 4.1 1.4 2.4 10.8 14.0 9.7 0.1 1.5 1.6 0.5 1.6 0.2 15.6 17.8 14.9 15.4 10.2 12.9
0.3 34.6 9.5 0.6 5.4 1.0 2.2 19.0 22.2 21.4 0.0 0.9 0.8 0.2 0.5 0.1 26.7 42.6 49.5 51.6 30.3 35.8
domain (convergent validity).10 We also sought evidence for discriminant validity by examining whether the items were more highly correlated to items within their domain than they were with items in the other domains. We also determined whether the correlations between domain scores were less than the estimates of internal consistency for each domain.11 Internal consistency. To examine the reliability of estimates that might be provided by total scores in each of the 4 domains, we derived Cronbach ␣ coefficients. We considered an ␣ of .70 to be sufficient to allow group comparisons within a domain.12 Stage ceiling and floor effects. Finally, combining the FIM items into the domains as described by Stineman,3 we determined the functional independence stages for patients in each domain and examined the ceiling and floor effects for the staging system at admission and discharge. We considered that in order for the measurement of a stage to be useful, no more than 20% of patients’ measurements should show ceiling or floor effects. Ceiling and floor effects show the limited ability of a measurement to discriminate among patients’ function at the high or low ends of the scale, respectively.
(table 3). Only these 4 factors had eigenvalues above 1.0. The ADL factor included grooming, bathing, dressing the upper body, dressing the lower body, toileting, toilet transfers, and chair transfers. For convenience, we term this factor “ADL and low-level physical function.” These 7 items accounted for 24.5% of the variance in functional independence. The sphincter management factor included bladder management and bowel management and accounted for 12.2% of the variance in functional independence. The mobility factor included only 2 items, walking or wheelchair mobility and stairs, and accounted for only 8.9% of the variance. Our term for this factor is “high-level physical function.” The executive function domain accounted for the greatest proportion of variance, 27.7%. The 6 items included in this factor were comprehension, expression, problem solving, social interaction, memory, and eating.
Item-Level Analysis Item means and distributions are shown in table 2. All response choices were endorsed for each item. With the exception of 2 items, the items within a domain had similar SDs and distributions of items were not highly skewed. Two items in the mobility domain, walking or wheelchair mobility and stairs, were skewed to the right (skewness⫽2.27, 7.83, respectively). The SD (.50) of the stairs item was small as compared with the SD of the other mobility items.
Corrected Item-Total Correlations Corrected item-total correlations ranged from .58 to .80 for the ADL domain, from .23 to .71 for the mobility domain, and from .78 to .88 for the executive function domain (table 4). The correlation between bowel and bladder management items was .84. These correlations were, for the most part, greater than the correlations with items in different domains. The tub transfer and toileting items correlated nearly as well with the mobility domain items (r⫽.31, r⫽.71, respectively) as they did with the ADL domain items (r⫽.32, r⫽.73, respectively), and the eating item correlated nearly as well with the executive function domain items (r⫽.55) as with the ADL domain items (r⫽.58). The tub transfer item did not correlate well (r⬍.40) with any domain. Similarly, the stair item did not correlate well with any domain, including the hypothesized mobility domain (r⫽.23). All ␣ coefficients were higher than correlations between the domains (table 5).
Factor Analysis The 4 factors derived by principle components analysis accounted for 73.4% of the variance in functional independence
Internal Consistency Cronbach ␣ coefficients ranged from .76 for the mobility domain to .96 for the executive function domain (see table 5).
RESULTS
Arch Phys Med Rehabil Vol 86, June 2005
1092
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette Table 3: Factor Analysis Factor
Low-Level Physical Function (ADL)
Sphincter Management
High-Level Physical Function (mobility)
Executive Function
Lower-body dressing Bathing Upper-body dressing Toileting Toilet transfer Grooming Chair transfer Tub transfer* Bladder management Bowel management Stairs Walking or wheelchair mobility Comprehension Memory Expression Problem solving Social interaction Eating*
.82 .77 .75 .74 .65 .64 .63 .51 .24 .24 .04 .30 .15 .15 .17 .18 .15 .46
.20 ⫺.02 .22 .39 .39 .20 .36 ⫺.06 .85 .84 ⫺.02 .27 .14 .14 .16 .16 .15 .17
.18 .06 ⫺.01 .18 .37 ⫺.06 .38 .40 .06 .06 .81 .62 .05 .06 .02 .06 .06 ⫺.12
.11 .19 .35 .17 .12 .49 .16 .03 .28 .33 .05 ⫺.002 .91 .91 .90 .88 .86 .58
NOTE. Values derived by principle component analysis with varimax rotation. Boldface indicates items most highly correlated with the factor. *Correlation ⱖ.40 for more than 1 factor.
Stage Ceiling and Floor Effects At admission, we found substantial floor effects for the sphincter management (34.4%) and mobility domains (43.1%) and ceiling effects for the executive function domain (26.7%) (see table 2).
Table 4: Corrected Item-Total Correlations Domain/Item
ADL Eating* Grooming Bathing Upper-body dressing Lower-body dressing Toileting* Sphincter management Bowel management Bladder management Mobility Chair transfer Toilet transfer Tub transfer† Walking or wheelchair mobility Stairs† Executive function Comprehension Expression Social interaction Problem solving Memory
ADL
Sphincter Management
Mobility
Executive Function
.58 .74 .60 .80 .74 .73
.42 .49 .31 .50 .43 .57
.35 .48 .46 .57 .66 .71
.55 .53 .32 .45 .29 .36
.54 .55
.84 .84
.47 .48
.44 .47
.65 .65 .32 .38
.48 .51 .17 .30
.71 .70 .31 .46
.33 .30 .09 .11
.16
.13
.23
.06
.48 .49 .45 .51 .48
.44 .45 .43 .46 .45
.26 .27 .28 .30 .27
.88 .84 .78 .88 .84
NOTE. Boldface indicates similar correlations with more than one factor or low correlations with all factors. *Correlates to similar extent with 2 domains. † Item does not correlate well (⬍.40) with any domain.
Arch Phys Med Rehabil Vol 86, June 2005
DISCUSSION Although Stineman et al2,3 did not propose adding the scores of items to create the functional independence domain stages, we used the types of psychometric testing one might employ to determine the legitimacy of combining item scores in this way to form domains. The findings of this study largely support the derivation of 4 domains of function based on the FIM for patients receiving rehabilitation in the SNF setting. With a few exceptions, the items within each hypothesized domain meet the criteria suggested for combining items into subscales.10-14 Item-Level Analysis For the most part, the items within each of the domains as originally suggested by Stineman et al,6 showed similar distributions, suggesting the legitimacy of combining the items into 1 domain. The stairs item is an exception. The responses to the item were highly skewed, with more than 95% of patients at the level of complete dependence. Stineman et al14 reported similar results, with the stairs item showing a low mean for patients in 16 of 20 impairment groups receiving rehabilitation in acute care settings. Equivalence of item means and item-response distribution are important in determining whether items may be legitimately combined into 1 summary score without weighting.10 Although Stineman2 did not endorse the summation of scores to attain domain stage assignments, this criterion seems relevant if a domain stage score is to be derived from scores on several individual items.
Table 5: Domain Correlations and Internal Consistency Domain
ADL
Sphincter Management
Mobility
Executive Function
ADL Sphincter management Mobility Executive function
(.89) .57 .67 .52
(.91) .50 .47
(.76) .29
(.96)
NOTE. Alpha coefficients are in parentheses.
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette
Factor Analysis We purposely constrained the factor analysis to 4 domains to explore the validity of the stages suggested by Stineman.2 Nevertheless, given a criterion of eigenvalues greater than 1 in determining viable factors, our findings support the appropriateness of aggregating FIM scores into 4 domains of functional for patients receiving rehabilitation in the SNF setting. The items within each domain, as determined by the factor analysis, however, varied from those previously proposed.2 All items related to transfers correlated more highly with the items that Stineman et al1-3 included in the ADL domain than with the mobility domain items of walking and stairs. Based on this finding, we conducted a factor analysis forcing extraction of 5 factors (data not shown). We found that the transfer items were related to 1 factor, while the daily activities related to personal care, including eating, were related to another factor. Walking and stairs remained a separate factor as in our 4-factor analysis. In our opinion, walking and stairs tend to require a high level of motor function, including strength, balance, endurance, and coordination, as compared with transfer activities, which require a lower level of function. Our analysis suggests that the first factor in our 4-factor analysis, which includes transfers, may be capturing the concept of low-level physical function, rather than strictly basic daily activities related to personal care. Corrected Item-Total Correlations The appropriateness of combining items into 1 domain can be also assessed by determining if an item has a higher correlation with other items in its hypothesized domain than with items in other domains (⬎2 standard errors [SEs]; in this study, .023).11,13 The result of this test of item discrimination is not definitive if the difference is smaller than 2 SEs and the test fails if the item correlation with another domain is higher than its correlation with the hypothesized domain. Similar to the results of the factor analysis, corrected item-total correlations suggested that transferring to the tub or shower was related to the ADL domain to a greater extent than to the mobility domain. Given that Stineman1-3 included the tub and shower transfer item in the mobility domain, we determined that this item failed to meet the test of item convergence and item discrimination. In our opinion, tub and shower transfer is intuitively related to the personal care activity of bathing oneself. Perhaps, for this reason, the item does not fall clearly into 1 domain. The item-total and item-other correlations of the toileting item are a possible failure of item discrimination. Although the correlation with the ADL domain was higher than that for the mobility domain, as proposed by Stineman,2 the difference was less than 2 SEs. In our opinion, the problem with this item may be very similar to that of the tub and shower transfer item. That is, although toileting represents a personal care activity, as defined by the FIM, it is linked intuitively to the ability to access the bathroom and commode. Finally, the eating item meets the requirements for item discrimination by a very narrow margin and factor analysis places the item in the executive function domain rather than the ADL domain. This finding could reflect the fact that we found the highest mean scores for eating and the items within the executive function domain. Eating may contain elements of chewing and swallowing as well as the action of moving the food to the mouth. It is possible that patients who are unable to communicate also have difficulty with chewing and swallowing—for example patients who are intubated or patients with brainstem infarction.
1093
Similar to the findings of Stineman et al,14 we found a relatively low corrected item-total correlation for the stairs item within the mobility domain (r⫽.23). This degree of correlation failed to meet the criterion for convergent validity (rⱖ.40). Factor analysis suggested that the stairs item, in combination with the walking item, was part of a high-level physical function domain. As suggested by our other findings, the stairs item is likely not measuring the same level of mobility as that required for transfers, explaining its low item-total correlation within its hypothesized domain. Additionally, the item scores showed very little variability with more than 95% of patients being totally dependent (score of 1), likely accounting for a low correlation. Internal Consistency We found that, although in 2 cases the item-total correlations did not meet the criterion for convergent validity, the domains showed adequate internal consistency. The mobility domain had the lowest internal consistency, reflecting the low itemtotal correlations of the tub transfer and stairs items and the low variation in the stair item. It has been suggested that a Cronbach ␣ of at least .90 is required to successfully apply scores to make decisions about an individual, and that an ␣ of at least .70 is required for comparing groups.12 If one were to combine the scores from the 2 sphincter management items or the 6 executive function items, as proposed by Stineman,2,3 these scales would meet the criterion for making decisions regarding individuals’ function in these domains. Stage Ceiling and Floor Effects The executive function and sphincter management domains showed substantial ceiling effects in this sample of patients in the SNF setting. Possibly, patients with poor executive function or lack of bladder and bowel control are not admitted for rehabilitation services because they are deemed unlikely to be able to participate in their own care. On the other hand, a measure that does not distinguish among patients at the top end of the scale may be insensitive to important improvements. Such ceiling effects may present problems when attempting to describe the benefits and outcomes of care, particularly care that addresses deficits in executive or sphincter function. The mobility domain showed important floor effects and was the only domain in which no patients were classified at stage 7. More than 90% of patients were in stage 1 or 2. This finding suggests that the mobility domain may not be able to distinguish among patients with differing levels of ability at the lower end of the continuum in mobility. For example, the walking item requires a score of 1 if the patient cannot walk at least 15m (50ft) and a score of 2 if the patient cannot walk at least 45m (150ft). It seems possible that patients entering SNF settings are able to walk with maximal or moderate assistance but for shorter distances than those required for scoring at higher levels. Similarly, Stineman14 suggested that the stairs item might be redefined to reduce the floor effect of that item. The limitations of this study include the use of secondary data. These data were generated for the major purpose of clinical and utilization management decision making and using them for other purposes requires cautious interpretation. Also, our sample included only patients covered through Medicare⫹Choice. The patients appeared to have basic demographic characteristics similar to a large national sample of patients treated in SNFs in 199815; however, they were less functional at admission as measured by the FIM. Our analyses also included a heterogeneous sample of patients in terms of their medical conditions. In previous work,1,14 authors have examined the psychometric performance of Arch Phys Med Rehabil Vol 86, June 2005
1094
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette
the FIM stratifying by type of medical condition, and the condition was found to affect the psychometric properties. Because the FIM is applied similarly to all patients on admission to rehabilitation, and therapists and other rehabilitation professionals often manage patients with a variety of medical conditions, we used the approach of determining the performance of the functional independence staging system across all patients in the SNF setting. In our opinion, the staging system described by Stineman et al2,3 is an important step in creating a measurement that is useful to clinicians and patients, that is, determining a stage can be done quickly and may be clinically useful across patients with the large variety of conditions encountered in the rehabilitation setting. Because the FIM is routinely used in practice, an important goal, in our opinion, is to determine if it can be used to derive an expression of functional independence that is meaningful in terms of making a prognosis and determining appropriate interventions for patients. For example, Feinstein16 has noted the inherent usefulness and simplicity of the Apgar score, stating that the variables represented in the Apgar score are clinically meaningful and can be understood and applied at the bedside. He also notes the value of identifying the various important individual clinical components in the scale. In our opinion, the staging system as proposed by Stineman,2,3 may be a first step toward allowing clinicians in rehabilitation to use the FIM items in a quick and inherently meaningful manner in patient management and establishing realistic goals, just as our colleagues in obstetrics and pediatrics use the Apgar score. CONCLUSIONS This study shows the validity of the functional independence stages, as proposed by Stineman3 in 2003, to patients in the SNF setting. Item-level analyses, factor analysis, item-total correlations, internal consistency, and ceiling and floor effects for each stage were examined. With a few exceptions, the items proposed for each functional independence domain met the criteria for supporting the convergent and discriminant validity of the domains. One suggestion for further exploration is to examine the psychometric performance of modified domains as suggested by our findings: sphincter management, executive function, ADL and low-level physical function and high-level physical function. Identifying easy-to-use, psychometrically sound, and clinically meaningful measures of functional independence is important for rehabilitation professionals in facilitating their management of patients. References 1. Stineman MG, Jette AM, Fiedler R, Granger CV. Impairmentspecific dimensions within the functional independence measure. Arch Phys Med Rehabil 1997;78:636-43.
Arch Phys Med Rehabil Vol 86, June 2005
2. Stineman MG, Ross RN, Fiedler R, Granger CV, Maislin G. Staging functional independence: validity and applications. Arch Phys Med Rehabil 2003;84:38-45. 3. Stineman MG, Ross RN, Fiedler R, Granger CV, Maislin G. Functional independence staging: conceptual foundation, face validity, and empirical derivation. Arch Phys Med Rehabil 2003;84: 29-37. 4. Gornick ME, Warren JL, Eggers PW, et al. Thirty years of Medicare: impact on the covered population. Health Care Financ Rev 1996;18:179-237. 5. Kramer AM, Steiner JF, Schlenker RE, et al. Outcomes and costs after hip fracture and stroke. A comparison of rehabilitation settings. JAMA 1997;277:396-03. 6. Stineman MG, Maislin G, Fiedler RC, Granger CV. A prediction model for functional recovery in stroke. Stroke 1997;28:550-6. 7. Centers for Medicare and Medicaid Services. Program information on Medicare, Medicaid, SCHIP, and other programs of the Centers for Medicare & Medicaid Services. June 2002. Available at: http://www.cms.hhs.gov/charts/series/sec3-b1.ppt. Accessed December 5, 2004. 8. Ottenbacher KJ, Hsu Y, Granger CV, Fiedler R. The reliability of the functional independence measure: a quantitative review. Arch Phys Med Rehabil 1996;77:1226-32. 9. Dodds TA, Martin DP, Stolov WC, Deyo RA. A validation of the functional independence measurement and its performance among rehabilitation inpatients. Arch Phys Med Rehabil 1993;74:531-6. 10. McHorney CA, Ware JE, Lu R, Sherbourne CD. The MOS 36item short-form health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse groups. Med Care 1994;32:40-66. 11. Campbell DT, Fiske DW. Convergent and discriminant validation by the mutitrait-multimethod matrix. Psychol Bull 1959;56:81105. 12. Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ 1997;314:572. 13. Stewart AL, Hays RD, Ware JE. Methods of constructing health measures. In: Stewart AL, Ware JE, editors. Measuring functioning and well-being. Durham: Duke Univ Pr; 1992. p 67-85. 14. Stineman MG, Shea JA, Jette A, et al. The Functional Independence Measure: tests of scaling assumptions, structure and reliability across 20 diverse impairment categories. Arch Phys Med Rehabil 1996;77:1101-8. 15. Iwanenko W, Fiedler R, Granger CV, Lee M. The Uniform Data System for Medical Rehabilitation: report of first admissions to subacute rehabilitation for 1998. Am J Phys Med Rehabil 2001; 80:56-61. 16. Feinstein AR. Multi-item “instruments” vs Virginia Apgar’s principles of clinimetrics. Arch Intern Med 1999;159:125-8.
ORIGINAL ARTICLE
Functional Independence Domains in Patients Receiving Rehabilitation in Skilled Nursing Facilities: Evaluation of Psychometric Properties Diane U. Jette, PT, DSc, Reg L. Warren, PhD, Christopher Wirtalla, BS ABSTRACT. Jette DU, Warren RL, Wirtalla C. Functional independence domains in patients receiving rehabilitation in skilled nursing facilities: evaluation of psychometric properties. Arch Phys Med Rehabil 2005;86:1089-94. Objective: To provide evidence for the validity of using FIM items to derive 4 domains of functional independence (mobility, activities of daily living [ADLs], sphincter management, executive function) in patients receiving rehabilitation in skilled nursing facilities (SNFs). Design: A retrospective analysis of secondary data using data from a privately owned administrative dataset. Setting: Seventy SNFs under contract with SeniorMetrix Inc health plan clients. Participants: Patients (N⫽7536) with a variety of conditions, covered by Medicare⫹Choice plans, and admitted to an SNF in 2002. Interventions: Not applicable. Main Outcome Measures: Item score distributions, corrected item-total correlations, factor correlations, internal consistency, and stage ceiling and floor effects for each hypothesized functional independence domain. Results: With the exception of 2 items, the items within a domain had similar standard deviations and distributions of items were not highly skewed. Four factors accounted for 73.4% of the variance in functional independence. Corrected item-total correlations ranged from .58 to .80 for the ADL domain, from .23 to .71 for the mobility domain, and from .78 to .88 for the executive function domain. The correlation between bowel and bladder management items was .84. Cronbach ␣ coefficients ranged from .76 for the mobility domain to .96 for the executive function domain. At admission, there were substantial floor effects for the sphincter management (34.4%) and mobility domains (43.1%) and ceiling effects for the executive function domain (26.7%). Conclusions: With a few exceptions, the items proposed for each functional independence domain met the criteria for supporting the validity of the domains. Key Words: Geriatric assessment; Rehabilitation; Skilled nursing facility. © 2005 by American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation
From the Physical Therapy Department, Simmons College, Boston, MA (Jette); and SeniorMetrix Inc, Nashville, TN (Warren, Wirtalla). At the time this study was initiated, Jette was affiliated with Boston University, Boston, MA. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated. Correspondence to Diane U. Jette, PT, DSc, Physical Therapy Department, Simmons College, 300 The Fenway, Boston, MA 02115, e-mail: [email protected]. Reprints are not available from the author. 0003-9993/05/8606-9289$30.00/0 doi:10.1016/j.apmr.2004.11.018
HE FIM INSTRUMENT IS used as a standard tool for T collecting data about patients’ functional performance on their entry into and discharge from an inpatient rehabilitation setting. The FIM is a measure of function in 13 motor areas and 5 cognitive areas. It contains 18 items, each rated on a 7-level scale, with 1 equating with total dependence and 7 equating to total independence. FIM scores are most often derived as a continuous total score (range, 18 –126) or as 2 continuous subscale scores for motor and cognitive abilities. Work by Stineman et al1 in 1997 suggested that the FIM could be viewed as a “multilayered, multidimensional” measure of function. They used factor analysis to derive dimensions that were more finely graded than the broad motor and cognitive dimensions. Although the dimensions they described differed slightly depending on the patients’ medical condition, 4 common domains of function emerged for 13 of the 20 medical conditions they examined. These domains included activities of daily living (ADLs) (FIM items for eating, grooming, bathing, dressing upper body, dressing lower body, toileting), sphincter management (FIM items for bladder management, bowel management), mobility (FIM items for bed, chair, and wheelchair transfer; toilet transfer; tub or shower transfer; walking or wheelchair mobility), and executive function (FIM items for comprehension, expression, social interaction, problem-solving, memory). In subsequent work, Stineman et al2,3 described a system that used each of the 4 domains of function to define profiles or stages of function. Each domain is comprised of 7 stages. Each stage represents a profile of scores within that functional domain based on FIM item scores. The stages suggest the average amount of assistance needed by the patient and the amount of effort required by the patient within each domain. For example, stage 1 in any domain equates with the patient being able to provide less than 25% of the effort required to accomplish the tasks in the domain; stage 4 indicates a minimal amount of assistance from another person and an effort of 75% on the part of the patient, and stage 7 indicates full independence. A patient must receive a designated minimum score on each item in the domain in order to be classified at a particular stage. For example, stage 4 in the mobility domain, which equates with needing a minimal amount of assistance, requires scores of at least 5 in chair and toilet transfers, at least 4 in tub transfer and walking or wheelchair mobility, and 1 in stairs. The functional domains described by Stineman1 were derived by using a sample of patients receiving care in inpatient rehabilitation facilities (IRF). Because a measurement instrument’s psychometric properties can vary depending on the sample of patients to which it is applied, the question arises as to the appropriateness of applying the 4 functional independence domains and stages to patients receiving rehabilitation in the skilled nursing facility (SNF) setting. SNFs are often chosen as the rehabilitation setting for patients who are moderately disabled yet need somewhat less intensive or comprehensive rehabilitation than that provided in IRFs.4 Patients receiving rehabilitation in SNFs are older and have lower cognitive and Arch Phys Med Rehabil Vol 86, June 2005
1090
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette
physical function on admission.5 Further, although factor analysis was applied to derive the domains, and internal consistency was tested,6 additional testing to determine the validity of the 4 domains was not reported. The purpose of the present study, therefore, was to examine the psychometric properties of the 4 domains of the FIM as described by Stineman.1 Specifically, we examined the validity of using the FIM items to derive 4 domains of function in patients receiving rehabilitation in SNFs. METHODS Design This study is a secondary data analysis using data from an administrative dataset compiled and owned by SeniorMetrix Inc. The company is privately held and has contracts with health plans to assist them to improve the delivery of rehabilitation services in SNFs for patients covered under Medicare⫹Choice reimbursement plans. Medicare⫹Choice is a plan that allows Medicare beneficiaries to choose among health maintenance organizations (HMOs), provider-sponsored organizations, and other fee-for-service plans that provide care under contract to Medicare. Under that program, HMOs contract with Medicare to provide the full range of Medicare benefits in return for monthly per-person payment rates. As of June 2002, of nearly 40 million Americans enrolled in Medicare, 13% had chosen to be in a Medicare⫹Choice managed care plan.7 More than 500,000 of the Medicare⫹Choice population are under care by providers contracting with SeniorMetrix health plan clients. This study was approved by the Institutional Review Board of Boston University. Sample The sample was derived from 9357 patients covered by Medicare⫹Choice plans who were admitted for short-term rehabilitation for the first time in 1 of 70 SNFs that were associated with SeniorMetrix in 2002. All facilities were in urban areas, and all but 3 were located in nursing home based, free-standing SNFs. Thirty-eight sites were in the northern Pacific region of the country, 17 were located in the southern Pacific region, 11 were in the mountain region, and 4 were in the mid-Atlantic region. We excluded 1064 patients who died during their SNF stay or who were admitted to an acute care facility from the SNF because, in our opinion, these patients likely had acute medical conditions that would affect their ability to attain functional improvements. We also excluded cases with negative values for time from impairment onset to SNF admission and cases with episode lengths of more than 100 days. This methodology resulted in a sample of 7526 patients. See table 1 for patient characteristics. Procedure Data were collected for all patients regarding demographics, medical complexity, length of stay, time from onset of primary condition to SNF admission, setting to which the patient was discharged, and functional status at SNF admission and at discharge. Functional status data included FIM scores on admission and discharge. Clinicians were trained and credentialed by SeniorMetrix in administering the FIM. The FIM instrument has good interrater reliability (intraclass correlation coefficient, ⬎.90).8 The validity of the FIM has been supported by its ability to discriminate among patients on the basis of age and comorbid status and among patients with different levels of Arch Phys Med Rehabil Vol 86, June 2005
Table 1: Patient Characteristics Characteristic
Value
Age, mean (95% CI) Male gender Ethnicity White, non-Hispanic African American Asian Native American Hispanic Other Impairment Stroke Neurologic, not stroke Orthopedic Cardiovascular/pulmonary Other Medical complexity score No other disease Inactive or irrelevant disease Active disease but not limiting Active and limiting disease Active and severely limiting disease Moribund Living in community before onset Discharged to community Days from onset to SNF admission, median (interquartile range) Length of stay, median (interquartile range)
76.3 (76.1–76.6) 2827 (37.6) 6344 (84.3) 469 (6.2) 233 (3.1) 3 (⬍1.0) 390 (5.2) 87 (1.2) 993 (13.2) 288 (3.8) 2711 (36.0) 1099 (14.6) 2435 (32.4) 95 (1.3) 197 (2.6) 1168 (15.5) 3850 (51.2) 2035 (27.0) 181 (2.4) 7277 (96.7) 6102 (81.1) 5 (5) 10 (13)
Abbreviation: CI, confidence interval. NOTE. Values are n (%) unless otherwise noted.
spinal cord injury and amputation level. Discharge destination is also related to discharge FIM score.9 Data Analysis To examine evidence for construct validity of the 4 FIM functional domains, we performed several types of analyses using the admission FIM scores. Item-level analysis. First, we examined the mean, standard deviation (SD), and skewness for each item within a stage as well as the frequency of item responses. We described the items in this way to determine if the items within each stage might contribute to the same degree to the determination of stage, that is, had similar means, SDs, and response frequencies. Factor analysis. We used principle components analysis with varimax rotation to determine how items would cluster in a 4-factor solution. Items were considered to belong to 1 factor if they were fairly well correlated within a factor (ⱖ.40) and less well correlated with other factors (⬍.40). Corrected item-total correlations. We looked for evidence that items in each domain as suggested by Stineman et al1-3 measured a common underlying construct by examining corrected item-total correlations (Spearman ). These analyses determined the correlations between the score for each item and the total of item scores for all other remaining items in the domain. For example, within the ADL domain, we determined the correlation between the score for the grooming item and the total of scores for eating, dressing the upper body, dressing the lower body, toileting, and bathing items. A minimum of r equal to .40 was deemed necessary to consider the items appropriate to combine into 1
1091
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette Table 2: Item-Level Analysis Stage/Item Score
Response Frequency (%)
Domain/Item
Mean ⫾ SD
Skew
1
2
3
4
5
6
7
ADL Eating Grooming Bathing Upper-body dressing Lower-body dressing Toileting Sphincter management Bowel management Bladder management Mobility Chair transfer Toilet transfer Tub transfer Walking or wheelchair mobility Stairs Executive function Comprehension Expression Social interaction Problem solving Memory
2.63⫾1.17 5.34⫾1.69 4.34⫾1.53 2.44⫾1.21 3.79⫾1.53 2.48⫾1.28 2.78⫾1.51 3.59⫾2.37 4.06⫾2.30 3.68⫾2.39 1.68⫾0.72 3.06⫾1.28 2.82⫾1.42 1.75⫾1.20 1.62⫾1.23 1.07⫾0.50 4.76⫾1.95 5.55⫾1.66 5.67⫾1.71 5.76⫾1.66 4.83⫾1.91 5.05⫾1.93
0.51 ⫺1.05 ⫺0.34 0.74 ⫺0.02 0.93 0.62 0.22 ⫺0.07 0.18 1.29 0.09 0.19 1.43 2.27 7.83 ⫺0.47 ⫺1.01 ⫺1.18 ⫺1.27 ⫺0.42 ⫺0.61
17.8 6.0 5.5 25.6 8.5 23.6 25.0 34.4 23.2 32.8 43.1 15.0 26.9 65.5 70.5 97.0 7.9 2.4 3.1 2.5 6.1 5.7
30.8 3.0 8.6 30.4 13.8 36.3 23.6 9.5 11.2 10.0 48.2 17.3 14.4 10.0 17.0 1.2 8.7 4.8 4.8 4.6 8.8 8.4
29.3 3.5 10.5 25.2 16.3 18.8 18.6 6.9 8.0 6.8 6.5 27.0 21.0 11.5 1.4 0.3 11.2 6.9 5.7 5.4 10.9 9.0
16.5 9.1 23.2 13.8 27.5 14.5 20.2 9.9 10.5 9.6 1.7 31.3 28.5 10.3 4.9 0.6 12.6 9.9 8.5 7.9 15.3 13.0
4.3 29.5 35.1 3.4 24.3 4.5 7.9 9.9 10.9 9.7 0.2 6.9 6.8 2.1 4.1 0.6 17.4 15.5 13.6 12.6 18.4 15.2
1.0 14.3 7.5 0.9 4.1 1.4 2.4 10.8 14.0 9.7 0.1 1.5 1.6 0.5 1.6 0.2 15.6 17.8 14.9 15.4 10.2 12.9
0.3 34.6 9.5 0.6 5.4 1.0 2.2 19.0 22.2 21.4 0.0 0.9 0.8 0.2 0.5 0.1 26.7 42.6 49.5 51.6 30.3 35.8
domain (convergent validity).10 We also sought evidence for discriminant validity by examining whether the items were more highly correlated to items within their domain than they were with items in the other domains. We also determined whether the correlations between domain scores were less than the estimates of internal consistency for each domain.11 Internal consistency. To examine the reliability of estimates that might be provided by total scores in each of the 4 domains, we derived Cronbach ␣ coefficients. We considered an ␣ of .70 to be sufficient to allow group comparisons within a domain.12 Stage ceiling and floor effects. Finally, combining the FIM items into the domains as described by Stineman,3 we determined the functional independence stages for patients in each domain and examined the ceiling and floor effects for the staging system at admission and discharge. We considered that in order for the measurement of a stage to be useful, no more than 20% of patients’ measurements should show ceiling or floor effects. Ceiling and floor effects show the limited ability of a measurement to discriminate among patients’ function at the high or low ends of the scale, respectively.
(table 3). Only these 4 factors had eigenvalues above 1.0. The ADL factor included grooming, bathing, dressing the upper body, dressing the lower body, toileting, toilet transfers, and chair transfers. For convenience, we term this factor “ADL and low-level physical function.” These 7 items accounted for 24.5% of the variance in functional independence. The sphincter management factor included bladder management and bowel management and accounted for 12.2% of the variance in functional independence. The mobility factor included only 2 items, walking or wheelchair mobility and stairs, and accounted for only 8.9% of the variance. Our term for this factor is “high-level physical function.” The executive function domain accounted for the greatest proportion of variance, 27.7%. The 6 items included in this factor were comprehension, expression, problem solving, social interaction, memory, and eating.
Item-Level Analysis Item means and distributions are shown in table 2. All response choices were endorsed for each item. With the exception of 2 items, the items within a domain had similar SDs and distributions of items were not highly skewed. Two items in the mobility domain, walking or wheelchair mobility and stairs, were skewed to the right (skewness⫽2.27, 7.83, respectively). The SD (.50) of the stairs item was small as compared with the SD of the other mobility items.
Corrected Item-Total Correlations Corrected item-total correlations ranged from .58 to .80 for the ADL domain, from .23 to .71 for the mobility domain, and from .78 to .88 for the executive function domain (table 4). The correlation between bowel and bladder management items was .84. These correlations were, for the most part, greater than the correlations with items in different domains. The tub transfer and toileting items correlated nearly as well with the mobility domain items (r⫽.31, r⫽.71, respectively) as they did with the ADL domain items (r⫽.32, r⫽.73, respectively), and the eating item correlated nearly as well with the executive function domain items (r⫽.55) as with the ADL domain items (r⫽.58). The tub transfer item did not correlate well (r⬍.40) with any domain. Similarly, the stair item did not correlate well with any domain, including the hypothesized mobility domain (r⫽.23). All ␣ coefficients were higher than correlations between the domains (table 5).
Factor Analysis The 4 factors derived by principle components analysis accounted for 73.4% of the variance in functional independence
Internal Consistency Cronbach ␣ coefficients ranged from .76 for the mobility domain to .96 for the executive function domain (see table 5).
RESULTS
Arch Phys Med Rehabil Vol 86, June 2005
1092
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette Table 3: Factor Analysis Factor
Low-Level Physical Function (ADL)
Sphincter Management
High-Level Physical Function (mobility)
Executive Function
Lower-body dressing Bathing Upper-body dressing Toileting Toilet transfer Grooming Chair transfer Tub transfer* Bladder management Bowel management Stairs Walking or wheelchair mobility Comprehension Memory Expression Problem solving Social interaction Eating*
.82 .77 .75 .74 .65 .64 .63 .51 .24 .24 .04 .30 .15 .15 .17 .18 .15 .46
.20 ⫺.02 .22 .39 .39 .20 .36 ⫺.06 .85 .84 ⫺.02 .27 .14 .14 .16 .16 .15 .17
.18 .06 ⫺.01 .18 .37 ⫺.06 .38 .40 .06 .06 .81 .62 .05 .06 .02 .06 .06 ⫺.12
.11 .19 .35 .17 .12 .49 .16 .03 .28 .33 .05 ⫺.002 .91 .91 .90 .88 .86 .58
NOTE. Values derived by principle component analysis with varimax rotation. Boldface indicates items most highly correlated with the factor. *Correlation ⱖ.40 for more than 1 factor.
Stage Ceiling and Floor Effects At admission, we found substantial floor effects for the sphincter management (34.4%) and mobility domains (43.1%) and ceiling effects for the executive function domain (26.7%) (see table 2).
Table 4: Corrected Item-Total Correlations Domain/Item
ADL Eating* Grooming Bathing Upper-body dressing Lower-body dressing Toileting* Sphincter management Bowel management Bladder management Mobility Chair transfer Toilet transfer Tub transfer† Walking or wheelchair mobility Stairs† Executive function Comprehension Expression Social interaction Problem solving Memory
ADL
Sphincter Management
Mobility
Executive Function
.58 .74 .60 .80 .74 .73
.42 .49 .31 .50 .43 .57
.35 .48 .46 .57 .66 .71
.55 .53 .32 .45 .29 .36
.54 .55
.84 .84
.47 .48
.44 .47
.65 .65 .32 .38
.48 .51 .17 .30
.71 .70 .31 .46
.33 .30 .09 .11
.16
.13
.23
.06
.48 .49 .45 .51 .48
.44 .45 .43 .46 .45
.26 .27 .28 .30 .27
.88 .84 .78 .88 .84
NOTE. Boldface indicates similar correlations with more than one factor or low correlations with all factors. *Correlates to similar extent with 2 domains. † Item does not correlate well (⬍.40) with any domain.
Arch Phys Med Rehabil Vol 86, June 2005
DISCUSSION Although Stineman et al2,3 did not propose adding the scores of items to create the functional independence domain stages, we used the types of psychometric testing one might employ to determine the legitimacy of combining item scores in this way to form domains. The findings of this study largely support the derivation of 4 domains of function based on the FIM for patients receiving rehabilitation in the SNF setting. With a few exceptions, the items within each hypothesized domain meet the criteria suggested for combining items into subscales.10-14 Item-Level Analysis For the most part, the items within each of the domains as originally suggested by Stineman et al,6 showed similar distributions, suggesting the legitimacy of combining the items into 1 domain. The stairs item is an exception. The responses to the item were highly skewed, with more than 95% of patients at the level of complete dependence. Stineman et al14 reported similar results, with the stairs item showing a low mean for patients in 16 of 20 impairment groups receiving rehabilitation in acute care settings. Equivalence of item means and item-response distribution are important in determining whether items may be legitimately combined into 1 summary score without weighting.10 Although Stineman2 did not endorse the summation of scores to attain domain stage assignments, this criterion seems relevant if a domain stage score is to be derived from scores on several individual items.
Table 5: Domain Correlations and Internal Consistency Domain
ADL
Sphincter Management
Mobility
Executive Function
ADL Sphincter management Mobility Executive function
(.89) .57 .67 .52
(.91) .50 .47
(.76) .29
(.96)
NOTE. Alpha coefficients are in parentheses.
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette
Factor Analysis We purposely constrained the factor analysis to 4 domains to explore the validity of the stages suggested by Stineman.2 Nevertheless, given a criterion of eigenvalues greater than 1 in determining viable factors, our findings support the appropriateness of aggregating FIM scores into 4 domains of functional for patients receiving rehabilitation in the SNF setting. The items within each domain, as determined by the factor analysis, however, varied from those previously proposed.2 All items related to transfers correlated more highly with the items that Stineman et al1-3 included in the ADL domain than with the mobility domain items of walking and stairs. Based on this finding, we conducted a factor analysis forcing extraction of 5 factors (data not shown). We found that the transfer items were related to 1 factor, while the daily activities related to personal care, including eating, were related to another factor. Walking and stairs remained a separate factor as in our 4-factor analysis. In our opinion, walking and stairs tend to require a high level of motor function, including strength, balance, endurance, and coordination, as compared with transfer activities, which require a lower level of function. Our analysis suggests that the first factor in our 4-factor analysis, which includes transfers, may be capturing the concept of low-level physical function, rather than strictly basic daily activities related to personal care. Corrected Item-Total Correlations The appropriateness of combining items into 1 domain can be also assessed by determining if an item has a higher correlation with other items in its hypothesized domain than with items in other domains (⬎2 standard errors [SEs]; in this study, .023).11,13 The result of this test of item discrimination is not definitive if the difference is smaller than 2 SEs and the test fails if the item correlation with another domain is higher than its correlation with the hypothesized domain. Similar to the results of the factor analysis, corrected item-total correlations suggested that transferring to the tub or shower was related to the ADL domain to a greater extent than to the mobility domain. Given that Stineman1-3 included the tub and shower transfer item in the mobility domain, we determined that this item failed to meet the test of item convergence and item discrimination. In our opinion, tub and shower transfer is intuitively related to the personal care activity of bathing oneself. Perhaps, for this reason, the item does not fall clearly into 1 domain. The item-total and item-other correlations of the toileting item are a possible failure of item discrimination. Although the correlation with the ADL domain was higher than that for the mobility domain, as proposed by Stineman,2 the difference was less than 2 SEs. In our opinion, the problem with this item may be very similar to that of the tub and shower transfer item. That is, although toileting represents a personal care activity, as defined by the FIM, it is linked intuitively to the ability to access the bathroom and commode. Finally, the eating item meets the requirements for item discrimination by a very narrow margin and factor analysis places the item in the executive function domain rather than the ADL domain. This finding could reflect the fact that we found the highest mean scores for eating and the items within the executive function domain. Eating may contain elements of chewing and swallowing as well as the action of moving the food to the mouth. It is possible that patients who are unable to communicate also have difficulty with chewing and swallowing—for example patients who are intubated or patients with brainstem infarction.
1093
Similar to the findings of Stineman et al,14 we found a relatively low corrected item-total correlation for the stairs item within the mobility domain (r⫽.23). This degree of correlation failed to meet the criterion for convergent validity (rⱖ.40). Factor analysis suggested that the stairs item, in combination with the walking item, was part of a high-level physical function domain. As suggested by our other findings, the stairs item is likely not measuring the same level of mobility as that required for transfers, explaining its low item-total correlation within its hypothesized domain. Additionally, the item scores showed very little variability with more than 95% of patients being totally dependent (score of 1), likely accounting for a low correlation. Internal Consistency We found that, although in 2 cases the item-total correlations did not meet the criterion for convergent validity, the domains showed adequate internal consistency. The mobility domain had the lowest internal consistency, reflecting the low itemtotal correlations of the tub transfer and stairs items and the low variation in the stair item. It has been suggested that a Cronbach ␣ of at least .90 is required to successfully apply scores to make decisions about an individual, and that an ␣ of at least .70 is required for comparing groups.12 If one were to combine the scores from the 2 sphincter management items or the 6 executive function items, as proposed by Stineman,2,3 these scales would meet the criterion for making decisions regarding individuals’ function in these domains. Stage Ceiling and Floor Effects The executive function and sphincter management domains showed substantial ceiling effects in this sample of patients in the SNF setting. Possibly, patients with poor executive function or lack of bladder and bowel control are not admitted for rehabilitation services because they are deemed unlikely to be able to participate in their own care. On the other hand, a measure that does not distinguish among patients at the top end of the scale may be insensitive to important improvements. Such ceiling effects may present problems when attempting to describe the benefits and outcomes of care, particularly care that addresses deficits in executive or sphincter function. The mobility domain showed important floor effects and was the only domain in which no patients were classified at stage 7. More than 90% of patients were in stage 1 or 2. This finding suggests that the mobility domain may not be able to distinguish among patients with differing levels of ability at the lower end of the continuum in mobility. For example, the walking item requires a score of 1 if the patient cannot walk at least 15m (50ft) and a score of 2 if the patient cannot walk at least 45m (150ft). It seems possible that patients entering SNF settings are able to walk with maximal or moderate assistance but for shorter distances than those required for scoring at higher levels. Similarly, Stineman14 suggested that the stairs item might be redefined to reduce the floor effect of that item. The limitations of this study include the use of secondary data. These data were generated for the major purpose of clinical and utilization management decision making and using them for other purposes requires cautious interpretation. Also, our sample included only patients covered through Medicare⫹Choice. The patients appeared to have basic demographic characteristics similar to a large national sample of patients treated in SNFs in 199815; however, they were less functional at admission as measured by the FIM. Our analyses also included a heterogeneous sample of patients in terms of their medical conditions. In previous work,1,14 authors have examined the psychometric performance of Arch Phys Med Rehabil Vol 86, June 2005
1094
PSYCHOMETRIC PROPERTIES OF FUNCTIONAL INDEPENDENCE DOMAINS, Jette
the FIM stratifying by type of medical condition, and the condition was found to affect the psychometric properties. Because the FIM is applied similarly to all patients on admission to rehabilitation, and therapists and other rehabilitation professionals often manage patients with a variety of medical conditions, we used the approach of determining the performance of the functional independence staging system across all patients in the SNF setting. In our opinion, the staging system described by Stineman et al2,3 is an important step in creating a measurement that is useful to clinicians and patients, that is, determining a stage can be done quickly and may be clinically useful across patients with the large variety of conditions encountered in the rehabilitation setting. Because the FIM is routinely used in practice, an important goal, in our opinion, is to determine if it can be used to derive an expression of functional independence that is meaningful in terms of making a prognosis and determining appropriate interventions for patients. For example, Feinstein16 has noted the inherent usefulness and simplicity of the Apgar score, stating that the variables represented in the Apgar score are clinically meaningful and can be understood and applied at the bedside. He also notes the value of identifying the various important individual clinical components in the scale. In our opinion, the staging system as proposed by Stineman,2,3 may be a first step toward allowing clinicians in rehabilitation to use the FIM items in a quick and inherently meaningful manner in patient management and establishing realistic goals, just as our colleagues in obstetrics and pediatrics use the Apgar score. CONCLUSIONS This study shows the validity of the functional independence stages, as proposed by Stineman3 in 2003, to patients in the SNF setting. Item-level analyses, factor analysis, item-total correlations, internal consistency, and ceiling and floor effects for each stage were examined. With a few exceptions, the items proposed for each functional independence domain met the criteria for supporting the convergent and discriminant validity of the domains. One suggestion for further exploration is to examine the psychometric performance of modified domains as suggested by our findings: sphincter management, executive function, ADL and low-level physical function and high-level physical function. Identifying easy-to-use, psychometrically sound, and clinically meaningful measures of functional independence is important for rehabilitation professionals in facilitating their management of patients. References 1. Stineman MG, Jette AM, Fiedler R, Granger CV. Impairmentspecific dimensions within the functional independence measure. Arch Phys Med Rehabil 1997;78:636-43.
Arch Phys Med Rehabil Vol 86, June 2005
2. Stineman MG, Ross RN, Fiedler R, Granger CV, Maislin G. Staging functional independence: validity and applications. Arch Phys Med Rehabil 2003;84:38-45. 3. Stineman MG, Ross RN, Fiedler R, Granger CV, Maislin G. Functional independence staging: conceptual foundation, face validity, and empirical derivation. Arch Phys Med Rehabil 2003;84: 29-37. 4. Gornick ME, Warren JL, Eggers PW, et al. Thirty years of Medicare: impact on the covered population. Health Care Financ Rev 1996;18:179-237. 5. Kramer AM, Steiner JF, Schlenker RE, et al. Outcomes and costs after hip fracture and stroke. A comparison of rehabilitation settings. JAMA 1997;277:396-03. 6. Stineman MG, Maislin G, Fiedler RC, Granger CV. A prediction model for functional recovery in stroke. Stroke 1997;28:550-6. 7. Centers for Medicare and Medicaid Services. Program information on Medicare, Medicaid, SCHIP, and other programs of the Centers for Medicare & Medicaid Services. June 2002. Available at: http://www.cms.hhs.gov/charts/series/sec3-b1.ppt. Accessed December 5, 2004. 8. Ottenbacher KJ, Hsu Y, Granger CV, Fiedler R. The reliability of the functional independence measure: a quantitative review. Arch Phys Med Rehabil 1996;77:1226-32. 9. Dodds TA, Martin DP, Stolov WC, Deyo RA. A validation of the functional independence measurement and its performance among rehabilitation inpatients. Arch Phys Med Rehabil 1993;74:531-6. 10. McHorney CA, Ware JE, Lu R, Sherbourne CD. The MOS 36item short-form health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse groups. Med Care 1994;32:40-66. 11. Campbell DT, Fiske DW. Convergent and discriminant validation by the mutitrait-multimethod matrix. Psychol Bull 1959;56:81105. 12. Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ 1997;314:572. 13. Stewart AL, Hays RD, Ware JE. Methods of constructing health measures. In: Stewart AL, Ware JE, editors. Measuring functioning and well-being. Durham: Duke Univ Pr; 1992. p 67-85. 14. Stineman MG, Shea JA, Jette A, et al. The Functional Independence Measure: tests of scaling assumptions, structure and reliability across 20 diverse impairment categories. Arch Phys Med Rehabil 1996;77:1101-8. 15. Iwanenko W, Fiedler R, Granger CV, Lee M. The Uniform Data System for Medical Rehabilitation: report of first admissions to subacute rehabilitation for 1998. Am J Phys Med Rehabil 2001; 80:56-61. 16. Feinstein AR. Multi-item “instruments” vs Virginia Apgar’s principles of clinimetrics. Arch Intern Med 1999;159:125-8.