Predicting elementary school participation in children with disabilities

339

Predicting Elementary School Participation With Disabilities

in Children

Mar&a C. Mancini, ScD, Wendy J. Coster, PhD, Catherine A. Trombly, ScD, Timothy C. Heeren, PhD ABSTRACT. Mancini MC, Coster WJ, Trombly CA, Heeren TC. Predicting elementary school participation in children with disabilities. Arch Phys Med Rehabil 2000;81:339-347. Objective: To identify predictors of participation in school activities from two sets of functional variables using classification and regression tree analysis. Design: Relational study. Participants: A nationwide sample of 341 children with various disabling conditions, including physical and cognitive/ behavioral types of impairment and various severity levels. Children attended public elementary school in 40 states in the United States. Main Outcome Measure: Overall participation in elementary school, combining children’s participation in six different environments (transportation, transitions, classroom, cafeteria, bathroom, and playground), as measured by the newly developed School Function Assessment. The children were dichotomized into full (n = 117) and limited (n = 224) participation categories. Results: Two classification trees were developed identifying a small set of predictors from variables measuring performance of functional tasks and discrete activities. Final predictive models included physical and cognitive-behavioral variables, suggested important interactions among predictors, and identified meaningful cut-off points that classified the sample into the outcome categories with about 85% accuracy. Conclusions: Limited participation was predicted by information about children’s physical capabilities. Full participation was predicted by a combination of physical and cognitivebehavioral variables. Findings underscore the relative utility of functional performance compared with impairment information to predict the outcome, and suggest pathways of influence to consider in future research and intervention efforts. Key Words: Children; Disability; Function; School. 0 2000 by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation From the Depanment of Occupational Therapy (Dr. Mancini), Universidade Federal de Minas Gerais, Belo Horizonte. Brazil; and the Departmem of Occupational Therapy (Drs. Caster. Trombly). Sargent College of Health & Rehabilitation Sciences. and the Department of Epidemiology and Biostafistics (Dr. Heeren). School of Public Health, Bosfon University, Boston. MA. This work was completed in partial fultillmem of M. C. Mancini’s dissenation requirement for the degree of Doctor of Science, Sargent College of Health & Rehabilitation Sciences, Boston University. Submitted for publication April 7. 1999. Accepted in revised form Augusr 12. 1999. Supported by a 4-year scholarship (Dr. Mancini) granred by the Conselho National de Desenvolvimento Cientffico e Tecnoldgico (CNPq). a Brazilian government agency, and by grant Hl33G30055 from the National Institute on Disability and Rehabililation Research, U.S. Depanment of Education. to Boston University. An organization with which one or more of the authors is associated has received or will receive financial benefits from a commercial party having a direct financial interest in the results of the research supporting this article. Reprinl requests 10 Wendy J. Cos~er, PhD. Chair, Department of Occupational Therapy. Sargenr College of Health & Rehabilitation Sciences. Bosron University, 635 Commonwealth Avenue, Boston. MA 02215. 0 2000 by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation 0003-9993/00/8103-5562$3.00/O

HE CURRENT EMPHASIS in the rehabilitation field is on T the integration into community living of individuals with disabilities.’ Clinical intervention goals have been redefined from an emphasis on reducing impairments to promoting full participation in society.’ This shift in outcome focus, however, has not included development of new theories about the pathways that lead to successful social integration-a situation that is especially evident in pediatric rehabilitation. As a result of federal legislation and public policy, children with disabilities have increasing opportunities to participate in mainstream social activities such as attendance at public schools.2-6More than 5 million children and adolescents with disabilities, aged 6 to 17yrs, now receive services in public schools across the United States.7.8This integration into mainstream school settings emphasizes the need for research that will help in understanding the factors that promote successful outcomes. An important first step has been the proposal of conceptual models of the disablement process.9-‘2These models identify a multilevel hierarchy that distinguishes the various consequences of a pathologic condition, ranging from its micro effect at the “organ level” (impairment) to intermediate levels described as “performance of whole body discrete activities” (functional limitations) and “performance of complex, integrative tasks” (disability), to its ultimate macro impact on the level characterized as “participation in a social context” (limitations in the performance of social roles).9.10 The disablement models have made important contributions to rehabilitation, including a common vocabulary that facilitates interdisciplinary communication and research,l3*14delimitation and articulation of key constructs, and a structure to guide the development of new instruments.‘5V17The models share a common limitation, however, in that they focus on the negative consequences of pathologic processes or limitations that an individual may experience, rather than the functional outcomes desired. This limitation has been addressed in the most recent revision of the International Classification of Functioning and Disability (ICIDH-2).18 This new version proposes three categories to describe function and disablement: body function and structure, activity, and participation. The ICIDH-2 also recognizes more clearly the important role that environmental and personal factors play in the level and extent of a person’s functioning. The language of the new classification system emphasizes the positive outcomes to be sought, rather than the negative consequences of pathologic processes or limitations. Thus, if we seek to understand factors that promote successful outcomes, it may be more useful to work with models like the ICIDH-2 that characterize these outcomes directly.18-21 Most of the research in rehabilitation has provided primarily descriptive information using measures that focus on specific levels of the disablement process. To move from a descriptive mode to an explanatory endeavor, investigations of the relationships among various levels are needed.22*23Theory building in rehabilitation will help organize information into a coherent and formal knowledge base that can suggest hypotheses regarding new, potentially effective, interventions. Some studies have begun to examine correlates of disablement outcomes; howArch

Phys

Med

Rehabil

Vol81,

March

2000

340

PARTICIPATION

OF CHILDREN

ever. most of the literature focuses on issues in the adult population.‘@* Many of the variables considered in these studies (eg, marital status. employment-related information, home maintenance. and financial resources) do not apply to children. Specific knowledge about childhood disablement and function is limited, especially on the broader social level of participation, and this has constrained further conceptual and clinical advances in pediatric rehabilitation. Research on correlates of function in school among children with disabilities has identified broad areas that should be considered. In general, studies suggest that severity of impairment is an important predictor of children’s educational placement as well as their social competence in school.“‘-s’ Successful functioning of children with disabilities within mainstream classrooms also was found to be influenced by a specific set of skills reflecting compliance with teachers’ directions and rules.-‘s-35More specifically. children who conform to classroom and school expectations have a better chance of being rated as successful than do children who have not mastered these behavioral competencies. Although they suggest general areas of concern, these studies have important limitations. including a narrow definition of outcome (eg, placement in an educational setting). and selection of predictors that reflect restricted domains of function (eg, social skills). Studies also have primarily examined main effects of predictors on the outcome and have not explored potential interactions among variables.36 This investigation addressed the need for information on important predictors of participation in school among children with disabilities. Based on three key features. it has the potential to expand existing knowledge. First. it focuses on functional outcomes rather than limitations or impairments. Second, it utilizes a more comprehensive outcome measure and broader set of predictor variables. Specifically, it examines not only classroom participation, but also integration into the cafeteria, playground, and other school settings. In addition, a large set of predictor variables was used to develop two predictive models of school participation. Conceptually, each predictive model incorporated information regarding a distinct level of function (as defined by current models) and examined unique relationships among variables at different levels. The ICIDH-2 classification recognizes that the activity level encompassesperformance of both simple and complex activities. This study addressed each of these aspects in a different predictive model. The first predictive model used functional tasks as predictor variables and explored the relation between the performance of complex activities and participation.g.‘O The second model also investigated the relation between activity and participation, but examined performance of discrete activities.gsiO Finally, the method used in this study was classification and regression tree (CART) analysis.“’ CART permits the identification of factors that might not intuitively seem to be strong predictors and therefore may not have received in-depth examination.25 It may also uncover clinically meaningful interactions among variables that may have been missed by previous studies that focused strictly on the investigation of main effects. Predictions from this nonparametric procedure are at the level of an individual rather than a group, and thus can be of great clinical usefulness. METHODS

Phys

Med

Rehebil

Vol81,

March

2000

DISABILITIES,

Mancini

September 1995 to January 1997. The majority of children in the sample were enrolled in public schools: 5 children were placed in specialized schools that were part of a public school system. The sample included children from urban (23%). suburban (51%). and rural (26%) schools. The data were collected as part of the standardization research on the School Function Assessment(SFA).is Children who participated in the standardization research were selected by local school professionals after they had obtained approval from the school administration and consent from the parents. Children were selected from among students receiving special education services with whom the participating professional had some current or previous involvement. Although formal stratilication was not performed, data collectors were given specitic guidelines for recruitment to ensure that the sample would represent the full spectrum of students with school-related functional difficulties. As data were returned. the sample composition was monitored to ensure it met a number of criteria including ( I ) inclusion of equal numbers of children from the various elementary school grades (ie, kindergarten to sixth grade); (2) racial and ethnic distribution reflecting US Census proportions; and (3) inclusion of children with a variety of disabling conditions and a full range of severity levels (table I ). Table

1: Descriptive Information of Children to Outcome Categories Outcome

Variable

Mean age (yrs)”

Limited Participation (n = 1171 9.02 (2.23)

According Categories Full Participation In = 2241 9.16

(2.16)

Gender’ Male

62 (53.9) 53 (46.1)

Female Disabling condition* Communication Motor impairment

disorder

Mental retardation Visual impairment Specific learning Emotional/behavioral Attention deficit Autism

disabilities difficulties

Hearing impairment Traumatic brain injury Primary type of impairment Physical Cognitive/behavioral Severity of impairment Mild Moderate Severe Racial/Ethnic White Black Asian/Pacific Hispanic Other

160 (71.4) 64 (28.6)

80 (68) 76 (65)

88 (39) 56 (25)

63 (54) 39 (33) 10 (9) 17 (15)

52 34 56 35

7 (6) 14 (12)

36 (16) 25 (11)

4 (3) 3 (3)

14 (6) 10 (5)

76 (65) 41 (35)

82 (37) 142 (63)

9 (8) 35 (30) 73 (62)

76 (34) 107 (48) 41 (18)

94 (81)

170 (76) 23 (10)

(23) (15) (25) (16)

Group

Islander

8 (7) l(1) 9 (8) 5 (3)

10 (4) 15 (7) 6 (3)

When descriptive information is provided as means, SDS are in parentheses. r When descriptive information is provided as frequencies, percentages are in parentheses. * Percentages add to more than 100% because many participants had more than one condition reported. l

Participants and Procedure The sample comprised 341 children attending elementary schools in 120 sites in 40 states and Puerto Rico, from Arch

WITH

PARTICIPATION

OF CHILDREN

Most childhood clinical disorders result in a variety of functional challenges for the child. Nevertheless. at this initial stage of investigation. we believed it would be useful to examine predictors of outcome based on whether the child’s most signilicant area of limitation was either physical or cognitive/behavioral. Although extensive diagnostic information was not collected during standardization, the professionals providing the data were asked to rate the severity of any impairment identified for each child on a list provided. This information was used primarily to help monitor the composition of the sample; however. for this study the ratings were used for initial exploration of the predictive utility of impairment information. Accordingly. children were further classified as having either a primary physical or cognitive-behavioral impairment. based on diagnostic and functional information recorded in designated sections in the data collection form, including primary diagnosis in the child’s school record, primary means of mobility. and severity of impairments identified. The physical impairment group included children with diagnoses such as cerebral palsy, motor impairment, spina bilida and muscular dystrophy. Severity of impairment for this group was based on the reported primary means of mobility: children who walked independently were classified as mild; children who walked with crutches. cane. or walker were identified as moderate; and children who used a wheelchair were classified as severe. The cognitive-behavioral group included children with mental retardation. autism, learning or language/communication disabilities, emotional disorders, or any kind of sensory impairment (visual, hearing). Severity of impairment for this group was based on the severity rating checked by the data collector for the cognitive-behavioral impairment most associated with that diagnosis. For example. for a child with Down syndrome, the severity reported under mental retardation was used. Sample monitoring secured a balanced representation of children in major impairment-type categories (physical and cognitivebehavioral). One of the authors (M.C.M.) did all the classification. Becauseclassifications were based strictly on the concrete information provided in the data forms, no reliability checks were performed. Data collectors were volunteers recruited through professional contact with the authors of the assessment, announcements at professional meetings and in publications, and mailings to individuals who had shown interest or who had participated in previous pilot versions during the development of the SFA. Professionals at each of the 120 sites involved were asked to complete assessmentson pairs of students (ie, between 1 and 3 pairs), including a child with special needs and a child with no special needs, matched by grade level. This procedure generated two distinct samples of children: 269 children with no special needs, and 341 children with special needs. In this study, we used only the sample of children with special needs. Project coordinators reviewed the data gathered on each child and pursued missing information to ensure complete profiles of the children in the sample. The SFA is a criterion-referenced test that uses a judgmentbased method of evaluation and thus does not require active participation of children. In most cases, the assessment was completed by a professional team that included the child’s teacher (ie, regular educator, special educator, or both) and rehabilitation professionals (ie, occupational and physical therapists, speech and language specialists, school nurses). Data collectors were asked to follow the instructions given in the test booklet, and no special training was provided on the administration of the assessment.The lack of a formal training on the

WITH

DISABILITIES,

341

Mancini

administration and scoring of items simulated how the instrument was intended to be used. Instrument and Operationalization of Variables The SFA was designed to measure the elementary school participation and profiles of functional performance in children with disabilities. The instrument is divided into three parts that may be administered separately. Data from parts I and III were used for this study. Part I examines the child’s participation in six different school settings: transportation to school, transitions between school locations, classroom (ie. either specialized or regular), mealtime or snacktime in cafeteria or other designated environments. bathroom/toileting, and playground/recess. This part of the instrument measuresfunction at the social participation role level. as delineated by the disablement and function models. In each setting, a 6-point rating scale is used to indicate the extent to which the child’s participation in the tasks and activities characteristic of that setting is similar to that of peers who are in the same grade. Possible ratings for each setting range from a score of 1, reflecting extremely limited participation, to a score of 6. indicating the child fully participates in all tasks and activities within the setting. Ratings across these six settings were summed to yield a composite score of overall participation in school. This score was used as the outcome measure of the study. Part III provides more detailed information on the child’s performance of school activities that are likely to be expected on a daily basis. These activities are grouped into 21 different task scales, reflecting 12 physical and 9 cognitive-behavioral task domains. Three of the physical tasks are optional and were not considered in this study. Possible ratings for each item on these scalesranged from a score of 1, “does not perform,” to 4, “performs consistently.” A brief description of each scale is provided in the appendix. Psychometric information, including results of a series of reliability and validity analyses on the SFA, is detailed in the user’s manual. All scales were developed using Rasch Item Response Theory (IRT) methodology3* to examine unidimensionality and to obtain interval level derived scores. Studies have provided favorable evidence of internal consistency and coherence, as well as stability of scores across assessment occasions (test-retest r values > .90). Factor analytic studies identified two primary dimensions underlying the activity items that were identified as physical function and cognitivebehavioral function.39 Information from part III was examined in two ways, characterizing functional performance at the task and activity levels.i9.20Summary scoresfrom each of the 18 functional task scales were used as predictor variables for one predictive model. At the task level, activity items are combined in a way that reflects a shared goal or focus (eg, travel, positive interaction). A second way to summarize activity information was through factor groupings. Activities grouped into factors lack the common goal characteristic of a task because the factors may combine information from different tasks. This approach servesa different purpose, which is to identify broad commonalities across items. In this study, factor groupings of activity items were entered as predictors in a second predictive model. Data Analyses Preliminary analyses. Two preliminary analysesusing IRT methodology and factor analyses were performed before classification analyses. Rasch analyses were conducted using the Arch

Phys

Med

Rehabil

Vol81,

March

2000

342

PARTICIPATION

OF CHILDREN

BIGSTEPS software package”O to transform task summary scores from ordinal to interval-level scaling as well as to characterize the outcome variable. The participation scale (from part I) and the I8 task scales (from part III) were each transformed to a 0 to 100 continuum. Results from these analyses were also examined to verify that each scale constituted a coherent measurement scale. Output from IRT analyses was used to characterize the outcome as a dichotomous variable (full participation and limited participation) using a cut-off score identified from BIGSTEPS output. 4’ Specifically, the variable map for the participation scale was examined to identify the score on the 0 to 100 continuum that distinguished two groups with different expected performance patterns. Children classified in the “limited participation” category, on average, were unable to participate meaningfully in most tasks across the six settings, unless constant supervision or help was provided (expected ratings of 1, 2, and 3 across the six settings). Children in the “full participation” category, on average, were able to perform most tasks and activities across the settings independently or with intermittent assistance (expected ratings of 4, 5, and 6 across settings). This categorization differentiated children who can only participate with constant supervision or help (eg, need a full time aide) from those who are capable of participating in the various school settings with occasional assistance. The selected cut-off is clinically meaningful because it identifies two participation categories which in practice may represent groupings of children for whom different levels of resourcesare required to support their performance of functional tasks and activities. (Additional detail about the use of variable maps to dichotomize outcomes is provided elsewhere.4’ The second set of preliminary analyses was conducted to group activity items into factors. Four factor analyses were performed on activities from four distinct functional subdomains: (I) gross-motor functional performance, including 58 activities from the tasks of travel, maintaining and changing positions, recreational movement, and manipulation with movement; (2) fine-motor performance, including 87 activities from the tasks of using materials, setup/cleanup, eating and drinking, hygiene, and clothing management; (3) cognitive function, including 5 1 activities from the tasksof functional communication, memory and understanding, task/behavior completion, and safety; and (4) social/behavioral function, including 69 activities from the tasks of following social conventions, compliance with adult directives/school rules, positive interaction, behavior regulation, and personal care awareness. Principal axis factoring was used in all factor analyses. Initial analyses consistently suggested a three-factor solution in each area (additional factors had eigenvalues of c2.0). Because the resulting factors were expected to be intercorrelated, oblique rotations were selected for the final solutions. Using the 3-factor solution from each of the 4 subdomains resulted in I2 factors characterizing functional activity information. The unidimensionality of each factor grouping was further examined through IRT analyses, and the transformed IRT scores (scaled on a 0 to 100 interval) were entered into the CART analysis as predictors characterizing functional activity performance. Predictive models of function. Two major predictive models of participation were developed using CART analyses.42 Because of sample size limitations in relation to the total number of potential predictors, each model of participation included composite variables that characterized different levels of disablement.9*‘0The first model was constructed from a set of 20 predictor variables, including the 18 functional tasks (9 physical and 9 cognitive-behavioral) from part III, and 2 Arch

Php

Med

Rehebil

Vol81,

March

2000

WITH

DISABILITIES,

Mancini

variables retlecting information on the children’s impairments (type and severity). This model searched for signiticant predictors of the social participation outcome from variables reflccting function and disablement at the levels of complex activity performance (activity limitation) and body structure and function (impairment). The second predictive model included the following I4 predictors: children’s impairment information (type and severity) and the I2 factors identified by the factor analysis, characterizing groupings of functional activities. In this model. participation outcome was predicted using information from the levels of discrete activity performance (activity limitation) and body function and structure (impairment). A third CART model was developed to compare the predictive power of impairment variables alone with the prediction by functional information. For this model. only the two impairment variables (type and severity) were included. The prediction accuracy of this reduced model was compared to that of the previous two models. CART is a nonparametric procedure and, for the purpose of this study, it was used to classify individuals according to the specified outcome categories (full and limited participation). For a more thorough description of the procedure. refer to Breiman and colleagues.-77For examples of application in the rehabilitation literature, see Falconer and coworkers,?’ Temkin and associatesZRHarrell and colleagues.JJ and Stineman and coworkers.jJ The classification tree output results from a two-step process. including the growing of a large tree and its subsequent pruning. The procedure starts by growing a large tree, through a series of binary recursive partitioning or splits. The overall process is called binary becauseeach parent node is always split into two child nodes. It is recursive because the process is repeated systematically, treating each child node as a parent node and further classifying the sample. Splitting continues recursively until further classification is impossible (the rule applied in this study) or stopped by a defined rule, such as that a node has reached the minimum number of cases allowed. For each split, all possible divisions on a predictor variable are evaluated until a cut-off point is selected that best classifies individuals into each of the two outcome categories. This process is repeated for all the remaining predictor variables. Subsequent “splits” identify possible interactions among predictors. The variable selected for each split represents the most powerful predictor of the outcome at that level of the tree, as compared with the other predictors. In the second stage of the CART procedure, the original large tree is pruned to yield a more parsimonious tree. Starting with the large complex tree, a sequence of progressively simpler trees is constructed by combining subgroups of individuals that are relatively similar to one another. This pruning stage can be thought of as contributing to validation, in which the original large tree is evaluated in terms of its classification accuracy and overall tree size. The validation stage of the overall procedure is used to examine whether predictions from the model will replicate on other samples or whether the original model was “tailor-made” to fit idiosyncrasies in the model development sample.z5.42J4 Validation can be conducted on an independent test sample or on the same sample used to develop the original tree. Because this study had insufficient data available for validation in a separate sample, the CART software used the computerintensive technique of cross-validation.42 During crossvalidation, the CART program divides the original sample into IO equal parts, each containing similar distributions of the dependent variable (full or limited participation). CART uses

PARTICIPATION

OF CHILDREN

the first Y parts to construct the largest possible tree, and the remaining part is used as a test subsample. The same process is repeated until each part of the data has been used once as a test subsample. The results of these IO minitest samples are then combined to form error estimates for trees of each possible size. The best pruned tree developed through this validation process is chosen based on the principle of parsimony, selecting the one that achieves the best balance between classification accuracy (overall classification error) and tree complexity (number of nodes). In this study. classification trees were developed using the default (GINI) method to specify splitting rules; prior class probabilities for classification of casesin the sample were set as equal. Furthermore, a tenfold cross-validation method was used to estimate misclassification rates.

WITH

DISABILITIES,

343

Mancini

Ten,s,tnl Node I

RESULTS

One hundred seventeen children were classified into the limited participation group and 224 children were classified into the full participation outcome category based on their scores on the participation composite (including six-item settings) from part I of the SFA. Descriptive information on children in each outcome category is summarized in table 1. The goodness of each classification was evaluated by two indices. Using “limited participation” as the reference group, the sensitivity of the classification trees expressed the probability that a child with a limited participation outcome would be correctly classified, and the specificity index characterized the probability that a child with a full participation outcome would be correctly classified, based on the predictive rules proposed by each model. Classification

Tree

1

This first classification tree identified important predictors of the participation outcome from the 18 functional task scales (appendix) and 2 impairment variables (type and severity). The final output tree shown in figure I selected 3 of the 20 predictors entered: clothing management, following social conventions, and compliance with directions/rules. This set of predictors includes one task from the physical domain (clothing management) and two from the cognitive-behavioral domain. With the three predictor variables, tree I correctly classified 100 of the I I7 children in the limited participation category (sensitivity, 85.5%) and I87 of the 224 children from the full participation category (specificity, 83.5%). Most of the children with a limited participation outcome were identified based on their scores on the clothing management variable, whereas children with a full participation outcome tended to be identified based on their scores on both clothing management and following social conventions functional tasks (fig I). Using information from IRT output, the selected cut-off scores from the first two splits can be translated into typical functional profiles for these task scales. Children with scores equal to or lower than 58.5 on the clothing management task, on average, could not perform the harder manipulative activities at all (rating of 1) (eg, manage hooks and buttons). These children had “inconsistent performance” as the highest score (ratings of 3 or lower) on many activities of medium difficulty level (eg, managing shoes and socks) and could “consistently perform” (rating of 4) only the easiest item in this scale (removes hat). In contrast, children with scores above 58.5 on this task, on average, tended to show “consistent performance” (rating of 4) on simple items (eg, removing front opening garment), and at least “inconsistent performance” (rating of 3 or higher) on activities of medium difficulty level. However, these children

Tmninal Node 4

Fig 1. Classification tree 1: functional tasks as predictor variables. In the tree output, squares represent terminal nodes that characterize final classification for that grouping of children; circles identify nonterminal nodes, meaning that children in such nodes were further classified using information from other predictor variables.

could have received ratings as low as “partial performance” (rating of 2) on the more difficult manipulative clothing management activities. Similarly, an IRT transformed score of 49.5 on the following social conventions scale suggests distinct functional profiles. Children with scores higher than the cut-off, on average, showed “consistent performance” on simpler items (eg, manners during eating). They also showed at least “inconsistent performance” on items measuring appropriate use of social manners (eg, “please, ” “thank you”). However, children could have “limited performance” (rating of 2) on the more difficult activities within this cognitive-behavioral scale (eg, respecting others’ privacy) and still attain a total score higher than the cut-off. Thus, the identified scores used for these two splits suggest interesting functional thresholds associatedwith participation outcomes. The impairment status of children classified in the terminal nodes (squares) was examined to provide additional infocmation. Terminal node 1 grouped most of the children identified as limited participants (n = 97). This group of children had a higher frequency of a physical type of impairment (74%, or 72/97) and tended to be classified as severe (68%, or 66197). Terminal node 4 gathered most of the children classified as full participants (n = 185). In general, these children showed higher percentages of cognitive-behavioral impairments (63%, or 116/185), and their severity was characterized as mild (39.5%, or 73/185) or moderate (46.5%, or 86/185). Because the model was developed and validated using the same sample, it is possible that its predictions might yield more classification errors when applied to a new sample. The cross-validation error rate, which suggestshow well the predictions from tree 1 would replicate on an independent sample, shows an overall misclassification of 15.8%, indicating that Arch

Phys

bled

Rehabil

Vol81,

March

2gW

344

PARTICIPATION

OF CHILDREN

more than 84% of the casesin a new sample would be correctly classified into one of the two outcomes. Classification Tree 2 The second classification tree was grown using the same sample to identify predictors from the group of 12 factors that summarize information on children’s performance of discrete activities and the 2 impairment variables (type and severity). The output tree shown in figure 2 included four variables: fine-motor factor 3. gross-motor factor 2, social/behavioral factor 2, and cognitive factor 3. This set of predictors includes one factor from each of the four subdomains (ie, gross-motor. manipulative, cognitive, and social/behavioral). The combination of four factor variables correctly classified 99 of the 117 limited participants (sensitivity, 84.6%) and 189 of the 224 full participants (specificity, 84.4%). Similar to tree I, most of the children with a limited participation outcome were identified using information from physical activities (combination of fine-motor and gross-motor factors), whereas those with a full participation outcome were mostly identified using information from motor and social/ behavioral factors (fig 2). Investigation of children’s impairments helped characterize those identified in terminal nodes. Most of the limited participants were grouped in terminal node 1 (n = 96). These children showed a higher frequency of physical impairments (73%, or 70/96), and the majority were considered severe (68%, or 65/96). Terminal node 5 had the largest frequency of full participants (II = 186). This group tended to combine children with cognitive-behavioral impairments (66%, or 123/186) of mild (37%. or 72/186) or moderate (49%, or 91/186) severity. Because the activities comprising the factor variables are drawn from various tasks, and thus, do not share the common focus of the task-based scales. the cut-off scores from these

Fine-motor

Factor 3:

Fine-motor

actor 2: Personal

Socialilxhavioral Fsclor 2: I Personal Care Awareness Aclivitia (spill: score < = 54.5)

II

Factor 3:

Full

x I I Limited Cogoitivo Factor 3: Basic Needs

Co@lve

Factor 3:

Termmol Node 4

Pig 2. Classification variables.

Arch

Phys

Med

tree

Rehabil

2: factor

groupings

Vol 81, March

2000

of activities

as predictor

WITH

DISABILITIES,

Mancini

variables are not as readily interpreted as the task-based variables were. The cross-validation error rate from classification tree 2 was 15.5%. suggesting that if validated in an independent sample. predictions from tree 2 would correctly classify 84.5% of the casesin that sample. Classification Using Impairment Predictors A third classilication tree was developed to examine the ability of impairment information alone to predict participation outcome. In this model. only two categorical variables were entered aspredictors: type of impairment (physical or cognitivebehavioral) and severity of impairment (mild. moderate. or severe). The resulting output tree selected only severity of impairment as an important predictor of the outcome. Children with severe impairments were classified as limited participants. whereas children with mild or moderate impairments were classitied as full participants. This predictive model correctly identified 73 of the 117 limited participants (sensitivity. 62%) and 183 of the 224 children with a full participation outcome (specificity. 82%). The cross-validation error rate from this classification tree was 25%. DISCUSSION Findings from this study contribute to both conceptual and clinical knowledge in pediatric rehabilitation. This discussion will focus first on the theoretical implications of the results, followed by their relevance for guiding clinical practice. Taken together, the results from the classification tree models support two important propositions of function and disablement models. First, the data support the premise that participation in school is a highly complex concept that encompassesinformation from various domains. The two predictive models reported in this study included predictors from multiple domains of functional performance: tree I showed a combination of physical and cognitive-behavioral tasks, and the model from tree 2 included factors from each of the four areas of activity performance. Future investigations of school participation outcome must attend to the complexity of factors underlying function at this level and make use of methodologies that allow the examination of important interactions among variables. Findings from this study further support premises regarding the relationship among levels of function and disablement. Results are congruent with the proposal that the impact of a clinical disorder on higher levels of disablement and function is indirect and mediated by the intermediate levels describing individuals’ functional limitations and disabilities’s In this study, functional information was better able to identify participation outcomes in children with disabilities than was information related to the specific pathologic conditions of these children (ie, type and severity of impairment). In the absenceof functional information, severity of impairment was selected over type of impairment to classify children into participation outcome groups. However, information on severity could more accurately identify those with a full participation outcome than those with a limited participation outcome. This finding suggests that limited participation outcome may be more strongly associated with a lack of particular functional competencies than the presence of a severe impairment, per se. The results do not imply that impairment information is irrelevant, but rather that impairment by itself cannot adequately specify likely functional outcomes for an individual child. This interpretation is in agreement with recent findings from the adult literature” reporting that functional limitations have a direct impact on the incidence of disability, whereas

PARTICIPATION

OF CHILDREN

inlluences higher levels of disablement only indirectly. once information about individuals’ clinical conditions is emheddcd in their functional performance of meaningful activities. Examination of the specilic variables comprising each predictive model helps characterize the functional features that may underlie participation outcomes among children with disabilities. Because they were often intercorrelated. other variables could also have been selected as predictors. However, those included in these models bring additional unique information to the prediction. Proper interpretation of the present findings requires that we recognize the selected predictors as “marker variables.” Thus, the composite variables included in each predictive model should be understood as the functional expression of underlying skills required for performance of important activities. The variable used for the first split in a classilication tree is considered the one with the strongest association with the outcome. in relation to the other variables in the model. In both classification trees. the variable selected for the first split reflected children’s physical ability to manage their clothing. Activities included in these scales (clothing management in tree I and personal care management in tree 2) can be thought of as functional indicators of severity of physical limitations because they represent a complex combination of major physical capabilities: ( I ) upper and lower body movement and control, as involved in putting on and taking off sweatshirts. sweaters, coats. and pants; (2) a combination of upper and lower body movement with postural control, as required for lowering and pulling up pants, removing shoes. and hanging clothing; and (3) line manipulative skills, as used in the management of fasteners, small buttons. and zippers. The predictive strength of this variable over others in both models may be explained by the broad combination of physical skills demanded by these activities and the importance of these skills in the school environment. Limited participation outcome was predicted primarily by one or two variables reflecting physical skills in both tree I and tree 2. Together. these results suggest that information on children’s physical capabilities may serve to identify a threshold of functional performance that is associated with participation outcomes. In tree I. the identified cut-off point on the clothing management scale suggests that children tended to be classified as limited participants when they could not perform items of medium difliculty level in this scale, such as those requiring whole-body and postural control. Children who showed at least inconsistent performance (rating of 3 or higher) on these activities were further classified using information from cognitive-behavioral variables. This interpretation is supported by the output from tree 2. In this tree, most of the limited participants identified in terminal node I were children who lack both fine-motor skills (lower scores on personal care management activities) and whole-body and postural control (lower scores on gross-motor factor 2). Thus, this finding suggests that a specific set of activities requiring a combination of upper and lower body movement, along with postural control, can be used to characterize the functional expression of children’s impairments. indicating those likely to have limited participation outcome. Prediction of successful participation was achieved with a combination of physical and cognitive-behavioral information. Findings showed that a full participation outcome required the physical capabilities discussed previously (whole-body and postural control), along with a specific set of social skills. In tree I. the identified cut-off on the following social conventions impairment

WlfH

DISABILITIES,

345

Mancini

scale suggests that children tended to be classified as full participants if they could show specific social skills, such as showing general good manners (“thank you,” “sorry.” “please”). maintaining appropriate social and physical boundaries. and asking permission when required. This set of skills constitutes the basic requirements for individuals to relate socially to others in this culture. Based on the results, the interaction between physical capabilities and social competence was the major pathway associated with full participation. This hnding underscores the complex array of functional skills related to children’s participation in elementary school. It further reinforces the need to assess various domains of function. gathering information on both social and physical abilities, to plan interventions that will support successful participation. Findings reported in this study do not imply a causal relationship between predictors and outcome. However, they can be used as a guide to help clinicians prioritize their clinical efforts. Results support current trends in rehabilitation toward more reliance on functional measurement rather than clinical diagnostic data to plan for children’s participation in mainstream elementary school programs. Furthermore, the classification trees identify possible pathways of influence associated with limited and full participation outcomes that should be considered during such intervention planning. For example. information about children’s physical capabilities may serve as a primary indicator to identify those who are at higher risk for limited participation in school. However, characterization of full participation appears to require a combination of physical and social/behavioral competence. As a first investigation in this area, this study also reflects several limitations that must be addressed in future research. First, although results from the trees show converging evidence, validation of the present findings will require the testing of these predictive models using independent samples. Future investigation should also consider the use of different measures to characterize the variables. In this study, both predictor and outcome variables were part of the same instrument, and therefore, an unknown amount of shared method variance affecting the results may be present. Also, lack of a sound classification method for severity of impairment limited the investigation of the relationship between impairment variables and functional outcomes. Further development of more carefully defined measures of impairment severity is needed. Finally, conceptual interpretation of the present findings may be restricted because of the cross-sectional nature of the data. Although results from this investigation provide preliminary validation of several premises of the function and disablement models, future longitudinal studies are needed for more complete testing of these modelsJs Despite the limitations of this study, the findings show high indices of accuracy provided by the two classification trees. This accuracy was achieved using a limited number of functional predictors that reflected different domains of petformance. The continuously measured predictors used in this study provided additional insights into the characterization of participation outcomes by allowing translation of the cut-off scores into specific profiles of functional performance associated with limited and full participation. As pediatric rehabilitation continues its knowledge-building process, it will benefit from studies that attend to the complex nature of this functional outcome and examine the meaningful relationships that contribute to success. Further identification of the important predictors of participation outcomes in subgroups of children sharing similar disabling conditions will help refine the general information Arch

Phys

Med

Rehabil

Vol81,

March

2000

346

PARTlClPATlON

OF CHILDREN

provided by this study, adding valuable information on the pathways of influence on outcomes for children with various impairments. Acknowledgment: The authors thank Dr. Jennifer Anderson for her expert guidance with the application of CART. I.

2. 3. 4. 5. 6. 7. 8.

9. 10. 11. 12. 13. 14. 15. 16.

17. 18. 19. 20. 21.

Arch

References Frey WD. Functional assessment in the SO’s: a conceptual enigma, a technical challenge. In: Haloem AS. Fuhrer MJ, editors. Functional assessment iirehabilitabon. Baltimore: Paul Brookes; 1984. p. 11-43. Butler C. Outcomes that matter [editorial]. Dev Med Child Neurol 1995;37:753-4. Kalscheur JA. Benefits of the American with disabilities act of 1990 for children and adolescents with disabilities. Am J Occuu Ther 1992;46:419-26. Pellegrino L. Cerebral palsy: a paradigm for developmental disabilities. Dev Med Child Neural 1995;37:834-9. Education for All Handicapped Children Act of 1975, Public Law No. 94-142.20 U.S.C. 51400 (1975). Individuals with Disabilities Education Act of 1990, Public Law 101-476.20 U.S.C. $1400 (1990). Center for the Future of Children. The future of children, Vol 6. Issue 1. Los Angeles: David & Lucile Packard Foundation; 1996. DeJong G. Sutton JP. Rehab 2000: the evolution of medical rehabilitation in American health care. In: Landrum PK. Schmidt ND, McLean A Jr, editors. Outcome-oriented rehabilitation: principles, strategies, and tools for effective program management. Gaithersbure (MD): Asoen Publishers: 1995. u. 3-42. Coster WJ,-iale; SG. Conceptualization a^nd measurement of disablement in infants and young children. Infints and Young Children 1992;4(4): 1l-22. National Center for Medical Rehabilitation Research. Digest of data on persons with disabilities. Washington, (DC): US Department of Education; 1993. Verbrugge LM, Jette AM. The disablement process. Sot Sci Med 1994;38:1-14. World Health Organization. International classification of impairments, disabilities and handicaps. Geneva: WHO; 1980. Guccione AA. Physical therapy diagnosis and the relationship between impairments and function. Phys Ther 199 1;7 1:499-504. Jette AM. Physical disablement concepts for physical therapy research and oractice. Phvs Ther 1994:74:380-6. Coster W. Dkeney T, Hhtiwanger J,‘Haley S. School Function Assessment. San Antonio (TX): The Psychological Corporation/ Therapy Skill Builders; 1998. Guide for Uniform Data Set for Medical Rehabilitation for Children (WeeFim SM). Version 4.0, Community Outpatient. Buffalo (NY): Uniform Data System for Medical Rehabilitation. State of New~York; 1994. Haley SM. Coster WJ, Ludlow LH, Haltiwanger JT, Andrellos PJ. Pediatric Evaluation of Disability Inventory (PEDI). Boston: New England Medical Center Hospitals; 1992. World Health Omanization. ICIDH-2-International Classification of Functionini and Disability. (Beta 2 Draft). Geneva: WHO; 1999. Trombly C. Anticipating the future: assessment of occupational function. Am J Occup Ther 1993;47:253-7. Trombly C. Occupation: purposefulness and meaningfulness as theraoeutic mechanisms. Am J Occuo Ther 1995;49:960-72. Cost& W. Occupation-centered askessment of children. Am J Occup Ther 1998;52:337-44.

Php

Med

Rehabil

Vol81,

March

2000

WITH

DISABILITIES,

Mancini

22. Fuhrer MJ. Overview of outcome analysis in rehabilitation. In: Fuhrer MJ. editor. Rehabilitation outcomes: analysis and measurement. Baltimore: Paul H. Brookes: 1987. p, l-15. 23. Jette AM. Outcomes research: shifting the dominant research paradigm in physical therapy. Phys Ther 1995:75:965-70. 24. DeJong G. Branch LG. Corcoran PJ. lndependent living outcomes in spinal cord injury: multivariate analyses. Arch Phys Med Rehabil 1984;65:66-73. 25. Falconer JA, Naughton BJ. Dunlop DD. Roth EJ. Strasser DC. Sinacore JM. Predicting stroke inpatient rehabilitation outcome using a classification tree approach. Arch Phys Med Rehabil 1994;75:619-25. 26. Jette AM, Branch LG. lmpairment and disability in the aged. J Chronic Dis 1985;38:59-65. 27. Lawrence RH. Jette AM. Disentangling the disablement process. J Gerontol 1996~5 1B(4 Suool): I73S-82s. 28. Temkin NR, Holubkov k: Machamer JE. Winn HR. Dikmen SS. Classification and regression trees (CART) for prediction of function at 1 year following head trauma. J Neurosurg 1995~82: 764-71. 29. Buysse V, Bailey DB Jr, Smith TM, Simeonsson RJ. The relationship between child characteristics and placement in specialized versus inclusive earlv childhood orograms. Too Earlv Child Special Educ 1994;14:419-35. . ’ 30. Glassberg LA. Students with behavioral disorders: determinants of placement outcomes. Behav Disord 1994; 19: I8 l-9 I. 31. Sinclair E, Alexson J. Relationship of behavioral characteristics to educational needs. Behav Disord 1992;17:296-304. 32. Thompson RJ Jr, Gustafson KE. Adaptation to chronic childhood illness. Washington (DC): American Psychological Association: 1996. 33. Ellett L. Instructional practices in mainstream secondary classrooms. J Learn Disabil 1993:26:57-64. 34. Fad KS, Ryser GR. Social/behavioral variables related to success in general education. Remedial Special Educ 1993; 14:25-35. 35. Reiher TC. Identified deficits and their congruence to the IEP for behaviorally disordered students. Behav Disord 1992;17: 167-77. 36. Rutter M. Psychosocial resilience and protective mechanisms. Am J Orthopsychiatry 1987;57:3 16-3 1. 37. Breiman L, Friedman JH, Olshen RA. Stone CJ. Classification and regression trees. New York: Chapman & Hall; 1993. 38. Wright BD. Stone MH. Best test design. Chicago: Mesa Press: 1979. 39. Coster WJ, Mancini MC. Ludlow LH. Factor structure of the School Function Assessment. Educ Psycho1 Meas 1999;59:665-77. 40. Linacre JM, Wright BD. A user’s guide to BIGSTEPS: Raschmodel computer program. Chicago: MESA Press; 1993. 41. Coster W, Ludlow L, Mancini M. Using IRT variable maps to enrich understanding of rehabilitation data. J Outcome Meas 1999;3: 123-33. 42. Steinberg D, Colla P. CART: tree-structured non-parametric data analysis. San Diego (CA): Salford Systems; 1995. 43. Harrell FE Jr, Lee KL, Matchar DB, Reichert TA. Regression models for prognostic prediction: advantages, problems and suggested solutions. Cancer Treat Rep 1985;69: 107 l-7. 44. Stineman MG, Hamilton B, Granger CV, Goin JE, Escarce JJ, Williams SV. Four methods for characterizing disability in the formation of function related groups. Arch Phys Med Rehabil 1994;75:1277-83. 45. Nagi S. Disability concepts revisited: implications for prevention. In: Pope AM, Tarlov AR, editors. Disability in America: toward a national agenda for prevention. Washington (DC): National Academy Press; 1991. p. 309-27.

PARTICIPATION

APPENDIX: Ratings: (Does

1

OF CHILDREN

WhH

DISABILITIES,

DESCRIPTION

OF FUNCTIONAL

2

not perform)

(Partial

(Inconsistent

perform)

Physical Tasks (n = 9) Travel: Moving around the school environment. Maintaining and Changing Positions: Changing positions Recreational Movement: Play activities that involve gross Manipulation With Movement: Retrieving and transporting Using Materials: Using classroom-relevant tools. Setup and Cleanup: Setting up/cleaning up as required Eating and Drinking: Eating a typical school meal. Hygiene: Toileting and other hygiene activities. Clothing Management: Clothing activities as typically Cognitive-Behavioral Tasks fn = 9) Functional Communication: Communicating information Memory and Understanding: Understanding/remembering Following Social Conventions: Demonstrating Compliance With Adult Directive and School Task Behavior/Completion: Showing behavior Positive Interaction: Initiating and maintaining

as required in school motor movement. materials. in school

required

347

Mancini

TASKS*

3 performance)

(Perform

(eg, to and from

desk

4 consistently)

chair).

teg, at lunchtime).

in school

to peers typical

(eg, putting

on a jacket).

and school personnel. school routines.

socially accepted behavior in different school Rules: Cooperating with typical school rules. necessary to accomplish school tasks. interaction with peers in school.

contexts

(eg, classroom,

Behavior Regulation: Regulating responses to negative events in school. Personal Care Awareness: Monitoring personal appearance in school. Safety: Showing caution around dangerous areas and equipment. l

More

complete

descriptions

of physical

and cognitive-behavioral

tasks

are provided

in the manual.15

Arch

Phys

Med

Rehabil

Vol81,

March

2000