Predicted and observed outcomes in preschool children following speech and language treatment: Parent and clinician perspectives

Available online at www.sciencedirect.com

Journal of Communication Disorders 42 (2009) 29–42

Predicted and observed outcomes in preschool children following speech and language treatment: Parent and clinician perspectives Nancy Thomas-Stonell a,b,*, Bruce Oddson c,1, Bernadette Robertson a,2, Peter Rosenbaum d,3 a

Bloorview Research Institute, Bloorview Kids Rehab, 150 Kilgour Road, Toronto, ON, Canada M4G 1R8 b Graduate Department of Speech-Language Pathology, University of Toronto, Canada c School of Human Kinetics, Laurentian University, 935 Ramsey Lake Road, Sudbury, ON, Canada P3E 2C6 d CanChild Centre for Childhood Disability Research, McMaster University, 1400 Main Street W., Hamilton, ON, Canada L8S 1C7 Received 15 November 2006; received in revised form 8 July 2008; accepted 18 August 2008

Abstract Parents of 210 preschool children (age 2–5.7) and their clinicians were asked to describe their expectations for therapy and the changes they observed following treatment. Based on content analysis of the parents’ and clinicians’ responses, it was apparent that the comments aligned with the World Health Organization’s (WHO) International Classification of Functioning, Disability and Health—Child and Youth version (ICF-CY) framework. Parents and clinicians observed positive changes across the ICF-CY domains of Body Functions, Activities and Participation and Personal Factors following therapy. Parents noted twice as many changes in the Participation and Personal Factors domains as clinicians. Parents described improvements in play, socialization, confidence and behaviour at home, school and in the community—changes not typically captured by other preschool speech and language outcome measures. New outcome measures need to be based on actual observations of change by both parents and clinicians to ensure that they measure a sufficiently broad-based range of skills. Learning outcomes: The reader will better understand (1) the parents’ expectations for therapy (2) the types of changes that can be associated with speech and language therapy and (3) the need for broad-based outcome measures that can evaluate speech and language outcomes. # 2008 Elsevier Inc. All rights reserved.

1. Introduction In order for speech–language pathologists to evaluate the effect of their interventions on children’s lives, it is essential that relevant and valid outcome measures be developed (Yorkston, Klasner, & Swanson, 2001). Outcomes

* Corresponding author at: Bloorview Research Institute, c/o Bloorview Kids Rehab, 150 Kilgour Road, Toronto, Ontario, Canada M4G 1R8. Tel.: +1 416 425 6220x3425; fax: +1 416 425 1634. E-mail addresses: [email protected] (N. Thomas-Stonell), [email protected] (B. Oddson), [email protected] (B. Robertson), [email protected] (P. Rosenbaum). 1 Tel.: +1 705 675 1151x1017; fax: +1 705 675 4845. 2 Tel.: +1 416 425 6220x3425; fax: +1 416 425 1634. 3 Tel.: +1 905 525 9140x27834; fax: +1 905 522 6095. 0021-9924/$ – see front matter # 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jcomdis.2008.08.002

30

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

research is needed to determine the impact of speech and language therapy and to inform evidence-based practice (Hammell & Carpenter, 2000). Clinicians need outcome information to learn which children, with which kinds of problems, will benefit most from various types of speech and language therapy (Beitchman, Nair, Clegg, & Patel, 1986). Policy makers need outcome information to help guide policy decisions regarding health and community-based service needs (Majnemer & Mazer, 2004). The first step in these processes is to identify the changes that are associated with speech and language therapy so that we can focus on the collection and evaluation of the most important and relevant outcome information. In the past 10 years, health care paradigms have shifted beyond a focus on impairments to a focus on enhancing a person’s ability to participate in their community. In 2001, the World Health Organization (WHO) published a revised model of Functioning, Disability and Health (ICF) to reflect this change. The new model represents a dynamic system, which has expanded ‘points of entry’ (Rosenbaum & Stewart, 2004). This framework recognizes that interventions at any or perhaps all levels may be important, appropriate and interlinked (Rosenbaum & Stewart, 2004). The new ICF model now includes Contextual Factors (i.e., Environmental and Personal Factors). Environmental Factors can be social, cultural or institutional in nature (see Fig. 1). Personal Factors include gender, upbringing, coping styles, social background and character style (Rosenbaum & Stewart, 2004; WHO, 2001). The ICF model acknowledges that the settings in which people live their lives play a central role in their capacity to function. Recently, the WHO published the International Classification of Functioning, Disability and Health—Children and Youth Version (ICF-CY). This derived classification of the ICF framework provides a common language for documenting and measuring health and disability in children and youth (WHO, 2007, p. vii). The ICF-CY adheres to the structure of the ICF while recognizing that child development is a dynamic process (WHO, 2007, p. xv) The challenge is to translate this framework into practice by shifting beyond a traditional biomedical, impairment-focused treatment model to a focus on adaptation of tasks, provision of environmental facilitators and removal of environmental barriers that interfere with a child’s ability to communicate in their home and community (Rosenbaum & Stewart, 2004). Several researchers have recommended that clinicians use the ICF framework to develop treatment goals that focus on Activities and Participation (McLeod, 2004; McLeod & Bleile, 2004; Threats & Worrall, 2004). To measure the impact of such treatment goals on children’s lives, we must have appropriate measures that allow the profession to document and report outcomes across the entire ICF framework from physical deficits to barriers to participation (Simmons-Mackie & Damico, 2001; Threats, 2003). A recent literature review found only two comprehensive speech–language pathology outcome measures for preschool children: the American-Speech-Language-Hearing Association National Outcome Measure System (Pre-K NOMS) and the Therapy Outcome Measures (TOMs) (ASHA, 1996; Enderby, 1997). The Pre-K NOMS was developed by a committee of content experts, peer reviewed for face validity, field-tested and then revised

Fig. 1. World Health Organization’s International Classification of Functioning, Disability and Health (WHO, 2001).

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

31

(Dobrzykowski, 1997). Although widely used across the United States and in parts of Canada, there have been no published studies of the reliability, validity or responsiveness of this measure. A responsiveness study completed by the authors indicated that the Pre-K NOMS significantly underestimated ‘real life’ changes associated with speech– language therapy (Thomas-Stonell et al., 2007). The Pre-K NOMS measures change primarily at the ICF levels of Body Function and Activities, but not at the levels of Participation, Environmental or Personal Factors. In addition, the Pre-K NOMS evaluates communication changes only in the specific areas that correspond to the treatment goals. Both clinicians and parents consistently observed changes in communication skills and abilities such as attention and socialization that were not treatment goals. Developed in England, the TOMs was based on a retrospective review of speech–language pathology charts (Enderby, 1997). It combined a medical model with an earlier version of the WHO framework—the International Classification of Impairment, Disability and Handicap (ICIDH) (WHO, 1980). Clients are assigned to specific diagnostic categories (e.g., dysarthria) and then rated in 4 levels: ‘Impairment’, ‘Disability’, ‘Handicap’ and ‘Wellness/Distress’. The TOMs reflect a broader view of outcome measurement than the Pre-K NOMS by including clinical, functional, and social issues (Roulstone, John, Hughes, & Enderby, 2004). A client receives one score in each rating scale, which has multiple descriptors (e.g., roles/social integration/decision making). This means that measured change cannot be attributed to specific descriptors and there is no way to know how many of the descriptors have changed. An extensive research project was completed in Australia to adapt the measures (AusTOMs) to their health care system, however, the AusTOMs continues to use the earlier linear ICIDH framework (Australian Institute of Health and Welfare, 2003; Perry & Skeat, 2004). A new generation of outcome measures is needed that reflects the revised ICF framework and permits more precise tracking of individual outcomes. Information generated by outcome studies is only useful if the measure is clinically applicable and scientifically sound (van der Putten, Hobart, Freeman, & Thompson, 1999). While the profession is under pressure to collect outcome information, neither available measure is well suited for today’s healthcare paradigm. The Pre-K NOMS has limited responsiveness and no published reliability or validity studies. The TOMs uses the outdated ICIDH framework and provides only general descriptions of change. New outcome measures are needed that incorporate the new ICF health-care paradigm and are based on empirical data collected from clients and families (Rosenbaum, King, & Cadman, 1992; Trigg, Wood, & Hewer, 1999). Qualitative data are needed to provide insights into the changes associated with treatment and to inform the development of future quantitative outcome instruments. A literature review in Medline, CINAHL and international speech–language pathology journals revealed no studies that described, from a holistic viewpoint such as the ICF framework, the changes associated with speech and language treatment. Some preschool program evaluation studies measured outcomes directly related to the program goals, but these did not investigate whether measured outcomes resulted in changes at home or in the community (Gaines & Gaboury, 2004; Girolametto, Pearce, & Weitzman, 1996). While speech–language pathologists have heard anecdotal comments from parents and clients about changes associated with speech and language therapy, these comments have not been collected and examined in a systematic fashion (Simmons-Mackie & Damico, 2001). Qualitative research methods are ideally suited for investigating details associated with treatment outcomes (Damico & Simmons-Mackie, 2003). They allow the researcher to view children in the context of their environment and to understand the dynamics of that environment, for it is there that the consequences of Activity Limitations and Participation Restrictions acquire meaning (Jongbloed, 2000). Content analysis is a research method that uses a set of procedures to make valid inferences from text (Weber, 1990). It combines attributes of both qualitative and quantitative techniques to produce counts of key categories and is ideally suited for analyzing open-ended survey questions in health care research (Hsieh & Shannon, 2005). In this study, content analysis was used to examine the observations of change provided by the parents and clinicians of 210 children who received speech and language therapy. 2. Methods Six organizations in Ontario that provide speech and language services to preschool children collaborated on this study. These organizations were: Bloorview Kids Rehab (Toronto), Children’s Hospital of Eastern Ontario (First Words program) (Ottawa), Ottawa’s Children’s Treatment Centre, Porcupine Health Unit (Timmins), Pathways Health Centre for Children (formerly Sarnia and District Children’s Treatment Centre) and the York Region Preschool Speech and Language Program (Beyond Words). Following ethical approval from all participating centres, all families who had children between the ages of 2 and 6 years receiving speech and language therapy were invited to participate in the

32

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

study. Families were included if their child was between 3 and 6 years of age during the treatment period. Organizations with small speech and language caseloads invited every family whose child met the age criterion to participate. The organizations that served large numbers of preschool children selected the front-line clinicians who typically treat preschool children with communication disorders in the absence of diagnosed medical conditions. These clinicians were selected to increase the cross-section of the population as two of the children’s treatment centres only treated children with associated medical conditions. These clinicians invited every family on their caseload whose child met the age criterion to participate in the study. Written information was made available in both English and French. Although interpreters were available if families did not speak either English or French, this service was not requested. This study enrolled 218 children and their families. Complete data were obtained on 210 children (96%). Eight children were lost to follow-up (four children moved, two clinician data forms were missing due to a job change and two parent follow-up forms were missing). Four questionnaires per child were completed (i.e., parent and clinician start of treatment questionnaires, and parent and clinician completion of treatment questionnaires). A total of 840 questionnaires were reviewed. 3. Demographic information In the present study, the children’s ages ranged from 2.5 to 5.7 years with a mean age of 4.1 years (S.D. = .66). Sixty-eight percent of the children were boys. Approximately 25% of families reported that more than one language was spoken in the home. Treatment was provided in either English (95%) or French (5%). Medical information was collected from the health records of the participating organizations. Thirty-eight percent of the children had associated medical diagnoses. Sixty-two percent had no medical diagnosis when entering treatment. The most prevalent diagnoses identified were Neuromotor Disorders (e.g., cerebral palsy, muscular dystrophy) at 14%, followed by Developmental Delay (10%). The prevalence of Autism and Related Disorders and syndromes (e.g., Down, Prader-Willi) was 5% each. None of the children had progressive conditions. Clinicians classified each child’s communication disorder using the International Classification of Diseases, Ninth revision, Clinical Modification (ICD-9-CM) disorder codes (National Center for Health Statistics, n.d.) While the children had a variety of communication disorders, the three most commonly identified were: Developmental Speech Disorders (41%), Developmental Language Production (22%) and Developmental Language Comprehension Disorders (16%). The mean duration of treatment, reflecting current community-based practice in this province, was 3.3 months, ranging from 2.1 to 6.0 months. To facilitate data collection, a maximum treatment interval of 6 months was established. Treatment frequency ranged from one session every six weeks to five sessions a week; however, consistent with current service delivery models in Ontario, most children (85%) received therapy once a week. The mean length of treatment provided during the treatment block was 8.7 hours (see Fig. 2). Eighty-five percent of therapy sessions lasted between 31 and 60 min. Individual treatment was provided 61% of the time and group treatment 32% of the

Fig. 2. Amount of treatment provided to children in the study. Note: Arrow denotes mean length of treatment.

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

33

time. In the remaining 7%, consultation was provided to community staff and/or parents who were provided with speech and language programs. At the start of treatment, parents and clinicians independently completed a prescribed questionnaire. They rated the child’s skills compared to their peers in six common communication areas: articulation/intelligibility, language comprehension, language production, pragmatics, attention/problem solving and swallowing/chewing. In addition, parents and clinicians completed 3 open-ended questions: 1. Does your child have any other communication needs that you are concerned about? Please describe. 2. What changes are you expecting for your child as a result of therapy? 3. Do you think these changes will make a difference in your child’s ability to communicate independently in real life situations? Yes/No. Explain At the completion of treatment, parents and clinicians re-rated the child’s communication skills compared to their peers using the same questionnaire. After completing these ratings, parents and clinicians were provided with the questionnaires they had completed at the start of treatment. Comparing the ratings from the start and completion of treatment, they then rated whether, and how much change had been observed in the 6 communication areas, using a 0–7 rating scale where 0 = ‘got worse’, 1 = ‘no change’ and 7 = ‘excellent change’. They also responded to a yes/ no question about whether or not they had observed any ‘real life’ communication changes. This methodology is recommended by Russell et al. (1989) to improve the reliability of judgments of change. If parents and clinicians felt that ‘real life’ changes had occurred, they were asked to complete three additional open-ended questions: 4. Following this treatment block, I have noticed my child is now able to . . . 5. Are there any other changes that you have noted in your child that occurred during this treatment block? Please describe. 6. In what way are these changes important to you/your child? Content analysis examines the content of written text in order to describe phenomena, in this case the types of change associated with speech and language treatment. Hsieh and Shannon (2005) recommend that categories be derived from the data to avoid using preconceived categories. The categories that emerge from the message pool are then organized into a set coding schema. Definitions for each category are developed and exemplars for each category identified from the data. The ‘codebook’ is then used to classify text into categories through a systematic coding process. The frequency of each code is counted so that the most recurrent categories can be identified (Hsieh & Shannon, 2005; Neuendorf, 2002, p. 172). Three researchers independently analyzed the questionnaires from 30 children to identify recurring categories in the descriptive data. The researchers subsequently compared and discussed findings in an iterative process, until 100% consensus on the categories was established. Once consensus was reached on the categories, a further 80 questionnaires were reviewed. No new categories emerged, indicating that all of the categories had been identified. A codebook was developed to define the categories for the coding schema and ensure the reliability of the coding. The codebook contained definitions and sample comments for each category. Once the codebook was completed, the balance of the data was analyzed. The reliability of the coding was assessed in two phases. First, 10% of the data were coded independently by two researchers. One of the researchers had participated in the original coding (above). The second researcher had had no prior experience with the data. Initial agreement for each category ranged between 60 and 100% with the majority of differences reflecting an omission of one category. An additional 10% of the definitions were recoded. A coinvestigator reviewed the coding until consensus was achieved. The revised codebooks were applied to the overall data by a third researcher. This procedure resulted in reliability of greater than 90%. 3.1. Coding schema After reviewing the data, the researchers noted that the emerging categories from the start and end of treatment questionnaires aligned with the WHO ICF-CY framework. Since many authors suggest that outcome measures should evaluate changes across ICF/ICF-CY domains, the researchers felt that this coding system would assist them to identify the impact of treatment changes from individual deficits to the home and community (Butler, 1995; Lollar, Simeonsson, & Nanda, 2000; Simmons-Mackie & Damico, 2001). Each component in the ICF-CY framework can be

34

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

described in both positive and negative terms, except for the Personal Factors domain (WHO, 2007, p. 10). Since our data clearly showed categories relating to both positive and negative Personal Factors, we also divided Personal Factors this way. The ICF-CY defines ‘Body Functions’ as ‘‘physiological function of body systems, including psychological functions’’ (WHO, 2007, p. 9). Examples of categories that were coded as Body Functions include: producing speech sounds or single words in isolation, increased tongue tip elevation, improved eye gaze and improved oral motor control. The ICF-CY defines Activity as ‘‘the execution of a task or action by an individual’’ and Participation as ‘‘involvement in life situations’’ (WHO, 2007, p. 9). Distinguishing between these domains has been an ongoing challenge for users of the ICF and the ICF-CY (Threats & Worrall, 2004). To help distinguish between them, comments that did not specifically mention a transfer of skills to a societal setting were coded as ‘Activities’ rather than ‘Participation’. Items within the Activities and Participation domains were further defined using the ICF qualifiers of Capacity and Performance. ‘Capacity’ is defined as ‘‘an individual’s optimal ability to execute a task or action in a standard environment’’ and ‘Performance’ is defined as ‘‘what an individual does in his current environment, which includes a societal context. (WHO, 2007, p. 13). Categories were coded as Activities when they referred to the acquisition of specific skills in the standardized environment of the clinical setting or when comments did not specify the use of specific skills in a ‘societal’ environment. Examples of categories that were coded as Activities/Capacity include: increased vocabulary, increased sentence length, more frequent communication and improved communication abilities. Categories were coded as Participation when they referred to observed changes at home, daycare and school. Examples of categories that were coded as Participation include: socialization with peers, answering questions in circle time and communicating better with other children. The ICF-CY defines Personal Factors as ‘‘the particular background of an individual’s life and living’’ which includes ‘‘features of the individual that are not part of a health condition or health states’’(WHO, 2007, p. 15). Personal Factors include such factors as gender, race, age, other health conditions, upbringing, coping styles, social background, past and current experience, overall behaviour patterns and character style (WHO, 2007, p. 15). Parents’ comments about their child’s frustration, shyness, anxiety and/or temper tantrums when trying to communicate were interpreted as representing the child’s coping style and/or overall behaviour pattern (Costa & McCrae, 1989; Feifel, Strack, & Nagy, 1987). They were coded as Personal Factors—Negative. Comments about such issues as confidence, increased willingness to communicate, decreased shyness and improved behaviour were coded as Personal Factors— Positive. These comments could also have been coded under the Mental Functions codes in the Body Function domain. The Personal Factors domain was selected because these children did not have identified personality impairments and this domain seemed more accurately to reflect the intent of the parents and clinicians who made the comments (McLeod & McCormack, 2007). Examples of parent and clinician comments coded according to the ICF Schema are included in Table 1. Table 1 Examples of parent and clinician comments according to the ICF schema Sample comments

ICF domain

Increased range of movement of jaws/lips Is stimulable for some target sounds Oral motor planning problems Lacks ability to initiate verbalizations Putting sentences together much better Expressing himself better with words Cannot communicate effectively Difficulties with expressive language More social at daycare Starting to play with other children We do not understand what he is trying to tell us Cannot communicate effectively with peers without adult assistance Very shy with limited eye contact I hope to see her communicate verbally rather than frustration and physical

Body Functions—Positive Aspect Body Functions—Positive Aspect Body Functions—Negative Aspect Body Functions—Negative Aspect Activities Activities Activity Limitation Activity Limitation Participation Participation Participation Restriction Participation Restriction Personal Factors—Negative Aspect a Personal Factors—Positive Aspecta

a

The Personal Factors domain was divided into positive and negative aspects for the purposes of this study.

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

35

4. Results The frequency of the categories mentioned by parents and clinicians for each open-ended question is presented below, however, only those domains and categories that were cited by a minimum of 10% of the parents and clinicians have been reported. Comments illustrate the most frequently coded category for parents and clinicians in each domain. 4.1. Start of treatment Question #1: Does your child/client have any other communication needs that you are concerned about? (in addition to communication concerns in: articulation/intelligibility, language comprehension, language production, pragmatics, attention/problem solving and chewing/swallowing, which had been described previously). Sample parent comments: Body Function Activity Limitation Participation Restriction Personal Factors—Negative

‘‘To pronounce his words correctly’’ ‘‘He cannot communicate effectively or clearly.’’ ‘‘She is unable to carry on a conversation that anyone can understand.’’ ‘‘She tends to get frustrated and cries when she is not understood.’’

Forty percent of the parents (84/210) had additional concerns. These concerns crossed the ICF-CY domains of: Body Functions, Activity Limitations, Participation Restrictions and Personal Factors. Parents’ comments focused predominately on their child’s difficulties in Body Functions (40%) and Activity Limitations (48%). The most frequently recurring categories in the Activity Limitations domain included: difficulties forming sentences, using grammar, carrying on a conversation and limited vocabulary. Recurring concerns in the Body Functions domain were the child’s difficulties vocalizing, producing speech sounds and stuttering. Parents also mentioned concerns in the Participation Restrictions (27%) and in the Personal Factors (25%) domains. They expressed concerns about not being able to understand their child, other adults and children not understanding their child, and social isolation. They also were concerned that poor communication skills were causing significant frustration and behavioural difficulties (e.g., tempers, outbursts). Sample clinician comments: Body Function Activity Limitation Participation Restriction Personal Factors—Negative

‘‘Has motor planning difficulties which affect expressive abilities.’’ ‘‘Difficulties with expressive language.’’ ‘‘He rarely engages socially with his peers.’’ ‘‘Worry about increasing frustration due to not being understood.’’

Thirty-five percent of clinicians (73/210) reported additional concerns, which also crossed the ICF-CY domains: Body Functions, Activity Limitations, Participation Restrictions and Personal Factors. Most of the clinicians’ concerns related either to impairments of Body Functions (45%) or Activity Limitations (50%). Recurring concerns related to Body Functions were difficulties with oral motor skills and fluency. Recurring concerns related to Activity Limitations were difficulties with expressive language and difficulties communicating effectively. Clinicians’ concerns at the level of Body Functions and Activity Limitations were similar to those noted by parents, however, clinicians were much less likely to comment on concerns in the Participation Restrictions and Personal Factors domains. A comparison of parent and clinician concerns is provided in Table 2. Question #2: What changes are you expecting for your child/client as a result of therapy? Sample parent expectations: Body Function Activities/Capacity Participation Personal Factors—Positive

‘‘Will pronounce words better.’’ ‘‘Able to put sentences/stories together and be understood.’’ ‘‘To communicate effectively in all situations (i.e., school, play dates, sports, birthday parties).’’ ‘‘His self-confidence will increase, frustration will decrease.’’

Ninety-seven percent of the parents (203/210) responded to this question. The parents predicted that speech and language therapy would make changes across the ICF-CY levels of Body Functions (49%), Activities (63%), Participation (29%). Parents predicted that treatment would result in improvements in their child’s ability to

36

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

Table 2 A comparison of clinicians’ and parents’ comments at the start and completion of treatment Clinicians

Parents

Start of treatment Concerns (Q1) Body Functions and Structures Activity Limitations Participation Restrictions Personal Factors—Negative

45 50 11 13

40 48 27 25

Predicted change (Q2) Body Functions and Structures Activity Participation Personal Factors—Positive

39 85 11 4

49 63 29 16

Completion of treatment Observed change (Q4) Body Functions and Structures Activity Participation Personal Factors—Positive

44 79 13 10

38 74 28 15

Other changes (Q5) Body Functions and Structures Activity Participation Personal Factors—Positive

14 62 16 47

21 53 27 44

Average Observed Changes (Q4 & Q5) Body Functions and Structures Activity Participation Personal Factors—Positive

29 71 15 29

30 64 28 30

Note: The values represent percentages calculated by dividing the number of comments in each domain by the total number of clinicians and parents who responded to each question.

communicate and that it would improve their child’s relations with peers and their readiness for school. A small percentage of parents (16%) predicted that there would also be improvements in the Personal Factors area, resulting in greater confidence and less frustration. Sample clinician expectations: Body Function Activities/Capacity Participation

‘‘Clearer speech. Produce /f/, /sh/ and /s/ blends.’’ ‘‘Have him be able to express himself in one to two word utterances.’’ ‘‘To functionally communicate wants and needs with familiar listeners.’’

All clinicians (209/210) predicted that treatment would be associated with changes primarily at the ICF-CY levels of Body Functions (39%) and Activities (85%). Clinicians’ expectations included: improved articulation/phonology, improved expressive language skills, increased mean length of utterance, expanded vocabulary and more communication attempts. Only 11% of clinicians expected treatment to make changes in the child’s ability to participate in the community compared to 29% of parents. Although the clinicians noted concerns in the Personal Factors areas, only 4% of them predicted that treatment would have a positive effect in this area. Question #3: Do you think these changes will make a difference in your child’s ability to communicate independently in real life situations? Explain. Sample parent comments: Activities/Capacity Participation Personal Factors—Positive

‘‘She very much knows what it is she is wanting to say, but isn’t saying it effectively. Our hope is that at end of therapy she’ll be able to do so.’’ ‘‘So that other people can interpret her likes and dislikes when mom and dad are not around.’’ ‘‘Give her more confidence so not as shy around others.’’

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

37

All 210 parents responded to this question. Eighty-four percent stated that they thought that therapy would make a ‘real life’ difference. They indicated that therapy would enhance Participation (71%), Personal Factors (39%) and Activities/Capacity (24%). Examples of predicted improvements in Participation were: being understood by extended family and teachers, being accepted by their peers and being ready for school. Predicted improvements in Personal Factors included: being less shy, reducing frustration and becoming more confident. Expected improvements in Activities/Capacity were: using more complete sentences, communicating better and more often. Sample clinician comments: Activities/Capacity Participation Personal Factors—Positive:

‘‘Allow him to be a better communicator.’’ ‘‘Socialization and expressive language skills from group treatment setting expected to carry over to other environments.’’ ‘‘Increased confidence to communicate verbally, less frustration when not understood.’’

Clinicians answered this question for all of the 210 participants. Eighty-seven percent indicated that therapy would be associated with real life changes in the domains of Activities (49%), Participation (66%) and Personal Factors (30%). All of the clinicians predicted that treatment would improve communication skills. Some thought that therapy might also improve the child’s attention. 4.2. Completion of treatment Question #4: Following treatment my child/client is now able to . . . Sample parent comments: Body Functions Activities/Capacity Participation Personal Factors—Positive

‘‘Pronounce words much more clearly, specifically /f/ and the /l/ sound when prompted.’’ ‘‘Speak in simple sentences from time to time, rather than keywords and labels.’’ ‘‘Play freely with others without children or adults always asking him to repeat himself because they can’t understand what it is he is saying.’’ ‘‘Has developed her confidence.’’

Parents described changes for 96% (201/210) of the participants. Parents observed changes at the ICF-CY levels of Body Functions (38%), Activities/Capacity (74%), Participation (28%) and Personal Factors—Positive (15%) (see Table 2). Recurring comments specific to communication skills pertained to improved expressive language skills, improved articulation/speech skills and improved language comprehension. Parents also commented on improvements in attention, socialization, emotions and quality of life. Sample clinician comments: Body Functions Activities/Capacity Participation Personal Factors—Positive

‘‘Good articulation at the word level.’’ ‘‘Using many spontaneous single words, some combinations in structured tasks. With some cueing, can produce complete sentence forms.’’ ‘‘Is greeting others; Is more oriented towards communication with others.’’ ‘‘Decreased shyness. Opened up!’’

Clinicians described changes for 98% (206/210) of the participants. They observed changes at the ICF-CY levels of Body Functions (44%), Activities/Capacity (79%), Participation (13%) and Personal Factors—Positive (10%). Comments pertained to improvements in expressive language skills, articulation, intelligibility and pragmatic skills. Clinicians also observed changes in attention and socialization. Clinicians’ and parents’ observations of change were similar, although parents were twice as likely to comment on changes at the level of Participation as the clinicians. Question #5: Are there any other changes you have noticed in your child/client during this treatment block? Sample parent comments: Body Functions Activities/Capacity Participation Personal Factors—Positive

‘‘Using complete words now.’’ ‘‘Trying to put words together and communicate. ‘‘His play with peers has improved in terms of sharing, turn-taking, following conversations.’’ ‘‘Now I feel comfortable leaving her at school and daycare and not worrying as much, I’m glad people can understand her now.’’

38

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

Seventy-six percent of parents (160/210) observed ‘other’ changes. These occurred at the ICF-CY levels of Body Functions (21%), Activities/Capacity (53%), Participation (27%) and Personal Factors—Positive (44%). Parents primarily noted positive changes in the Activities and Personal Factors—Positive domains. Parents commented that their child was communicating better and was happier, more confident and having fewer temper tantrums. Parents’ observations of change generally matched their expectations, although they noted fewer changes at the ICF-CY level of Body Functions and more changes in the Personal Factors—Positive domain than expected. Sample clinician comments: Body Functions Activities/Capacity Participation Personal Factors—Positive

‘‘More willing to attempt words and sounds.’’ ‘‘She is also able to maintain a conversation beyond one turn.’’ ‘‘This child is able to share ‘news’ and novel information from one setting/communication partner to another, e.g. home to school and back.’’ ‘‘Reduced frustration and ‘acting out’ behaviours as she can now use words to express her needs better.’’

Seventy percent of the clinicians (146/210) noted ‘other’ changes. These included positive changes at the ICF-CY level of Activities (62%), Personal Factors—Positive (47%), Participation (16%) and Body Functions (14%). Clinicians commented primarily on improvements in speech and language areas that had not been the primary focus of treatment. Pragmatic skills were most frequently cited by clinicians as having improved. Clinicians also commented on improvements in the child’s confidence, attention and an increased willingness to try to communicate with others. Clinicians observed fewer changes than expected in the Body Functions and Activities/Capacity domains. Only 4% of clinicians predicted changes in the Personal Factors—Positive domain, however, 29% of clinicians commented on improvements in this area (see Table 2). Question #6: In what ways are these changes important to you/your child? Sample parent comments: Activities/Capacity Participation Personal Factors—Positive

‘‘He is putting together sentences much better.’’ ‘‘Able to express himself in play groups without parents.’’ ‘‘She is a happier, easier-going child. She isn’t overlooked at school because now the teacher can understand her.’’

Eighty-six percent of the parents (181/210) responded to this question. While most of the comments (75%) pertained to the importance of the achieved changes on their child’s life currently, some parents commented on the importance of these changes for their child’s future. Anticipated benefits of treatment focused on improvements at the level of Participation. Parents felt that treatment would improve their child’s quality of life. They commented that the child would be more ready for school and have greater academic and life success. When parents commented on the importance of the achieved changes, the majority (72%) noted the impact of changes in Participation domain. They talked about their child’s ability to carry on conversations, socialize with their peers and participate in school. Parents also commented on the impact of these changes in the Personal Factors— Positive (48%) and Activities/Capacity (29%) domains. Parents commented on improvements in the child’s confidence and quality of life (e.g., doing better in school, becoming more independent). Sample clinician comments: Body Functions Activities/Capacity Participation Personal Factors—Positive

‘‘Client is attempting speech sounds.’’ ‘‘Communication book has given her means to further expand her expressive language and generate novel sentences.’’ ‘‘This child has begun to separate from his mom and relate to another boy in the group. He began to smile and laugh and react in a positive manner.’’ ‘‘More confident communicator.’’

Eighty-two percent of clinicians (172/210) commented on the importance of the observed changes. Most comments (77%) pertained to the impact of achieved changes on the child’s life currently. Clinicians commented on the impact of these changes at the level of Activities/Capacity (55%) and Participation (52%). One third of the clinicians (33%)

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

39

commented on positive changes in the child’s emotions (Personal Factors—Positive). They noted that the children were more intelligible and becoming successful communicators. Improvements were also noted in socialization and independence. 5. Discussion At the start of treatment, both parents and clinicians expressed concerns about the children’s Body Functions and Activity Limitations. Fifty-two percent of parents stated that their child was experiencing Participation Restrictions or negative Personal Factors (e.g., shyness, anxiety, frustration, behaviour problems, temper tantrums) as a result of their communication impairments. Parents were twice as likely as clinicians to note concerns in these areas. This difference reflects the profession’s bias towards an impairment-based model of speech and language treatment. As McLeod and Bleile (2004) suggest, this bias likely reflects the medical and behavioural frameworks of the profession. The use of the ICF-CY framework as a conceptual model for treatment helps ensure that clinicians consider all aspects of a child’s life (McLeod, 2006). Clinicians need to become more aware of the impact of communication disorders on children’s ability to ‘participate’ within their communities and on their emotional health. They should routinely obtain more information about the child and family’s life contexts so that they are able to identify important Participation Restrictions that are clearly obvious to parents. Clinicians and parents differed in their expectations of treatment. Although parents expected to see some changes across the ICF-CY domains of Body Functions, Activities/Capacity, Participation and Personal Factors, over 72% hoped treatment would improve their child’s ability to ‘participate’ at home and in the community. Almost 40% of parents hoped that treatment would also have a positive impact on their child’s emotions and behaviour. Clinicians expected that treatment would primarily be associated with changes in Body Functions and Activities/Capacity and focused their treatment goals on these domains (reflecting the traditional biomedical focus of our interventions). Although 13% of clinicians noted concerns in the Personal Factors domains, only 4% predicted that there would be positive changes in this domain following treatment. These findings suggest a need for greater collaboration between parents and clinicians in the formulation of therapy goals to ensure that treatment goals address both parent and clinician concerns. Predicted changes were compared to the changes (average of responses to Questions #4 and #5) actually observed by both parents and clinicians. Both parents and clinicians reported that therapy had had an important positive effect on the children’s lives. Some of the observed improvements occurred in areas predicted by parents and clinicians, but some improvements were noted in areas not predicted by either parents or clinicians (see Table 2 for a comparison of Predicted vs. Average Observed Changes). Both parents and clinicians observed fewer improvements in the Body Function domain than predicted. Parents observed changes in the Personal Factors domain twice as often as predicted. Clinicians observed changes in the Personal Factors domain seven times more often than predicted. Parents were more aware of their child’s participation restrictions than clinicians. They noted improvements in this area after treatment twice as often as clinicians. Clinicians need to broaden their treatment goals to include Participation and Personal Factors. The new ICF-CY model is multi-directional, and provides clinicians the freedom to broaden their treatment approaches. It suggests that therapy focused on improving ‘participation’ skills such as improved play, attention and socialization may result in improved ‘communication’ skills. Spending more time playing with their peers increases children’s exposure to positive language models and provides more opportunities for practicing communication skills. As predicted, improvements were observed in communication skills. Parents and clinicians noted positive changes in expressive language, receptive language, vocabulary, pragmatics and articulation skills after treatment. In addition, parents and clinicians consistently reported improvements in attention, socialization and the child’s emotions (i.e., more confidence, reduced frustration, fewer tempers). Thirty-nine percent of parents stated that therapy had had a positive impact on their child’s quality of life. Parents reported that their child was playing more with friends and siblings and doing better in school. They noted positive changes in school participation and in academic skills such as reading readiness and independence. It is obviously important that speech–language pathology outcome measures for preschool children ask questions about these important skill areas, if they are to be responsive to treatment changes. Ma, Worrall, & Threats (2007) state that ‘‘The translation between the ICF conceptual framework and assigning specific codes is not always straight-forward nor will it likely receive universal agreement.’’ The authors interpreted the parents’ and clinicians’ comments regarding their child’s frustration and/or confidence in communication situations as referring to the children’s coping styles and coded them in the Personal Factors domain (Costa & McCrae,

40

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

1989; Feifel et al., 1987). It was felt that becoming frustrated in response to communication breakdown was an ageappropriate coping strategy for a young child and it was possible that these children were confident in noncommunicative contexts. As McLeod and Threats (2008) point out developmental ages and stages must be considered when applying the ICF codes to children. The parents and clinicians who commented that the children were more confident communicators after therapy felt that speech and language therapy had removed an environmental barrier to communication (Contextual Factor), thus diminishing the amount of stress experienced by the child. Alternatively, these comments could be interpreted as referring to impairments of Personality and Temperament in the Mental Functions section of the Body Function domain or as restrictions in the Activities and Participation domain (d240— Handling stress and other psychological demands or d250—Managing ones own behaviour). As Ma et al. (2007) note, there are grey classification areas in speech–language pathology, which are beyond the scope of this paper. This study examined parents’ and clinicians’ perceptions of change following speech and language treatment. The descriptive data were gathered prospectively from the parents and clinicians of 210 children who received speech and language therapy. While there were no independent measures collected to ‘prove’ that these changes occurred, both parents and clinicians independently described similar changes. The similarity and frequency of the comments across such a large sample of children is compelling. Clinicians’ and parents’ comments may reflect a bias towards improvements however it is important to note that not all of the clinicians and parents observed improvements. Also, the types of change observed by parents and clinicians differed from their predictions or expectations at the start of treatment. This study suggests several areas that may be associated with the provision of speech and language treatment, but it does not prove that therapy caused these changes. The study does, however, highlight the need for new quantitative outcome measures that can evaluate changes across the ICF-CY framework systematically. Such outcome measures would be a valuable resource for current clinical practice. The observations from this study can form the basis for the creation of an outcome measure, which reflects the ICF/ICF-CY framework. Once developed, this outcome measure could be used to evaluate the efficacy of speech and language therapy. 6. Conclusions Parents and clinicians observed important positive changes in children following speech and language therapy. After receiving an average of 8 hours of treatment, improvements were noted in communication skills, attention, play, socialization, confidence and behaviour. Overall, parents’ and clinicians’ comments described changes in the same domains, although there was somevariation in the frequency of observed changes within these domains. Parents were twice as likely as clinicians to note the negative impact of a communication disorder on the child’s ability to participate in school/ play activities and on their emotional health (e.g., shyness, anxiety, frustration, behaviour problems, temper tantrums). This research has implications for the development of outcome measures. It suggests that the content of outcome measures should be based on actual observations of change collected from parents and clinicians. Outcome measures such as the ASHA NOMS, developed using expert clinician groups to predict which areas should be measured, do not capture a sufficiently broad-based range of skills, resulting in an instrument that cannot measure many of the positive changes associated with speech and language treatment. This research supports the recommendation of Trigg et al. (1999) that qualitative information be obtained prior to test development and used to generate test items. This ensures that the items are relevant to the population concerned and ensures that the measure has high content validity (Rosenbaum et al., 1992; Frytak, 2000). Clinical practice is shifting from a traditional biomedical, impairment-focused treatment model to a focus on adaptation of tasks and the removal of environmental barriers that interfere with a child’s ability to communicate in their home and community (Rosenbaum & Stewart, 2004). Outcome measures need to be developed that reflect the new ICF-CY framework and echo this broader approach to rehabilitation. We believe that work such as we have described here is an important first step in this process and will be a valuable addition to our resources to evaluate clinical practice. Acknowledgments The authors wish to acknowledge the financial support of Ontario Ministry of Health and Long-Term Care and Ministry of Community, Family and Children’s Services; and The Hospital for Sick Children Foundation. The authors

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

41

express sincere appreciation to the families who participated in this research project. We especially wish to acknowledge the Research Coordinators and Speech-Language Pathology clinical staff at the: Children’s Hospital of Eastern Ontario – First Words Program, Ottawa Children’s Treatment Centre, Pathways Health Centre for Children (formerly Sarnia and District Children’s Treatment Centre), Porcupine Health Unit, York Region Preschool Speech and Language Program (Beyond Words), and Bloorview Kids Rehab for their clinical expertise and assistance with data collection. Appendix A. Continuing education (1) The International Classification of Functioning, Disability and Health is a framework for the description of health and health-related states. True/False (2) Outcome measures need to be valid, responsive and relevant to clinical practice. True/False (3) Outcome measures allow the profession to measure the impact of treatment. True/False (4) Content analysis uses attributes of both qualitative and quantitative techniques to make valid inferences from text. True/False (5) Outcome measures need to reflect a broader approach to rehabilitation. True/False References American Speech-Language-Hearing Association (ASHA). (1996). National Outcome Measurement System (NOMS) for Speech–Language Pathology. Rockville Pike, MD: ASHA. Australian Institute of Health and Welfare. (2003). The ICF Australian User Guide Version 1.0. Retrieved August 16, 2005. Beitchman, J. H., Nair, R., Clegg, M., & Patel, R. (1986). Prevalence of speech and language disorders in 5-year-old kindergarten children in the Ottawa-Carleton region. Journal of Speech and Hearing Disorders, 51, 98–110. Butler, C. (1995). Outcomes that matter [Editorial]. Developmental Medicine & Child Neurology, 37, 753–754. Costa, P. T., & McCrae, R. R. (1989). Personality, stress and coping: Some lessons from a decade of research. In B. S. Markides & C. L. Cooper (Eds.), Aging, stress and health (pp. 269–285). New York: John Wiley and Sons. Damico, J. S., & Simmons-Mackie, N. (2003). Qualitative research and speech–language pathology: A tutorial for the clinical realm. American Journal of Speech–Language Pathology, 12, 131–143. Dobrzykowski, E. A. (1997). Functional communication measures. Journal of Rehabilitation Outcomes Measurement, 3(3), 57–58. Enderby, P. (1997). Therapy outcome measures: Speech–language pathology user’s manual. London, England: Singular Publishing Group Inc. Feifel, H., Strack, S., & Nagy, V. T. (1987). Coping strategies and associated features of medically ill patients. Psychosomatic Medicine, 49, 616– 625. Frytak, J. (2000). Measurement. Journal of Rehabilitation Outcomes Measurement, 4, 15–31. Gaines, B. R., & Gaboury, I. (2004). Toddler talk: Outcomes from a parent-focused intervention for children with speech/language problems. Journal of Speech–Language Pathology, 28(4), 173–183. Girolametto, L., Pearce, P. S., & Weitzman, E. (1996). Interactive focused stimulation for toddlers with expressive vocabulary delays. Journal of Speech and Hearing Research, 39(6), 1274–1283. Hammell, K., & Carpenter, C. (2000). Evaluating qualitative research. In K. W. Hammel, C. Carpenter, & I. Dyck (Eds.), Using qualitative research: A practical introduction for occupational and physical therapists (pp. 1–12). New York: Harcourt Publishers. Hsieh, H., & Shannon, S. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. Jongbloed, L. (2000). Choosing the methodology to explore the research. In K. W. Hammel, C. Carpenter, & I. Dyck (Eds.), Using qualitative research: A practical introduction for occupational and physical therapists (pp. 13–21). New York: Harcourt Publishers Ltd. Lollar, D. J., Simeonsson, R., & Nanda, U. (2000). Measures of outcomes for children and youth. Archives of Physical Medicine and Rehabilitation, 81, S46–S52. Ma, E.P.-M., Worrall, L., & Threats, T. T. (2007). The International Classification of Functioning, Disability and Health (ICF) in Clinical Practice. Seminars in Speech and Language, 28(4), 241–243. Majnemer, A., & Mazer, B. (2004). New directions in the outcome evaluation of children with cerebral palsy. Seminars in Pediatric Neurology, 11(1), 11–17. McLeod, S. (2004). Speech pathologists’ application of the ICF to children with speech impairment. Advances in Speech–Language Pathology, 6(1), 75–81. McLeod, S. (2006). An holistic view of a child with unintelligible speech: Insights from the ICF and ICF-CY. Advances in Speech–Language Pathology, 8(3), 293–315. McLeod, S., & Bleile, K. (2004). The ICF: A framework for setting goals for children with speech impairment. Child Language Teaching and Therapy, 20(3), 199–219. McLeod, S., & McCormack, J. (2007). Application of the ICF and ICF-Children and Youth in children with speech impairment. Seminars in Speech and Language, 28(4), 254–264.

42

N. Thomas-Stonell et al. / Journal of Communication Disorders 42 (2009) 29–42

McLeod, S., & Threats, T. T. (2008). The ICF-CY and children with communication disabilities. International Journal of Speech–Language Pathology, 10(1–2), 92–109. National Center for Health Statistics. (n.d.) International Classification of Diseases, Ninth Revision, Clinical Modification. Retrieved December 11, 2007, from US Department of Health and Human Services Centre for Disease Control website: http://www.cdc.gov/nchs/about/otheract/icd9/ abticd9.htm. Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage Publications, Inc.. p. 172. Perry, A., & Skeat, J. (2004). Aus TOMS for speech pathology. Melbourne: LaTrobe University. Rosenbaum, P., King, S. M., & Cadman, D. (1992). Measuring processes of caregiving to physically disabled children and families. Part I: Identifying relevant components of care. Developmental Medicine and Child Neurology, 34, 103–114. Rosenbaum, P., & Stewart, D. (2004). The world health organization international classification of functioning, disability and health: A model to guide clinical thinking, practice and research in the field of cerebral palsy. Seminars in Pediatric Neurology, 11(1), 5–10. Roulstone, S., John, A., Hughes, A., & Enderby, P. (2004). Assessing the construct validity of the therapy outcome measure for preschool children with delayed speech and language. Advances in Speech–Language Pathology, 6(4), 230–236. Russell, D. J., Rosenbaum, P., Cadman, D. T., Gowland, C., Hardy, S., & Jarvis, S. (1989). The Gross Motor Function Measure: A means to evaluate the effects of physical therapy. Developmental Medicine and Child Neurolology, 31, 341–352. Simmons-Mackie, N., & Damico, J. S. (2001). Intervention Outcomes: A clinical application of qualitative methods. Topics in Language Disorders, 21(4), 21–36. Thomas-Stonell, N., McConney-Ellis, S., Oddson, B., Robertson, B., & Rosenbaum, P. (2007). An evaluation of the responsiveness of the PreKindergarten ASHA NOMS. Canadian Journal of Speech–Language Pathology and Audiology, 31(2), 74–82. Trigg, R., Wood, V. A., & Hewer, R. L. (1999). Social reintegration after stroke: The first stages in the development of the subjective index of physical and social outcome (SIPSO). Clinical Rehabilitation, 13, 341–353. Threats, T. T., & Worrall, L. (2004). Classifying communication disability using the ICF. Advances in Speech–Language Pathology, 6(1), 53–62. Threats, T. T. (2003). The framework for ASHA’s revised scope of practice in speech–language pathology. Speech Pathology Online. Retrieved October 25, 2006, from www.speechpathology.com. van der Putten, J., Hobart, J. C., Freeman, J. A., & Thompson, A. J. (1999). Measuring change in disability after inpatient rehabilitation: Comparison of the responsiveness of the Barthel Index and the Functional Independence Measure. Journal of Neurology Neurosurgery & Psychiatry, 66(4), 480–484. Weber, R. P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage Publications Inc. World Health Organization. (1980). International Classification of Impairments, Disabilities and Handicaps. Geneva: World Health Organization. World Health Organization. (2001). International Classification of Functioning, Disability and Health. Geneva: World Health Organization. World Health Organization (2007). International Classification of Functioning, Disability and Health—Children and Youth Version. Geneva: World Health Organization. Yorkston, K., Klasner, E., & Swanson, K. (2001). Communication in context: A qualitative study of the experiences of individuals with multiple sclerosis. American Journal of Speech–Language Pathology, 10, 126–137.