Assessment of social behavior in children with autism: The development of the Behavioral Assessment of Social Interactions in Young Children

Research in Autism Spectrum Disorders 5 (2011) 351–360

Contents lists available at ScienceDirect

Research in Autism Spectrum Disorders Journal homepage: http://ees.elsevier.com/RASD/default.asp

Assessment of social behavior in children with autism: The development of the Behavioral Assessment of Social Interactions in Young Children Jennifer M. Gillis a,*, Emily H. Callahan b, Raymond G. Romanczyk b a b

Auburn University, United States Binghamton University, United States

A R T I C L E I N F O

A B S T R A C T

Article history: Received 7 April 2010 Accepted 13 April 2010

There are a limited number of assessments available to examine social skills deficits in young children with Autism Spectrum Disorders (ASDs). The Behavioral Assessment of Social Interactions in Young Children (BASYC) was developed as a direct assessment of social deficits in young children with ASD relative to children without ASD. The BASYC is a semi-structured assessment designed to be administered by clinicians and teachers working with children with a possible ASD. The purpose of this study was to determine whether the BASYC discriminates social behaviors between children with and without ASD. There were 77 participants (n = 48 children with ASD; n = 29 children without ASD) in this study. Scores on the BASYC significantly predict group membership. Sensitivity and specificity of the BASYC was .977 and .871, respectively. Item discrimination indices revealed that the majority of items on the Social Responsivity scale discriminated well between groups; however, this was not the case for the Social Initiation scale. Although additional research is required, the BASYC is currently an instrument that is easy to administer, discriminates well between children with and without ASD based on social behaviors and may assist in goal planning and monitoring of social skills treatment progress. ß 2011 Elsevier Ltd. All rights reserved.

Keywords: Autism Social interactions Behavioral Assessment

1. Introduction Delays and deficits in the development of social interaction skills impede an individual’s ability to develop meaningful social relationships and have been associated with poor academic achievement, peer rejection, anxiety, depression, and other forms of psychopathology (Bellini, Peters, Bennett, & Hopf, 2007). Social interaction deficits are considered to be the core deficit of autism (Kanner, 1943; Mundy, Sigman, & Kasari, 1994; Travis & Sigman, 1998) and some researchers suggest that the presence of deficits in reciprocal social behavior distinguishes autism from other psychiatric disorders (Constantino & Todd, 2003). The diagnostic and statistical manual of mental disorders, 4th ed., text revision (DSM-IV-TR) of the American Psychiatric Association (2000), the current standard for diagnosis, specifies that the Pervasive Developmental Disorders (PDDs) include: Autistic Disorder, Asperger’s Disorder, Pervasive Developmental Disorder—Not Otherwise Specified (PDD-NOS), Rett’s Disorder, and Childhood Disintegrative Disorder. The term ‘ASD’ or autism spectrum disorder is now in common use, but is imprecise. The National Institute of Mental Health (NIMH) includes all of the disorders within the Pervasive Developmental Disorders as part of their definition of ASD (http://www.nimh.nih.gov/health/publications/autism/complete-index.shtml) whereas the American Academy of Pediatrics includes only Autistic Disorder, Asperger’s Disorder, and PDD-NOS under their use of the term ASD (http://www.aap.org/healthtopics/Autism.cfm).

* Corresponding author at: 226 Thach, Department of Psychology, Auburn University, Auburn, AL 36830, United States. Tel.: +1 334 844 6477; fax: +1 334 844 4447. E-mail address: [email protected] (J.M. Gillis). 1750-9467/$ – see front matter ß 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.rasd.2010.04.019

352

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

For our purposes we will refer to ASD as defined by the American Academy of Pediatrics. The three primary categories of symptom presentation in DSM-IV-TR for Pervasive Developmental Disorder are: 1. qualitative impairment in social interaction; 2. qualitative impairments in communication; 3. restricted repetitive and stereotyped patterns of behavior, interests, and activities. The one category that Autistic Disorder, Asperger Disorder, and Pervasive Developmental Disorder—Not Otherwise Specified all share is qualitative impairment in social interaction, and is required for diagnosis. Interestingly, this mirrors the original observations of Leo Kanner who first described autism. He wrote that ‘‘There is from the start an extreme autistic aloneness that, whenever possible, disregards, ignores, shuts out anything that comes to the child from the outside.’’ Kanner summed up his observations by stating that ‘‘The outstanding, ‘‘pathognomonic’’, fundamental disorder is the children’s inability to relate themselves in the ordinary way to people and situations from the beginning of life’’. p. 242 (Kanner, 1943). This emphasis on social interaction is generally agreed upon as the defining characteristic in ASD as well as a significant treatment challenge. 1.1. Development of social behavior Requesting and offering are two early social behaviors often observed in children between 12 and 24 months of age (Franco & Butterworth, 1996; Hay & Murray, 1982). Offers and requests shown through pointing and preverbal communication methods, like gestures and vocalizations, serve as both responding and initiating behaviors for social interaction. Social initiation behaviors occur independent of a prompt from another person; whereas responding occurs after a social initiation from another person. Both are considered developmental predictors of social competence (i.e., information processing of social situation that involves interpretation of interpersonal and affective cues as social, and a decision to make a behavioral response appropriate to the situation), and additional complex social skills (i.e., perspective-taking, goaldirected social exchanges, etc.; Crick & Dodge, 1994; Hay & Murray, 1982; McEvoy, Rogers, Pennington, 1993). 1.2. ASD and social behavior Social deficits observed in children with ASD include difficulties understanding the facial expressions of others, initiating social interactions, responding to the social bids of others (Hauck, Fein, Waterhouse, & Feinstein, 1995), and responding to the emotions of others (Sigman, Kasari, Kwon, & Yirmiya, 1992). Previous research has also identified specific social behaviors that discriminate children with ASD from those without autism at 2 years of age. These behaviors include:  deficits in showing (directing attention) (Stone, Coonrod, Ousley, 2000);  decreased attention to the voice of others (Townsend, Harris, & Courchesne, 1996);  the use of another’s body as a tool (e.g., taking another’s hand and putting it on a toy in an attempt to have the person turn it on);  little to no interest in other children;  limited use of pointing to communicate with others;  deficits with understanding gestures;  lack of seeking to share enjoyment with others (joint attention) (Lord, 1995). Research has also shown that the social interactions of individuals with ASD are significantly different from those with other developmental disabilities (e.g., Down syndrome). Individuals with ASD have been found to be less likely than those with other developmental disabilities to orient to social stimuli, to respond to the social bids of others, and to initiate social interactions with same aged peers and adults (Dawson, Meltzoff, Osterling, Rinaldi, & Brown, 1998; Hauck et al., 1995; Jackson et al., 2003; Sigman & Ruskin, 1999). 1.3. Assessing social deficits Most habilitative curricula developed for individuals with ASD include specific social skills instruction or training. Social deficits in children (both typically developing and with autism) have been correlated with lower levels of academic achievement and lower scores on measures of cognitive and emotional development (Hartup & Moore, 1990; Hughes & Hall, 1987; McClellan & Katz, 2001). Attention to the development of specific interventions to improve social skills in individuals with ASD has been increasing (Barry et al., 2003; Bellini & Peters, 2008; Elliott, Gresham, Frank, & Beddow, 2008; Koegel, Koegel, & McNerney, 2001; Strain & Hoyson, 2000; White, Koenig, & Scahill, 2007). Thus, the accurate assessment of children’s social skills in order to select appropriate goals, specific behaviors, and skills for intervention would seem critical. 1.3.1. Measures currently used to assess social deficits Currently there are a number of tools designed to assess the development of social skills (for a review see Matson & Wilkins, 2009, for a review of social skills assessments for children with ASD, see Matson & Wilkins, 2007). Of the tools that have been

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

353

used with children with ASD, the majority rely on third-party or self-report; for example, the Pervasive Developmental Disorders Behavior Inventory (PDDBI; Cohen & Sudhalter, 2005), and the Social Communication Questionnaire (Rutter, Bailey, & Lord, 2003). Wing and Gould developed a social sub-typing system to assess social skills development in children with autism (Wing, 1981; Wing & Gould, 1979). Other assessments that are useful for identifying children who may be displaying deficits or delays in social functioning but do not provide information about the specific nature of the social difficulties for a child with ASD relative to a child without ASD include the Matson Evaluation of Social Skills with Youngsters (MESSY; Matson, Rotatori, & Helsel, 1983), the Vineland Adaptive Behavior Scales, 2nd ed. (Sparrow, Cicchetti, & Balla, 2005), and the Social Skills Improvement System (SSIS; Gresham & Elliott, 2008). Each of the above referenced assessment instrument is described in more detail below to highlight what is missing from the field in terms of social skills assessment. 1.3.1.1. Wing and Gould’s social sub-typing system. Wing and Gould, 1979; Wing and Gould (1979; Wing, 1981) developed a social sub-typing system for children with autism that includes three subtypes based on severity of social impairment: (1) Aloof, (2) Passive, and (3) Active but Odd. The aloof subtype is characterized by a general indifference to others and represents the most severely impaired social interaction. In contrast, the passive subtype is characterized by an acceptance of approaches made by others but a failure to initiate with others spontaneously. The active but odd subtype is characterized by approaching others in a way that is one-sided and often involving idiosyncratic preoccupations. Wing and Gould’s classification system uses a descriptive approach, (that has recently been called into question, Eagle, Romanczyk, & Lenzenweger, in press), and it does not provide a direct measure of behavior. 1.3.1.2. The Vineland Adaptive Behavior Scales-II (VABS-II). The VABS-II (Sparrow et al., 2005) is a commonly used measure of adaptive functioning in the domains of communication, social skills, daily living skills, and motor skills for individuals ages birth through 90. Two survey forms, the Survey Interview Form and the Parent/Caregiver Form are used to attain information about a child’s adaptive behavior. For the Survey Interview Form a parent or caregiver completes a rating scale whereas the Interview Form is administered by a trained professional in a semi-structured interview format. A third form, the Teacher Rating Form, provides assessment of behaviors in the four domains but focuses on observable behaviors exhibited in the classroom setting. The VABS-II provides a broad estimate of social skills development based solely on third-party report and thus does not provide a direct measure of social behavior. 1.3.1.3. The Pervasive Developmental Disorders Behavior Inventory (PDDBI). One of the most recently developed measures used in assessing social behaviors in individuals with autism is the PDDBI (Cohen & Sudhalter, 2005). The PDDBI is a behavior rating scale that was designed to assist in the assessment of children who have been diagnosed with or are suspected of having a Pervasive Developmental Disorder (PDD), such as autism. Completed by an individual’s parent or teacher, the PDDBI assesses maladaptive and adaptive behaviors, the presence or absence of which are associated with PDDs. The PDDBI is different from previously used measures in that it was standardized with a sample of individuals with autism rather than with a typically developing group of individuals. The social approach behaviors subscale of the PDDBI provides a general measure of an individual’s social interaction abilities. The PDDBI has more clinical utility than previous measures in that it assesses behaviors specifically related to autism and other PDDs and that it provides information about an individual’s behavior relative to individuals with PDDs. It relies solely on third-party report and thus does not provide a direct measure of social behavior. 1.3.1.4. Social Communication Questionnaire (SCQ). The SCQ (Rutter et al., 2003) was developed as a screening tool to identify children who are at risk for having an ASD. The SCQ is designed to be used as a third-party report for children ages over 4 years old with a mental age of 2 years. The SCQ is reported to have a sensitivity and specificity of 93% and 58%, respectively, for children between the ages of 2 and 6 years of age (Allen, Silove, Williams, & Hutchins, 2007). Even though the SCQ is helpful as a screener, its use has not been extended to intervention planning or development. 1.3.1.5. Matson Evaluation of Social Skills in Youngsters (MESSY). The MESSY (Matson et al., 1983) is another measure used to assess the development of social skills in children. Intended for individuals between the ages of 4 years and 18 years, the MESSY was designed to evaluate the extent to which an individual’s problems may be related to poor social skills, evaluate the effects of intervention programs, and assess social skills for Individualized Education Programs (IEP). The MESSY consists of two forms, a self-report form and a third-party report form. Informants use 5 point Likert-type scale to rate a range of verbal and non-verbal behaviors related to social skills. While, the validity of the MESSY has been demonstrated extensively in the research literature, it was not designed specifically for use with individuals with ASD. However, more recently the scale has also been used successfully to discriminate between typically developing children and children with autism (Matson, Compton, & Sevin, 1991). 1.3.1.6. Social Skills Improvement System (SSIS). Another third-party report that has been used with children with ASD is the SSIS (Gresham & Elliott, 2008). The SSIS is appropriate to use for children between the ages of 3 and 19 years old. It is used to screen children for social difficulties and also to assist in selecting intervention goals, but is not specifically designed for individuals with ASD.

354

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

Currently, many of the tests and assessment techniques used to evaluate social skills rely solely on self and/or third-party report. They provide only general information about the development of social behaviors (i.e., are they delayed) and were not designed to be used frequently or to track treatment gains. 1.3.2. The Behavioral Assessment of Social Interactions in Young Children (BASYC) Initial development of the Social Interaction Inventory (SII; Romanczyk & Lockshin, 1999) was followed by a series of subsequent versions (e.g., Gills, Romanczyk, & Lockshin, 2005; Callahan, Gillis, & Romanczyk, 2007) and informed the development of the BASYC (Gillis, Romanczyk, & Callahan, unpublished measure) to address the limitations of the original SII and other assessments of social skills. The BASYC was developed to objectively measure and describe developmental differences of social behaviors of children with ASD in order to inform diagnostic assessment, intervention planning, and treatment monitoring. The BASYC uses direct behavioral observation in a naturalistic, semi-structured setting. A benefit of analog Behavioral Assessment is increased control over the type of behaviors selected for assessment (Kazdin, 2003). Another advantage of this approach to social assessment is that the development of behavior, cognitive, and affective skills takes place during activities and routines within a child’s natural, social environment (Wetherby & Woods, 2008). Assessment of social behaviors through an observational approach, in which the assessment procedure presents each item as an interactive, behavioral task or interaction, provides a direct measure of a behavioral deficit. Designed to provide an assessment of level of social interaction relative to children without ASD, the BASYC measures two domains of social interaction: (1) Social Initiation, and (2) Social Responsivity. From the BASYC, three scores are obtained: (1) Social Initiation (SI), (2) Social Responsivity (SR), and (3) Social Interaction Combined (SIC). The SI score provides information about social initiations made by the child to the examiner during the BASYC administration. The SR score provides information about a child’s responses to social initiations made by the examiner. The SIC is a combination of the SI and SR scores. 1.4. Purpose of current study The overarching purpose of this study was to examine the BASYC as an instrument that discriminates social behaviors between children with and without ASD. We hypothesized that individuals with ASD will score significantly lower on the BASYC. We also sought to examine the predictive validity of the BASYC. A final aim of the study was to examine how well each item on both the SI and the SR scales discriminate between the two groups. 2. Methods 2.1. Participants As per IRB approval, informed consent was obtained for all participants from their parents/legal guardians. A total of 77 participants (n = 29 participants without ASD and n = 48 with an ASD) participated in this study. The mean age of the ASD group was 6.8 years (SD = 2.9, range of 2–12 years old) and 79.2% (n = 38) were male. The mean age of the control group (i.e., participants without ASD) was 3.2 years (SD = .88, range of 2–5 years old) and X were male. The participants without ASD were specifically recruited for the purpose of this study. The participants with an ASD participated in a larger-scale study conducted by the authors. Prior to participation in this study, each participant was required to have a diagnosis of an ASD conducted by an independent psychologist or physician. In addition, a DSM-IV-TR checklist was completed for each participant to provide a confirmatory research diagnosis for participants in the current study. The diagnostic checklist presents DSM-IV-TR diagnostic criteria across the three symptom domains for Autistic Disorder, PDD-NOS and Asperger’s Disorder (APA, 2000). Experienced individuals trained and familiar with DSM-IV-TR criteria and the diagnosis of ASD completed the checklists. See Appendix B for the diagnostic checklist. 2.2. Procedure and measurement 2.2.1. Administration of the BASYC. The BASYC was conducted with each participant in this study. As described previously, the BASYC measures early social behavior across two scales, Social Responsivity and Social Initiation. During administration of the BASYC, a series of 20 items are presented to the child. The examiner presents each item verbally to the child in declarative or interrogative forms as appropriate. If the child does not respond to an item within a 10-s interval, the examiner presents the item again and records the child’s behavior. Materials presented during administration include a variety of mechanical and simple toys and common objects (e.g., paper, piece of felt, etc.) from a large, clear toy chest. Administration time is approximately 12 min. 2.2.2. Scoring of the BASYC Social behaviors (both initiations and responses) are evaluated for each of the 20 analog, semi-structured, social queries. An adult examiner presents an item (see Appendix A) and records the behavior of the child as a social response, social initiation, or no response. If the child does not make a response, the adult examiner presents the item again and records the behavior as described above. Recorded child behaviors are scored across two scale scores: Social Responsivity (SR) and Social Initiation (SI). Scale scores are coded in terms of presence or absence (‘‘yes’’ or ‘‘no’’ for occurrence).

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

355

A social response is defined as a verbal or gestural response the child makes to the examiner within a 10-s interval following the examiner’s initiation. Social responses can be either a verbal or a gestural response. Verbal responses are defined as nonimitative verbalizations that communicated intent to respond to the examiner’s initiation. Gestural responses are defined as motor behaviors such as nodding or pointing with communicative intent based on the examiner’s behavior (or social initiation). Responses that are appropriate for the presented item are scored as an occurrence (i.e., a score of 1). Social initiations (SIs) describe verbal or gestural behaviors the child makes independent of the examiner’s behaviors. An SI must occur at least 15 s before/after an examiner’s social initiation and be discernible as an initiation, not a response. Initiations demonstrate intent to communicate and interact with the examiner through play, objects, or verbalizations. Based on the definition for an SI, there are only certain items where there is an opportunity for the child to initiate. These are Items 2, 3, 4, 5, 6, 10, 14, and 15. For this study, the examiner administering the BASYC simultaneously completed the scoring. Assessment of reliability and procedural integrity was conducted using videotapes of the BASYC administrations. Video scoring was conducted independently in a quiet, laboratory setting, and research assistants wore headphones to decrease interference from background noise. Interrater reliability for BASYC sessions was .91, demonstrating a high level of agreement and consistency across coding. 2.3. Procedural integrity Twelve of the videotaped BASYC administrations were scored for procedural integrity. A checklist was used to determine whether or not the examiner administered each of the 20 items on the BASYC accurately. Percent of items administered correctly was calculated for each videotape and then averaged across the 12 videotapes. The result was an average score of 92% for correct administration of the items on the BASYC. Upon further examination of the data no specific items were shown to be more difficult (i.e., less accurate administration) to administer than others; however, most of these errors were administering Item 20 outside of the examination room or upon leaving the room making it difficult to corroborate the scored item. 3. Results 3.1. Group differences The first aim of the study was to examine whether the groups would differ on scores obtained by the BASYC. Three Mann– Whitney tests were performed to evaluate differences between each scale (SI, SR, and SIC) of the BASYC. The means and standard deviations for each group on the three scales are presented in Table 1. Each of these tests was significant (Social Response scale, U = 426.50, p = .004; Social Initiation scale, U = 449.50, p = .008, Social Interaction Composite, U = 428.50, p = .005). 3.2. Predictive validity The BASYC yielded scores that were significantly different on each scale between the two groups. Next, whether the scores obtained on the BASYC can predict group membership was examined. In order to test for this, a logistic regression was conducted using the SIC as the predictor variable. Age was included in the model because the participants in this study could not be matched on chronological age. By controlling for age, we decrease the likelihood that the groups differed simply as a result of development. A test of the full model against a constant only model was statistically significant, indicating that the SIC score reliably discriminated the two groups (chi square = 33.51, p < .001 with df = 2). Nagelkerke’s R2 of .85 indicated a moderately strong relationship between the SIC score (i.e., the prediction) and grouping. See Table 2 for estimates of the model parameters. Sensitivity (i.e., the proportion of participants identified as having ASD given their SIC score who in fact do have an ASD) and specificity (i.e., the proportion of participants identified as part of the control group based on their SIC score who in fact are part of the control group) were calculated for this sample. The SIC score on the BASYC yielded high sensitivity and specificity (.977 and .871, respectively). The positive predictive value was .916, meaning that the SIC score accurately predicted group membership for 91.6% of the ASD participants. Negative predictive value, or prediction of group member ship for participants in the control group, was found to be .964. 3.3. Item analysis Each of the items included in the BASYC was selected because of its purported relationship to characteristics of the social impairment specific to ASD. To account for the potential effects of social development on item performance, we matched the Table 1 Average scores obtained on the BASYC scales for the ASD and control groups. ASD (n = 48)

Social Response scale (SR) Social Initiation scale (SI) Social Interaction Composite (SIC)

Control (n = 29)

M (SD)

M (SD)

7.88 (4.74) 2.46 (1.60) 10.33 (6.26)

11.69 (1.97) 3.59 (.57) 15.28 (2.40)

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

356

Table 2 Summary of logistic regression analysis for variables predicting group membership.

b

Predictor variable Block 1 Age Social Interaction Composite

SE b

Wald  2

.06 .17

9.46* 10.25*

Improvement  2 33.51*

.18 .54

*

Statistical significance is denoted as follows: p < .01 (N = 77).

control and ASD participants on chronological age resulting in 13 participants for each group (N = 26). Comparison of how the two groups scored on each item of the BASYC yielded information about the different item’s ability to discriminate the groups. No specific hypotheses were generated given the heterogeneity of ASD. Thus these analyses are considered exploratory. A Discrimination Index (DI) was calculated for each item within the SR and SI scales. To calculate DI, the number of participants who scored positively on an item from the ASD group was subtracted from the number of participants who scored positively on an item from the Control group and then divided by 1 (see Tables 3 and 4). The DI represents how well each item discriminates between two groups (using values of 0–1). The generally accepted rule for appropriate/good items on a measure includes those items that have a discrimination index of .3–.7. The total discriminability index, as calculated by averaging all item DIs, was .37 and .17 for the SR and SI scales, respectively. On the SR scale, the range of item discriminability indices was .08 to .69. Item 5 received a discriminability index of .08. This value was negative because the ASD group had a slightly higher number of social responses on this item in comparison to the Control group. There were 13 (of 20) items on the Social Responsivity scale that discriminated well between the groups. The items with the lowest item discriminability indices (below .20) included Items 2, 4, 5, and 8. Items with the highest item discriminability indices (over .50) included Items 3, 9, 15, 18, 19, and 20.

Table 3 Social Responsivity—discriminability indices. SR scale

Control group

Items

Number of correct items

Percent correct

Number of correct items

ASD group Percent correct

DI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

9 10 13 11 10 10 11 4 12 12 7 11 9 10 12 12 7 11 12 13

69.2 76.9 100.0 84.6 76.9 76.9 84.6 30.8 92.3 92.3 53.8 84.6 69.2 76.9 92.3 92.3 53.8 84.6 92.3 100

6 8 6 9 11 7 5 2 5 8 2 5 4 4 5 7 4 4 4 4

46.2 61.5 46.2 69.2 84.6 53.8 38.5 15.4 38.5 61.5 15.4 38.5 30.8 30.8 38.5 53.8 30.8 30.8 30.8 30.8

.23 .15 .54 .15 .08 .23 .46 .15 .54 .31 .38 .46 .38 .46 .54 .38 .23 .54 .62 .69

Table 4 Social Initiation—discriminability indices. SI scale

Control group

Items

Number of correct items

Percent correct

Number of correct items

ASD group Percent correct

DI

2 3 4 5 6 10 14 15

9 7 5 6 5 8 9 10

69.2 53.8 38.5 46.2 38.5 61.5 69.2 76.9

5 5 4 8 5 3 6 5

38.5 38.5 30.8 61.5 38.5 23.1 46.2 38.5

.31 .15 .08 .15 .00 .38 .23 .38

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

357

4. Discussion The primary deficit in children with ASD is in the domain of social behavior. Methods used to describe social behavior typically involve third-party report. It is important that social behavior is quantified objectively and in a context that is ecologically valid. To date there are no direct assessments of social behavior of children with ASD that can be administered in a short period of time. There are several advantages to such an assessment, namely, describing and measuring social behavior deficits in order to assist with diagnosis, treatment planning, and monitoring. Observation of social behavior in a naturalistic environment aids in the validation of third-party report as well. The development of the BASYC occurred in response to the lack of an objective measure that could be administered quickly and provide information on the social behavior of children with ASD relative to those without ASD. The primary purpose of this preliminary study was to examine whether this measure of social behavior would discriminate between children with and without ASD. Each of the three scales of the BASYC (i.e., SR, SI, and SIC) was examined across groups. Scores were significantly different for the ASD and Control groups on each scale, with all scores lower for the ASD group as compared to the Control group. This finding is expected given that individuals with ASD show impairment in the area of social behavior. An interesting finding was that the control group’s scores were not highly variable on each of the scales. This is in contrast to the ASD group’s scores that showed greater variability. The limited range of scores in the control group can be interpreted as demonstrating that the social behaviors selected for inclusion in the BASYC are quite common in young children without ASD and serve as appropriate behaviors expected in preschool age children. In other words, the items in the BASYC select for social behaviors that are often impaired in children with ASD but not impaired in children without ASD. It would be expected that scores on the BASYC would increase as children with ASD social behavior improves over time (due to maturation or intervention). Future research is required to support our substantial clinical experience that the BASYC is an effective assessment measure of improvement in social behavior following intervention. We also sought to examine whether the BASYC can accurately predict group membership based on obtained scores. The BASYC has high sensitivity and specificity. Controlling for age, strong predictive validity was demonstrated. The importance of the strong predictive validity is that the BASYC can now be used as a measure of social behaviors that are consistent with the symptom presentation of ASD. The addition of each point to the individual’s score on the BASYC means that the individual is demonstrating social behaviors more similar to the control group. Although the BASYC was not originally intended as an assessment instrument to assist clinicians in the diagnostic process, the results from this study suggest that the BASYC may have application in this area. The ability of the BASYC to accurately predict group membership using the total score (SIC) is important. However, examination of whether certain items are better discriminators than others also assists in the evaluation of an assessment instrument at a more detailed level. On the SR scale, which uses all 20 items, 13 of those items were shown to have adequate discriminability; whereas this was true for only three of the eight items on the SI scale. Examination of the items on the SR scale indicated that most fell within the categories of simple conversation (i.e., What was your favorite toy? Do you like the tea?), demonstration (i.e., How does this toy work), joint play (i.e., Can I play with you? Do you want to play a game?), and responding to emotional cues (i.e., Ouch! That hurts!). The three items that discriminated well on the SI scale did not appear to follow a pattern. These items included, Item 2: What toy do you want to play with?; Item 10: Do you want to play (game 1) or (game 2)?; and Item 15: Are there any more toys you want to play with? Note that these items also discriminated well on the SR scale. It is important to take into consideration the heterogeneity of the young ASD population. Given the wide range of symptoms within the social and communication domains of impairment associated with ASD, a breadth of items are necessary. The BASYC was primarily developed for the clinical practitioner or educator to use as part of the intervention planning and monitoring process for children with ASD. Training individuals to administer the BASYC is relatively straightforward, as there are explicit written instructions with administration prompts on the BASYC scoring sheet. Despite the behavioral challenges that individuals with ASD sometimes present, procedural integrity was high at 92%. Anecdotally, individuals reported that learning the BASYC, including administration and scoring, was relatively straightforward. The administration time was relatively short (i.e., approximately 12 min) allowing for the BASYC to be administered at convenient times for the participant. This is important if the BASYC were to be included as part of educational or therapeutic interventions. It needs to be manageable for the therapist/teacher and also efficient at obtaining information about progress on social behaviors without interfering too much with intervention time. 4.1. Limitations One limitation of this study was that the groups could not be matched on age or mental age. To control for these differences in groups, a logistic regression was used. The results of the logistic regression indicated that age is a significant predictor; however, when age was controlled for the total score (SIC) was also a significant predictor variable. 4.2. Implications and future directions Though the BASYC is designed to be a measure of social interactions commonly impaired in the ASD population in order to assist with goal planning, it is our experience that it can also be used to measure progress in social interactions over time. To

358

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

date, there have been no behavioral measures that can be conducted quickly and easily to assess changes in social interactions of children with ASD. Recently, we have begun to examine this question by examining whether a child’s score on the BASYC reflects progress on specific social skills interventions. Additional research is needed to not only replicate the current findings, but also to explore other assessment and monitoring uses for the BASYC. There has been a lack of instruments for the direct assessments of social behavior of children with ASD. In general, direct assessments are considered less biased than third-party reports and are also useful for assessing behavior in a controlled yet natural, ecologically valid context. The BASYC is a time cost-effective measure that assists in describing the nature of social interactions of children with ASD, especially those social interactions that are commonly impaired in this population. The current study demonstrated that children with ASD score differently than children without ASD and that the BASYC accurately predicts group membership. The BASYC appears to be a good measure for determining types of social interactions, specifically social initiations and responses, that are impaired in the ASD population and differentiates these social interactions from those of children without ASD. Appendix A Diagnostic checklist.

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

359

Appendix B Item#

BASYC item

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Would you like to play with me? What toy do you want to play with? Can I play with (toy)? Do you want to play with this toy? (Pick out an interesting toy (one that moves/lights up)) Do you want to play with this toy? (Pick out a ‘‘boring’’ toy (e.g., tissue, piece of paper)) Do you want some tea? (Pick out the teapot and cups and prepare two cups of pretend tea) Do you like the tea? I’m hungry. Rub stomach Do you want to play a game? Do you want to play (game 1) or (game 2)? (Give the child a choice of two games such as catch and Simon says) Ouch, that hurts! (Pretend to hurt finger) How does this toy work? Take a toy that requires manipulation (see ‘n say/toy truck with pull string) and show it to the child. Wow. I’m hot in here! (Wave hand near face) Look at the (toy far away)! Would you like to see it? (Place a toy out of reach of the child and point to it) Are there any more toys you want to play with? It’s time to clean up. Would you help me please? Let’s put all of the toys into the bin. I had fun playing with you. What was your favorite toy? Would you like to play with me again? Thanks again for playing with me! Bye.

References Allen, C. W., Silove, N., Williams, K., & Hutchins, P. (2007). Validity of the Social Communication Questionnaire in assessing risk of autism in preschool children with developmental problems. Journal of Autism and Developmental Disorders, 37, 1272–1278 doi:10.1007/s10803-006r-r0279-7. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washington, DC: American Psychiatric Association. (pp. 70–75). Barry, T. D., Klinger, L. G., Lee, J. M., Palardy, N., Gilmore, T., & Bodin, S. D. (2003). Examining the effectiveness of an outpatient clinic-based social skills group for high-functioning children with autism. Journal of Autism and Developmental Disorders, 33, 685–701 doi:10.1023/B:JADD.0000006004.86556.e0. Bellini, S., Peters, J. K., Bennett, L., & Hopf, A. (2007). A meta-analysis of school-based social skills interventions for children with autism spectrum disorders. Remedial and Special Education, 28, 153–162 doi:10.1177/07419325070280030401. Bellini, S., & Peters, J. K. (2008). Social skills training for youth with autism spectrum disorders. Child and Adolescent Psychiatric Clinics of North America, 18, 857–873 doi:10.1016/j.chc.2008.06.008. Callahan, E. H., Gillis, J. M., & Romanczyk, R. G. (2007). Norming and use of the Social Interaction Inventory—Revised for children with autism spectrum disorders. Presented at the biennial meeting of the Society for Research in Child Development. Cohen, I. L., & Sudhalter, V. (2005). The PDD behavior inventory. Lutz, FL: Psychological Assessment Resources Inc. Constantino, J. N., & Todd, R. D. (2003). Autistic traits in the general population: A twin study. Archives of General Psychiatry, 60, 524–530 doi:10.1001/ archpsyc.60.5.524. Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social information-processing mechanisms in children’s social adjustment. Psychological Bulletin, 115, 74–101 doi:10.1037/0033-2909.115.1.74. Dawson, G., Meltzoff, A. N., Osterling, J., Rinaldi, J., & Brown, E. (1998). Children with autism fail to orient to naturally occurring social stimuli. Journal of Autism and Developmental Disorders, 28, 479–485 doi:10.1023/A:1026043926488. Eagle, R., Romanczyk, R. G., & Lenzenweger, M. F. (in press). Classification of children with autism spectrum disorders: A finite fixture modeling approach to heterogeneity. Research in Autism Spectrum Disorders, in press, doi:10.1016/j.rasd.2010.02.001. Elliott, S. N., Gresham, F. M., Frank, J. L., & Beddow, P. A. (2008). Features of assessments that link results to treatment plans. Assessment for Effective Interventions, 34, 15–24 doi:10.1177/1534508408314111. Franco, F., & Butterworth, G. (1996). Pointing and social awareness: Declaring and requesting in the second year. Journal of Child Language, 23, 307–336 doi:10.1017/S0305000900008813. Gillis, J. M., Romanczyk, R.G., & Callahan, E.H. (unpublished measure). The Behavioral Assessment of Social Behavior in Young Children. Binghamton, NY: Institute for Child Development. Gills, J. M., Romanczyk, R. G., & Lockshin, S. L. (2005). The social interaction inventory—Revised. Binghamton, NY: Institute for Child Development. (unpublished measure). Gresham, F. M., & Elliott, S. N. (2008). The social skills improvement system. San Antonio, TX: PsyCorp. Hartup, W. W., & Moore, S. G. (1990). Early peer relations: Developmental significance and prognostic implications. Early Childhood Research Quarterly, 5, 1–17 doi:10.1016/0885-2006(90)90002-I. Hauck, M., Fein, D., Waterhouse, L., & Feinstein, C. (1995). Social initiations by autistic children to adults and other children. Journal of Autism and Developmental Disorders, 25, 579–595 doi:10.1007/BF02178189. Hay, D. F., & Murray, P. (1982). Giving and requesting: Social facilitation of infants’ offers to adults. Infant Behavior and Development, 5, 301–310 doi:10.1016/ S0163-6383(82)80039-8. Hughes, J. N., & Hall, R. J. (1987). Proposed model for the assessment of children’s social competence. Professional School Psychology, 2, 247–260 doi:10.1037/ h0090544. Jackson, C. T., Fein, D., Wolf, J., Jones, G., Hauck, M., Waterhouse, L., et al. (2003). Responses and sustained interactions in children with mental retardation and autism. Journal of Autism and Developmental Disorders, 33, 115–121 doi:10.1023/A:1022927124025. Kanner, L. (1943). Autistic disturbances of affective contact. Nervous Child, 2, 217–250. Kazdin, A. E. (2003). Clinical applications of analogue behavioral observation: Dimensions of psychometric evaluation. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (3rd ed., pp. 235–264). Washington, DC: American Psychological Association. Koegel, R. L., Koegel, L. K., & McNerney, E. K. (2001). Pivotal areas for intervention for autism. Journal of Clinical Child and Adolescent Psychology, 30, 19–32 doi:10.1207/S15374424JCCP3001_4. Lord, C. (1995). Follow-up of two-year-olds referred for possible autism. Journal of Child Psychology and Psychiatry, 36, 1365–1382 doi:10.1111/j. 1469-7610.1995.tb01669.x. Matson, J. L., Rotatori, A. F., & Helsel, W. J. (1983). Development of a rating scale to measure social skills in children. The Matson Evaluation of Social Skills with Youngsters (MESSY). Behaviour Research and Therapy, 21, 335–340 doi:10.1016/0005-7967(83)90001-3.

360

J.M. Gillis et al. / Research in Autism Spectrum Disorders 5 (2011) 351–360

Matson, J. L., Compton, L. S., & Sevin, J. A. (1991). Comparison and item analysis of the MESSY for autistic and normal children. Research in Developmental Disabilities, 12, 361–369 doi:10.1016/0891-4222(91)90032-N. Matson, J. L., & Wilkins, J. (2007). A critical review of assessment targets and methods for social skills excesses and deficits for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 1, 28–37 doi:10.1016/j.rasd.2006.07.003. Matson, J. L., & Wilkins, J. (2009). Psychometric testing methods for children’s social skills. Research in Developmental Disabilities, 30, 249–274 doi:10.1016/ j.ridd.2008.04.002. McClellan, D. E., & Katz, L. G. (2001). Assessing young children’s social competence. Champaign, Il. ERIC Clearinghouse on Elementary and Early Childhood Education (ED450953). McEvoy, R. E., Rogers, S. J., & Pennington, B. F. (1993). Executive function and social communication deficits in young autistic children. Journal of Child Psychology and Psychiatry, 34, 563–578 doi:10.1111/j.1469-7610.1993.tb01036.x. Mundy, P., Sigman, M., & Kasari, C. (1994). Joint attention, developmental level, and symptom presentation in autism. Developmental Psychopathology, 6, 389–401 doi:10.1017/S0954579400006003. Romanczyk, R. G., & Lockshin, S. (1999). The Social Interaction Inventory—An extension of the IGS Curriculum. Binghamton, NY: Institute for Child Development. (unpublished measure). Rutter, M., Bailey, A., & Lord, C. (2003). SCQ: The Social Communication Questionnaire—Manual. Los Angeles, CA: Western Psychological Services. Sigman, M., Kasari, C., Kwon, J., & Yirmiya, N. (1992). Responses to the negative emotions of others by autistic, mentally retarded, and normal children. Child Development, 63, 796–807 doi:10.2307/1131234. Sigman, M., & Ruskin, E. (1999). Continuity and change in the social competence of children with autism, Down Syndrome, and developmental delays. Monographs of the Society for Research in Child Development, 64, 114 doi:10.1111/1540-5834.00010. Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). The Vineland Adaptive Behavior Scales (2nd ed.). Circle Pines, MN: American Guidance Service Inc. Stone, W. L., Coonrod, E., & Ousley, O. (2000). Brief report: Screening Tool for Autism in Two-Year Olds (STAT): Development and preliminary data. Journal of Autism and Developmental Disorders, 30, 607–612 doi:10.1023/A:1005647629002. Strain, P. S., & Hoyson, M. (2000). The need for longitudinal, intensive social skill intervention: LEAP follow-up outcomes for children with autism. Topics in Early Childhood Special Education, 20, 116–125 doi:10.1177/027112140002000207. Townsend, J., Harris, N. S., & Courchesne, E. (1996). Visual attention abnormalities in autism: Delayed orienting to location. Journal of the International Neuropsychological Society, 26, 541–550 doi:10.1017/S1355617700001715. Travis, L. L., & Sigman, M. (1998). Social deficits and interpersonal relationships in autism. Mental Retardation and Developmental Disabilities Research Reviews, 4, 65–72 doi:10.1002/(SICI)1098-2779 (1998) 4:2<65::AID-MRDD2>3.0.CO;2-W. Wetherby, A. M., & Woods, J. J. (2008). Developmental approaches to treatment. In K. Chawarska, A. Klin, & F. R. Volkmar (Eds.), Autism spectrum disorders in infants and toddlers: Diagnosis, assessment, and treatment (pp. 170–206). New York, NY: Guilford Press. White, S. W., Koenig, K., & Scahill, L. (2007). Social skills development in children with autism spectrum disorders: A review of the intervention research. Journal of Autism and Developmental Disorders, 37, 1858–1868 doi:10.1007/s10803-006-0320-x. Wing, L. (1981). Language, social and cognitive impairments in autism and severe mental retardation. Journal of Autism and Developmental Disorders, 11, 31–44 doi:10.1007/BF01531339. Wing, L., & Gould, J. (1979). Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification. Journal of Autism and Developmental Disorders, 9, 11–29 doi:10.1007/BF01531288.