How special is social looking in ASD

O. Braddick, J. Atkinson and G. Innocenti (Eds.) Progress in Brain Research, Vol. 189 ISSN: 0079-6123 Copyright Ó 2011 Elsevier B.V. All rights reserved.

CHAPTER 12

How special is social looking in ASD: A review Terje Falck-Ytter{,{,* and Claes von Hofsten{,} {

Center of Neurodevelopmental Disorders at Karolinska Institutet (KIND), Astrid Lindgren Children’s Hospital, Stockholm, Sweden { Department of Psychology, Uppsala University, Uppsala, Sweden } Department of Psychology, Oslo University, Oslo, Norway

Abstract: This review is primarily concerned with the view that individuals with autism spectrum disorder (ASD) look less at the eyes and more at the mouth compared to typically developing (TD) individuals. Such performance in ASD could reflect that the eyes are not meaningful or that they are perceived as threatening, two ideas that may seem intuitively appealing. However, our review shows that despite the fact that the excess mouth/diminished eye gaze hypothesis fits with clinical common sense and initial data from adults, it does not—as a generalization across ages and contexts—fit with the emerging pattern of eye-tracking data. In adolescents and adults, there is only partial support for the excess mouth/diminished eye gaze hypothesis, and regarding children, most studies do not support this hypothesis. In particular, independent studies have found longer looking durations on the mouth in TD children than in children with ASD, and no difference for the eye area. We describe recent evidence that mouth fixations are functional responses related to (early) stages of normative language development. We conclude that although individuals with ASD often give less preferential attention to social objects and events (faces, people, and social actions) than TD individuals, the excess mouth/ diminished eye gaze hypothesis of ASD is not generally supported. Therefore, this hypothesis needs to be reevaluated, as do related theories of social perception in ASD. Keywords: autism; ASD; face scanning; development; language.

actions, conveys crucial information about the other person. Their movements and especially their facial gestures provide visual information about their intentions and emotions. Gaze direction provides information about where in the surrounding attention is directed and thus about what the other person is interested in. What the

Introduction Looking is extremely important in social interaction. Looking at other people, their faces and *Corresponding author. Tel.: þ46 (0)8 517 79635; Fax: þ46 (0)8 517 77349 E-mail: [email protected] DOI: 10.1016/B978-0-444-53884-0.00026-9

209

210

other person is doing provides information about his or her goals and intentions. In addition, looking is a part of the social interaction itself. Looking at other people during social encounters helps establishing and maintaining communication and regulates the flow of interaction. Autism spectrum disorders (ASD) are pervasive developmental disorders that affect around 0.6% of the population (Fombonne, 2005) and are defined by sociocommunicative impairments (American Psychiatric Association, 1994), impairments that have been linked to deficits in face processing (Hobson et al., 1988). This link has support from imaging studies, showing that unlike typical controls, individuals with ASD recruit object-specific areas when looking at faces, and the fusiform face area is activated less than controls (e.g., Schultz et al., 2000). Face processing impairments specifically, and social impairments more generally, have previously been linked to where in the face individuals with ASDs look when they observe others’ faces (Dalton et al., 2005; Klin et al., 2002b). Klin et al. studied the eye movements of verbally able adolescents and young adults with ASDs as they observed clips from the 1967 film version of Edward Albee’s “Who’s Afraid of Virginia Woolf?” The authors observed a striking difference between ASDs and control participants. The ASDs group looked much less in the eye area (there was no overlap in looking performance across the two groups) and more in the mouth area. This result has received wide recognition, both in the scientific literature and elsewhere, perhaps, because the finding seemed to confirm the clinical observation that individuals with ASDs look less in other people’s eyes. Now, after almost a decennium, considerable new data show that the face scanning performance of ASD is a more complex issue than previously assumed. In this chapter, we describe existing hypotheses relating to face scanning alterations in ASDs, followed by a summary of the available empirical data. We will then

evaluate alternative views of the nature of social looking alterations in ASD more broadly. Hypotheses related to face scanning alterations in ASD One influential hypothesis holds that seeing faces, and eyes in particular, leads to an increased (negatively valenced) emotional response in ASD compared to TD controls (Dalton et al., 2005). From the perspective, no particular preference for the mouth is expected, but excess mouth fixations would be compatible with this hypothesis, simply as biproducts of avoiding the eye area. Support for this view was found in a study of adult participants with ASD (Dalton, 2005). This group looked less in the eye area than controls, and the amount of looking time in that area was positively related to activity in the limbic system. According to another view, excess mouth/diminished eye gaze is driven by a failure to use information from the eye area, in combination with an ability to use visual information from the mouth for speechrelated processing (Klin et al., 2002b). A stronger version of this latter hypothesis holds that excess mouth fixations in ASD can be explained by lowlevel sensory contingencies, namely the audiovisual synchrony created by a speaking mouth (Klin et al., 2009). Finally, we have suggested that the balance between the level of socioemotional skills (relates to eye fixations) and communication skills (relates to mouth fixations) of young children with ASD may explain individual looking patterns (Falck-Ytter et al., 2010), but that on a group level there may be no clear differences between ASD and controls. About this review We searched relevant databases (ISI Web of Science, PubMed) for articles reporting eye-tracking research into viewing patterns in face observation in ASD. When reviewing these articles, we were interested in the following aspects (which we

211

judged to highly relevant in order to understand the results): type of task during eye tracking (e.g., recognition, passive viewing), type of stimulus (e.g., photographs, video), sample size, diagnosis and age of participants, reference groups (and type of matching, if any), and type of measure (e.g., absolute or relative measures). However, to reduce the complexity of the review, we focus on the factors that we judged to be most informative for understanding the overall pattern of results. The devices used for measurement of eye movements have an impact on the results. Eye movements used to be very difficult to measure in children with ASD because the devices used required the subjects to sit still and not move their head. The devices used today allow the subjects to move the head freely and nothing is attached to them. This has resulted in low attrition rates in studies of eye movements and more studies being completed. This review is restricted to studies using modern eye-tracking techniques. The stimuli presented to subjects in the reviewed studies were very heterogeneous. Some studies have used videos of social events, while others, in fact the majority, have used pictures. Only one study has used live stimuli (Young et al., 2009). It is clear that the fixation patterns obtained are affected by the choice of stimuli. Another variable of great importance is age. Most investigations have studied highfunctioning adolescents or adults, and the results of these studies have received much attention. Less attention has been given to the important new data showing that it is very problematic to generalize these findings to children with ASD, or to verbally low-functioning individuals with ASD. We begin the review by considering the influence of age on looking performance. The age of the subjects Because visual face scanning patterns are likely to change with age, we reasoned that it is natural to discuss the studies of children separately from the adult studies. Klin et al. (2002a,b) as well as many other studies have investigated adolescents and/or

adults with ASD. The findings from these studies have had a great impact, leading many to believe that the excess mouth/diminished eye gaze pattern described by Klin et al. (2002a,b) generalizes to all individuals with ASD, including children and infants. Regarding adults/adolescents with ASD, abnormal performance related to eye or mouth looking during face observation has been documented in multiple studies (Corden et al., 2008; Klin et al., 2002b; Nakano et al., 2010; Speer et al., 2007). However, there are many examples of studies failing to find group differences in adults/adolescents (Fletcher-Watson et al., 2009; Rutherford and Towns, 2008; Spezio et al., 2007).1 Dalton et al. (2005), linking gaze performance to amygdala dysfunction, only partially replicated the original finding of Klin et al. in terms of looking data. In addition, they reported absolute looking times in the face areas of interest, leaving it possible that the somewhat lower looking times in the eye area would not remain if controlling for looking time in the face. Another often cited study by Pelphrey et al. (2002) found a similar pattern as Klin et al. (2002a,b). However, this study included only five subjects, which makes it difficult to generalize across all individuals with ASD. A careful look at studies of face scanning in children (defined as 12 years or younger) with ASD reveals that only one study has replicated the excess mouth/diminished eye gaze pattern found in older individuals (Jones et al., 2008). Other studies have either failed to find the excess mouth/diminished eye gaze in children with ASD (Dapretto et al., 2006; Falck-Ytter et al., 2010; van der Geest et al., 2002) or found evidence of longer looking time in the mouth area for TD children than for children with ASD combined with no difference for the eye area (Chawarska and Shic, 2009; de Wit et al., 2008; Nakano et al., 2010). Initial data from infants with an older brother or sister with ASD seemed to confirm the original

1

The Spezio et al. (2007) study found differences for filtered (Bubbles) faces but not for unfiltered faces.

212

pattern described in adults/adolescents with ASD by Klin et al. (2002b). Merin et al. (2007) studied 6-month-old children with an older sibling with ASD. They used a modified still-face paradigm where the interaction was performed via a closedcircuit TV–video system. In this paradigm, the mother interacts with the infant, then freezes and displays a neutral, expression-less face, and then resumes interaction. Eye-tracking data on infant visual fixation patterns were recorded during the three episodes of the experiment. Using a hierarchical cluster analysis, Merin et al. identified a subgroup of infants demonstrating diminished gaze to the mother’s eyes relative to her mouth during the still-face episode. In all, 10 of the 11 infants in this subgroup had an older sibling with ASD. However, when Young et al. (2009) made a longitudinal follow-up of this study where they tested the predictive utility of gaze behavior and affective behaviors at 6 months relative to the diagnostic outcome data obtained longitudinally over the following 18 months, it turned out that none of the children previously identified as showing lower rates of eye contact had any signs of ASD.2 In contrast, all infants who were diagnosed with autism demonstrated consistent gaze to the eye region and typical affective responses at 6 months. Individual differences in face scanning and affective responsivity during the live interaction were not related to any continuous measures of symptom frequency or symptom severity, a result that mimics many other baby sibling studies (e.g., Elsabbagh and Johnson, 2010). Language development, on the other hand, was significantly related to fixation of the mother’s mouth during live interaction. Such performance predicted higher levels of expressive language at outcome and greater rates of growth. These findings suggest that although gaze behavior at 6 months may not provide early markers

for ASD as initially assumed, gaze to the mouth, in particular, may be useful in predicting individual differences in language development (see next section). It also underlines that it is problematic to use a result from adult studies (reduced eye fixation, increased mouth fixation in ASD; Klin et al., 2002b) as the basis for an infant study. Mouth looking as a normative process linked to language development The sibling study by Young et al. (2009) fits well with other data suggesting that mouth fixations in young TD children may be related to language development. Nakano et al. (2010) recently documented that, in TD children, there was a strong tendency to look at the mouth of a speaking person (a child). This tendency was not observed in children with ASD, adults with ASD, or in TD adults. The finding contradicts a recent hypothesis stating that children with ASD orient to spatial locations with correlated changes in visual motion and audio amplitude (Klin et al., 2009). Indeed, if this hypothesis was correct, one would expect a clear preference for the mouth during speech in ASD (assuming that the mouth contained most audiovisual synchrony in these stimuli). In TD individuals, there was a strong decline in mouth preference from childhood to adulthood (and vice versa for the eyes).3 In ASD, this change was not evident. That is, the relative distribution of eye versus mouth fixations remained relatively constant across the two age groups in ASD. When summarizing studies of mouth fixations in TD individuals, an interesting developmental picture emerges. In infancy, mouth fixations are much more dominant than typically assumed (Hunnius and Geuze, 2004) and predict later vocabulary development (Young et al., 2009). Mouth fixations are more frequent in TD children than in children and adults with ASD and TD adults during periods of observed speech (Nakano et al., 2010). The fact

2

Moreover, with more infants included in the study, the authors were not able to replicate the original group level (risk status) difference in infancy.

3

An interpretation based on cross-sectional data.

213

that this difference was most clear during periods of speech and was not observed in ASD suggests a functional behavior, rather than a behavior related to low-level visual saliency (e.g., mouth movements per se). In TD adults, more eye fixations are found, and no increase in mouth looking during speech periods, which probably reflect that TD adults are able to perceive speech accurately via the auditory channel in isolation, or at least without focal visual attention on the mouth (see below). When speech is masked by noise, TD adults have been shown to increase their looking time at the mouth (Vatikiotis-Bateson et al., 1998). The functional role of mouth fixations in ASD during speech perception is less clear. Judging from Nakano et al., the normative age-related decrease in mouth fixations found in TD individuals is not observed in ASD. However, individual differences related to communicative/ language skills (both in absolute terms and in relation to emotional skills) have been shown to predict mouth looking in two ASD samples (Falck-Ytter et al., 2010; Norbury et al., 2009; Fig. 1). These pieces of evidence suggest that individual differences in communication/language skills in ASD can explain individual differences in viewing data, but perhaps, group-level viewing patterns remain more stable from childhood to adulthood in individuals with ASD than in TD individuals (Nakano et al., 2010). As already noted, the change in TD individuals from childhood to adulthood could be related to increasingly effective auditory (unimodal) processing of speech. However, there is clear evidence that TD adults are highly influenced by mouth movements during speech perception (e.g., demonstrated by McGurk effects). Thus, it is likely that TD adults still use mouth information during speech processing despite the fact that they do not look at the mouth as frequently/long as children (Nakano et al., 2010). The difference between TD children to TD adults may instead be linked to increasingly skilled configural processing of faces, allowing the adult observer to give focal attention to the eyes, while still effectively processing the mouth movements of the observed face. Research suggests that adult-level configural

processing of faces is a late development (Joseph et al., 2006; Passarotti et al., 2007; Taylor et al., 2001). There is evidence that adults with ASD remain more reliant on featural processing during face observation (Schultz et al., 2000). Also, eye-tracking data from our labs indicate that children with ASD process the sociocommunicative visual information from the mouth in a featural manner (Falck-Ytter, 2008; Falck-Ytter et al., 2010; see Fig. 1). The exact link between face scanning patterns and configural processing skills remains unclear, but it is interesting to note that the developmental eye-tracking data reported in Nakano et al. (2010), with clear differences between ASD and TD individuals, parallel the (discordant) development of configural processing in TD individuals versus individuals with ASD. The nature of the stimuli Some of the earliest attempts to use eye tracking to study patterns of gaze performance in social settings were those by Klin et al. (2002a,b). The subjects were high-functioning adolescents and young adults with ASD and the controls were matched according to age, sex, and verbal IQ. As noted previously, using clips from the movie “Who’s Afraid of Virginia Woolf?” they found a clear difference in terms of looking time at the mouth (individuals with ASD > TD individuals) and eye (individuals with ASD < TD individuals). Increased fixation on mouths was positively associated with adaptation skills and negatively associated with autistic social impairments.4

4

Klin et al. (2002a) points out that behind the overall looking time statistics, there are perhaps more interesting dynamic effects hidden. Several examples were given where, in very tense situations in the film, it seemed the subjects with ASD did not get the point (because they looked at apparently irrelevant of insignificant parts of the scene). Klin et al. (2003) stressed the fact that social cognition is very dynamic, closely related to social actions, and only meaningful with reference to social actions. A social gesture gets its meaning from the sequence of events within which it exists. For instance, some subjects with ASD tended to miss the meaning of gesture like pointing by failing to react properly to them, but when asked, they had no difficulty in defining the meaning of the gesture.

214 TYP

(b)

Ratio of looking time

Ratio of looking time

(a)

1 0.8 0.6 0.4 0.2 0 Eyes

ASD + Eyes Mouth 1 0.8 0.6 0.4 0.2

+

0 −4 −2 0 2 4 6 Difference score (SI-CI)

Mouth

(c)

+ + + + + + +++ + ++ + +

(d)

Looking time

Fig. 1. Based on genetic and behavioral data indicating that the triad of symptoms are fractionable in ASD (Ronald et al., 2006), we (Falck-Ytter et al., 2010) hypothesized that children with ASD who are (relatively) better at emotional behaviors (such as showing facial expressions, wanting to share joy, social smiling, etc.) than they are at relatively nonemotional communicative behaviours (such as pointing and imitating actions) would be looking more at the eyes, while those with the opposite profile would look more at the mouth. This hypothesis was supported in a group of preschoolers with ASD. This figure illustrates the connection between individual differences in sociocommunicative skills and looking behavior during face observation in preschool children with ASD. (a) For typically developing children (TYP), box plots show lower, median, and higher quartiles (whiskers indicate whole range). (b) For ASD, the x-axis represents the difference score between social and communicative impairments (Social Impairments [SI] minus Communication Impairment [nonverbal; CI] of the Autism Diagnostic Interview—Revised]. Interestingly, the positive correlation between the difference score and looking time in the mouth area was also found for inverted faces. This is in line with the idea that there is an enhanced reliance on featural information in ASD, as inversion mainly disrupts configural information. In (a) and (b), the y-axis represents the looking time ratio in eyes and mouth relative to the whole face. (c) Gaze plot from a child with more severe (nonverbal) communicative than social impairments. (d) Gaze plot from a child with less severe (nonverbal) communicative than social impairments. In (c) and (d), individual gaze data from all stimuli are superimposed on one static representation. Reproduced with permission from Falck-Ytter et al. (2010).

Speer et al. (2007)5 replicated and extended the studies by Klin et al. (2002a,b, 2003) by including The authors entitled their article “Face processing in children with autism: Effects of stimulus contents and type” although the age range was from 9 to 18 years (mean 13.6 years). In this chapter, this study is not considered a study of children, which we defined as less than 12 years. 5

more conditions (social and isolated dynamic scenes and social and isolated static scenes). Again, the stimuli were taken from the film “Who’s Afraid of Virginia Woolf?” The socialdynamic stimuli showed several people interacting with each other, while isolated dynamic scenes only include one person. In the same way, social static scenes depicted more than one

215

person, while isolated static scenes only depicted one subject. Participants with ASD differed from their TD peers only for social-dynamic stimuli; fixation durations were decreased for eye regions and increased for body regions. Further, these fixation durations predicted scores on a measure of social responsiveness. Norbury et al. (2009) criticized Klin’s study from the standpoint that the video clips used (from “Who’s Afraid of Virginia Woolf?”) did not reflect typical events in everyday life. Therefore, they produced new video clips of peers interacting in familiar situations. A group of 14year-old adolescents with ASD was compared to a group of TD adolescents of the same age. The authors divided the ASD group into one group without language impairments and one group with language impairments. It was found that only the latter subgroup looked less at the eyes of the models in the videos and was slower to fixate the eyes compared to the TD children. Further, adaptive communication skills correlated positively with mouth looking in ASD. As already described, Nakano et al. (2010) used video stimuli and found that adults with typical development looked more at other people’s eyes than adults and children with ASD as well as children with typical development (who were the most extreme mouth lookers). In addition to the above studies on how highfunctioning adolescents/adults with ASD look at videos of social scenes, there are a number of studies that have used pictures. Rutherford and Towns (2008) showed pictures of emotional faces to high-functioning adults with ASD and a control group of typical adults. They found that both groups looked longer at the eyes than at the mouth. No main differences were found between the groups (but it was found that the subjects with ASD looked less into the eyes of the faces with more complex emotions). Corden et al. (2008) studied facial affect recognition (also using pictures) in a group of adults with Asperger’s syndrome (AS) and a matched control group. It was found that the AS subjects were impaired in their

recognition of fearful and sad expressions and spent significantly less time fixating the eye region of all faces. Spezio et al. (2007) studied eight highfunctioning male adult subjects with ASD and a group of age-matched controls. The participants with ASD were as good as the other group at judging emotions in still faces, but did so using less eye information and more mouth information as shown by the Bubbles technique. This method varies the amount/type of information that is transmitted from a given region of a face on each trial, allowing the investigator to determine the information present in a region when a participant looked at that region, compared to when the participant looked at the other regions. Observing such filtered faces, the ASD group looked less at the eyes than controls. However, when observing unfiltered faces, there was no group difference in looking time for the two key face areas. Fletcher-Watson et al. (2009) showed pictures of social and nonsocial scenes to adolescents and adults with ASD and to typical subjects. They found no difference between these two groups regarding fixation of eyes, but in social scenes where the depicted person looked in a certain direction, the subjects with ASD had less tendency to follow gaze than for the typical subjects. Kliemann et al. (2010) used facial photographs and cuing toward either the mouth or the eyes, and found that adults with ASD were more likely than controls to gaze away from the eye area (if initially cued to look there). However, the two groups were equally likely to gaze toward the eye area (if initially cued to look at the mouth). Freeth et al. (2010) studied adolescents with ASD during observation of photographs of faces within complex scenes and found no difference in terms of total looking time within the upper and lower part of the face compared to controls. This pattern was found in two experiments comparing long and short stimulus exposures. In summary, the reduced tendency to fixate the eyes of depicted subjects in photographs is not uniformly present in adolescents/adults with

216

ASD in contrast to the video studies that appear somewhat more homogenous in their results (Klin et al., 2002a,b; Nakano et al., 2010; Speer et al., 2007), but see Norbury et al. (2009) for an indication that these results from video studies may be restricted to verbally high-functioning individuals with ASD. Although fewer studies are available, the photograph versus movie distinction does not seem to be of high relevance during childhood (Falck-Ytter et al., 2010; Jones et al., 2008; Nakano et al., 2010).

Attending to social versus nonsocial objects and events If there is no clear abnormal preference for mouth over eyes, what does the abnormal social looking in ASD look like? In the original studies by Klin et al. (2002a,b, 2003), it is pointed out that the subjects with ASD often fixate nonsocial parts of a scene, like a doorknob, instead of the dynamic social interactions depicted. This finding has been documented in a number of studies. Riby and Hancock (2008), for instance, found that adolescents with ASD fixated things in the background more often than the faces in the

photographs. Another example of this tendency can be found in one of the video clips studied by Nakano et al. (2010). It showed a young girl being introduced on TV and the name then appeared in written form simultaneously at the bottom of the video. Children with ASD were much more attracted by these letters than typical subjects. This tendency to look at atypical aspects of a complex scene is also demonstrated by the fixation pattern of children with ASD in the conversation video used by von Hofsten et al. (2009). A shadow cast by one of the models proved to be very attractive to the ASD children (see Fig. 2). We were, in fact, even unaware of this shadow until we examined what had caught the attention of the ASD subjects. At least in adults with ASD, preference for nonsocial aspects of the scene cannot be explained by “low level” image feature saliency (Fletcher-Watson et al. 2009). Recently, Pierce et al. (2011) showed that in a preferential looking paradigm (social events vs. physical events presented on each side of a screen), there was a very clear difference between toddlers with ASD and TD and developmentally delayed toddlers (without ASD). The ASD group looked more at the nonsocial side of the screen than the other groups. Notably, not all children

Fig. 2. The average gaze pattern of a group of ten 3- to 6-year-old children with ASD (left) and a group of 12 TD 3-year-old children (right). The intensity of fixations goes from yellow–green–blue–red. Note the more spread out fixations in the lower face of the ASD children and the substantial fixation on the shadows casted in between the models.

217

with ASD showed this pattern, but those who did tended to produce fewer saccades when they were viewing their preferred side (nonsocial) of the screen than all other groups. During periods when children in this ASD subgroup occasionally looked at their nonpreferred social side of the screen, they produced more saccadic eye movements than the other (sub-) groups. Importantly, it is not yet known whether saccade frequency is truly independent of looking time data (it could be a filter issue) Klin et al. (2009) also studied a group of toddlers with ASD and similar to Pierce et al., they compared their gaze performance with gaze data from two control groups (age-matched controls and mental age-matched controls). In agreement with Pierce et al. (2011), nonsocial events were given preferential attention by the ASD group (they tended to look at locations on the screen containing much audiovisual synchrony). The other groups ignored this type of information. The control groups looked instead at biological motion (point light displays of an upright person). While these two studies contribute importantly to our understanding of early developmental trajectory of ASD as a group, they leave many questions unanswered. One question concerns the reason for the observed lack of significant relationship between viewing performance and diagnostic scales such as the Autism Observation Diagnostic Schedule (ADOS) or the ADI-R, despite the striking group differences. Another question concerns the ability of these measures to classify ASD versus non-ASD on an individual level (an important question given the young age of the subjects). The Pierce et al. study showed that while extreme preference for the nonsocial events was strongly predictive of a concurrent ASD diagnosis, most children with ASD actually preferred the social side and where undistinguishable from the other two groups. The Klin et al. study made no claims that their paradigm could be used to classify group membership in a clinically meaningful way.

Predicting social events As the previous section shows, individuals with ASD show less preference for social (compared to nonsocial) events than do children with typical development. As noted by many investigators, this could originate in impaired “social networks” in the brain or indirectly in other cognitive processes like event prediction and/or more basic perceptual process such as motion perception (Dinstein et al., 2008). Von Hofsten et al. (2009) set out to investigate these different possibilities in a study of predictive gaze shifts in a social versus nonsocial situation. If the problems experienced by children with ASD are caused by corrupted motion perception, then these problems should also turn up in other nonsocial tasks that rely on functioning event perception, but if they are determined by a dysfunctioning social network in the brain, they should only affect predictions related to social tasks. Von Hofsten et al. (2009) measured predictive looking in 3- to 6-year-old children with ASD and in two reference groups with TD children, one with 3-year olds and another with 1-year olds. Two prediction tasks involving physical events were used. In the first, one children’s predictive smooth pursuit of a sinusoidally moving object was measured and in the second their predictive gaze shifts in anticipation of the reappearance of an occluded object. The third task involved social predictions. The participants viewed a conversation between two participants turned toward each other. The tendency to shift gaze to the next speaker before she started to speak was measured. The results showed that the children with ASD had no problems with predicting the physical events. In fact, they were equally good or better than the TD children. Regarding the social task, three results stood out. First, the children with ASD looked much less at the faces of the speakers than the TD children (see also Riby and Hancock, 2008). In fact, 3-year-old TD children looked on the average three times as much

218

at the faces as the children with ASD. Second, when TD children as well as children with ASD looked at the faces, they almost always looked at the mouth (over 90% of the time), a finding that mimics the recent findings by Nakano et al. (2010). Finally, the TD children predicted twice as many turns as the children with ASD (see Fig. 2). Thus, this study indicates that children with ASD do not have a general problem with predicting events but a specific one related to social situations. The study did not specifically ask questions regarding looking to the eyes or the mouth, and the results are therefore less informative on that issue.6 However, we (Falck-Ytter, 2010) have found that in a simpler social predictive task—anticipating the goal of others’ manual actions—children with ASD perform just as well as other children. In that study, children were shown movies in which a female actor moved toys into a bucket. The arrival of the hand at the bucket was compared to the arrival of the gaze. Such goal-directed gaze performance in action observation is thought to reflect an engagement of action plans during action observation (Cannon and Woodward, 2008; Falck-Ytter, 2010; FalckYtter et al., 2006; Flanagan and Johansson, 2003), and is thus of great theoretical interest given the influential idea that ASD is characterized by an impairment in the mirror neuron system (Iacoboni and Dapretto, 2006). The MNS is thought to be an important neural basis for action understanding (Rizzolatti and Sinigaglia, 2010), and action prediction more specifically (Aglioti et al., 2008; Kilner et al., 2004). In light of the results reported by Falck-Ytter (2010), we conclude that there is no global social prediction impairment in ASD. Predictive impairments may be restricted to complex social events involving social interaction between people.

6

Although the von Hofsten et al. (2009) study indicated a social specific impairment, we acknowledge that general attention/perceptual abnormalities are likely to exist in ASD and probably play a role in social perception as well (for a review, see Dinstein et al., 2008).

General discussion This chapter has been concerned with the question where children with ASD devote their visual attention. There is an influential idea that there is an excess mouth and diminished eye gaze bias in ASD. As noted in the introduction, there are two main hypotheses about why there should be less eye gaze in ASD, one stating that people with ASD avoid the eyes (Dalton et al., 2005) and a second stating that people with ASD do not understand visual information from the eyes (Klin et al., 2002a,b). Both these hypotheses were linked to initial data indicating a pattern of excess mouth/diminished eye gaze in ASD, and of course, both hypotheses are questionable if the initial pattern of results is not replicated. Our chapter shows that the support for this idea is mixed in adolescents and adults and has received very little support in children. What are the theoretical implications? The large amount of negative findings most clearly speaks against the view that individuals with ASD (and particularly children with ASD) avoid the eye area, due to a hyperactivation of limbic structures in response to visual information from the eye area (Dalton et al., 2005).7 If this was the case, one would expect much more homogenous data across studies. Although eye avoidance is probably evident

7 In the combined eye tracking and fMRI study by Dalton et al., 2005, it was found that during free looking (no instruction were given to look in a particular area of the face), adults with ASD looked less in the eye area compared to controls, and there was a positive correlation between looking time in the eye area and limbic activation in the ASD group. This was taken as evidence that the ASD group was reacting negatively to the eyes. This view is of course compatible with the group difference, but what about the within-group correlation? Why do the individuals with ASD who look most at the eyes have most limbic activation? If they react negatively to the seeing the eyes, why do they look at them more than their “mouth looking” peers? This issue is not addressed by the authors, leaving many questions open regarding the link between looking time and limbic activity in ASD (for a related finding, see Adolphs et al., 2005).

219

in some individuals with ASD, this seems not to be the “typical” autistic performance. The other main hypothesis states that individuals with ASD fail completely to understand information from the eye area and therefore do not look at that area (Klin et al., 2002b, p. 814). This view was based on the striking difference in looking time at the eye area for adolescents/ adults with ASD compared to controls and the fact that looking time in the eye area was found to be unrelated to the level social dysfunction within the ASD sample. Given the overall emerging picture from the studies reviewed here, we conclude that this extreme position is unlikely to be correct (indeed, other studies have found correlations between looking time in the eye area and social dysfunction; e.g., Falck-Ytter et al., 2010). The group difference without correlation with symptom level found by Klin et al. (2002a,b) may reflect the heterogeneity of ASD (Happe et al., 2006). The data reviewed here suggest that “excess” mouth fixations are indeed normative in early stages of language development in TD children (Chawarska and Shic, 2009; de Wit et al., 2008; Hunnius and Geuze, 2004; Nakano et al., 2010; von Hofsten et al., 2009; Young et al., 2009), and that individual differences in communication/language skills relate to differences in mouth bias within ASD samples as well (Falck-Ytter et al., 2010; Norbury et al., 2009). Perhaps, TD children at certain levels of language development tend to take more use of visual mouth information than children with ASD who very often have language delays. One recent study supported this view, showing that TD children tend to look more at the mouth during speech than other groups, including children with ASD (Nakano et al., 2010). As noted earlier, this finding contradicts a recent hypothesis stating that children with ASD look at the mouth because the mouth includes high levels of audiovisual synchrony (Klin et al., 2009). If this synchrony hypothesis was correct, one would expect ASD children to increase their looking time at

the mouth during periods of speech. The study by Nakano et al. (2010) demonstrates that, at least in school-aged children with ASD, this is not the case. As indicated above, there is more support for the excess mouth/diminished eye gaze hypothesis of ASD in adults than in children. This opens for the interesting idea that reduced eye contact in adults with ASD is not the cause of their dysfunction, but rather its consequence. Moreover, it is possible that the popular idea that children with ASD do not look in the eyes may in fact reflect that children with ASD look less at the people (relative to objects) in general and less at faces more specifically (von Hofsten et al., 2009). We know of no study that has investigated how well an adult observer can judge where in his/her face a child is looking when the child is looking at her. It is, however, well known how precisely adults can judge where another adult is looking when looking at them (Cline, 1967; Gibson and Pick, 1963). The precision is surprisingly high. When the model is facing them, the standard deviation of the judgments corresponds to about half the distance between the eyes at an observation distance of 60 cm. We suspect, however, that most people are not very aware of the exact location of gaze in social encounters. For example, we suspect that few people have been noticing that TD infants/children tend to look so much at the mouth as they actually do (Hunnius and Geuze, 2004; Nakano et al., 2010). We stated in the introduction that there are two main hypotheses relating to where in the face individuals with ASD look. In light of the emerging pattern of results, we suggest that a third hypothesis needs to be considered, one that simply claims that children with ASD are less likely to look at social stimuli than are TD individuals and therefore tend to look at other things rather than people’s faces if they are available. If this hypothesis is valid, children with ASD may be expected to look less in both the eyes and mouth of other people when other more interesting objects are present (Nakano et al., 2010), but

220

not to show a special interest for one particular of the two parts. There are several potential explanations why faces are not as interesting for children with ASD compared to controls (e.g., underdeveloped brain areas of importance for face processing, unusual visual preferences, local perceptual bias). The important point is that while this third hypothesis (nonsocial > social preference in ASD relative to controls) does have massive support (Klin et al., 2009; Nakano et al., 2010; Pierce et al., 2011; von Hofsten et al., 2009), the hypotheses regarding specific face part do not. We have reviewed evidence that the amount of time spent focusing on key features within the face is not unambiguously related to ASD in adolescents/adults, and even less so in children. Eye tracking of videos give somewhat different results than of pictures. Dynamic events, of course, provide much richer information of what is going on than pictures and would for that reason be more reliable. Emotional expressions are less ambiguous, and actions better specified. In addition, social events are more interesting to an observer than a picture and catch the attention better. Therefore, videos are generally preferred in eye-tracking studies of ASD, unless the research questions/ methodological limitations require using pictures (Spezio et al., 2007). Future studies using videos should include fine-grained analyses of looking performance in dynamic (micro-) events; summary statistics for the whole video may hide important differences (Klin et al., 2002a; Nakano et al., 2010). Even better than videos is the use of live eye tracking. The real social world is multimodal, and the observer is seldom a passive receiver of social information, but rather social actor. Therefore, live studies are generally preferable, as long as this format does not compromise rigorous experimental control (Klin et al., 2009). Ideally, future studies of gaze performance in face observation should employ longitudinal designs, include both speaking faces and nonspeaking faces with and without the presence of other nonsocial objects, as well as

repeated measurements of both language development and sociocommunicative impairments (ASD symptoms). Before such studies are undertaken, our understanding of social looking in ASD and typical development will remain rather fragmentized.

Acknowledgments The work was supported by grants to C. v. H. from the Tercentennial Fund of the Bank of Sweden (P09-0933:1) and the Norwegian Directorate for Children, Youth, and Family Affairs (06/34707).

References Adolphs, R., Gosselin, F., Buchanan, T. W., Tranel, D., Schyns, P., & Damasio, A. R. (2005). A mechanism for impaired fear recognition after amygdala damage. Nature, 433(7021), 68–72. Aglioti, S. M., Cesari, P., Romani, M., & Urgesi, C. (2008). Action anticipation and motor resonance in elite basketball players. Nature Neuroscience, 11(9), 1109–1116. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association. Cannon, E., & Woodward, A. L. (2008). Action anticipation and interference: A test of prospective gaze. Paper presented at the Proceedings of the Meetings of the Cognitive Science Society, Austin, Texas. Chawarska, K., & Shic, F. (2009). Looking but not seeing: Atypical visual scanning and recognition of faces in 2 and 4-year-old children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 39(12), 1663–1672. doi:10.1007/s10803-009-0803-7. Cline, M. G. (1967). Perception of where a person is looking. The American Journal of Psychology, 80(1), 41–50. Corden, B., Chilvers, R., & Skuse, D. (2008). Avoidance of emotionally arousing stimuli predicts social-perceptual impairment in Asperger’s syndrome. Neuropsychologia, 46 (1), 137–147. doi:10.1016/j.neuropsychologia.2007.08.005. Dalton, K. M., Nacewicz, B. M., Johnstone, T., Schaefer, H. S., Gernsbacher, M. A., Goldsmith, H. H., et al. (2005). Gaze fixation and the neural circuitry of face processing in autism. Nature Neuroscience, 8(4), 519–526. Dapretto, M., Davies, M. S., Pfeifer, J. H., Scott, A. A., Sigman, M., Bookheimer, S. Y., et al. (2006). Understanding

221 emotions in others: Mirror neuron dysfunction in children with autism spectrum disorders. Nature Neuroscience, 9(1), 28–30. de Wit, T., Falck-Ytter, T., & von Hofsten, C. (2008). Young children with Autism Spectrum Disorder look differently at positive versus negative emotional faces. Research in Autism Spectrum Disorders, 2(4), 651–659. Dinstein, I., Thomas, C., Behrmann, M., & Heeger, D. J. (2008). A mirror up to nature. Current Biology, 18(3), R13–R18. Elsabbagh, M., & Johnson, M. H. (2010). Getting answers from babies about autism. Trends in Cognitive Sciences, 14 (2), 81–87. Falck-Ytter, T. (2008). Face inversion effects in autism: A combined looking time and pupillometric study. Autism Research, 1(5), 297–306. Falck-Ytter, T. (2010). Young children with autism spectrum disorder use predictive eye movements in action observation. Biology Letters, 6(3), 375–378. Falck-Ytter, T., Fernell, E., Gillberg, C., & von Hofsten, C. (2010). Face scanning distinguishes social from communication impairments in autism. Developmental Science, 13(6), 864–865. Falck-Ytter, T., Gredebäck, G., & von Hofsten, C. (2006). Infants predict other people’s action goals. Nature Neuroscience, 9(7), 878–879. Flanagan, J. R., & Johansson, R. S. (2003). Action plans used in action observation. Nature, 424(6950), 769–771. Fletcher-Watson, S., Leekam, S. R., Benson, V., Frank, M. C., & Findlay, J. M. (2009). Eye-movements reveal attention to social information in autism spectrum disorder. Neuropsychologia, 47(1), 248–257. doi:10.1016/j.neuropsychologia.2008.07.016. Fombonne, E. (2005). Epidemiology of autistic disorder and other pervasive developmental disorders. Journal of Clinical Psychiatry, 66, 3–8. Freeth, M., Chapman, P., Ropar, D., & Mitchell, P. (2010). Do gaze cues in complex scenes capture and direct the attention of high functioning adolescents with ASD? Evidence from eye-tracking. Journal of Autism and Developmental Disorders, 40(5), 534–547. doi:10.1007/ s10803-009-0893-2. Gibson, J. J., & Pick, A. D. (1963). Perception of another person looking behavior. American Journal of Psychology, 76(3), 386. Happe, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9 (10), 1218–1220. Hobson, R. P., Ouston, J., & Lee, A. (1988). Whats in a face— The case of autism. British Journal of Psychology, 79, 441–453. Hunnius, S., & Geuze, R. H. (2004). Developmental changes in visual scanning of dynamic faces and abstract stimuli in infants: A longitudinal study. Infancy, 6(2), 231–255. Iacoboni, M., & Dapretto, M. (2006). The mirror neuron system and the consequences of its dysfunction. Nature Reviews Neuroscience, 7(12), 942–951.

Jones, W., Carr, K., & Klin, A. (2008). Absence of preferential looking to the eyes of approaching adults predicts level of social disability in 2-year-old toddlers with autism spectrum disorder. Archives of General Psychiatry, 65(8), 946. Joseph, J. E., Gathers, A. D., Liu, X., Corbly, C. R., Whitaker, S. K., & Bhatt, R. S. (2006). Neural developmental changes in processing inverted faces. Cognitive, Affective and Behavioral Neuroscience, 6(3), 223–235. Kilner, J. M., Vargas, C., Duval, S., Blakemore, S. J., & Sirigu, A. (2004). Motor activation prior to observation of a predicted movement. Nature Neuroscience, 7(12), 1299–1301. Kliemann, D., Dziobek, I., Hatri, A., Steimke, R., & Heekeren, H. R. (2010). Atypical reflexive gaze patterns on emotional faces in autism spectrum disorders. Journal of Neuroscience, 30(37), 12281–12287. doi:10.1523/ jneurosci.0688-10.2010. Klin, A., Jones, W., Schultz, R., & Volkmar, F. (2003). The enactive mind, or from actions to cognition: Lessons from autism. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 358(1430), 345–360. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002a). Defining and quantifying the social phenotype in autism. American Journal of Psychiatry, 159(6), 895–908. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002b). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59 (9), 809–816. Klin, A., Lin, D. J., Gorrindo, P., Ramsay, G., & Jones, W. (2009). Two-year-olds with autism orient to non-social contingencies rather than biological motion. Nature, 459(7244), 257–261. Merin, N., Young, G. S., Ozonoff, S., & Rogers, S. J. (2007). Visual fixation patterns during reciprocal social interaction distinguish a subgroup of 6-month-old infants at-risk for autism from comparison infants. Journal of Autism and Developmental Disorders, 37(1), 108. Nakano, T., Tanaka, K., Endo, Y., Yamane, Y., Yamamoto, T., Nakano, Y., et al. (2010). Atypical gaze patterns in children and adults with autism spectrum disorders dissociated from developmental changes in gaze behaviour. Proceedings of the Royal Society B: Biological Sciences, 277(1696), 2935. Norbury, C. F., Brock, J., Cragg, L., Einav, S., Griffiths, H., & Nation, K. (2009). Eye-movement patterns are associated with communicative competence in autistic spectrum disorders. Journal of Child Psychology and Psychiatry, 50(7), 834–842. Passarotti, A. M., Smith, J., DeLano, M., & Huang, J. (2007). Developmental differences in the neural bases of the face inversion effect show progressive tuning of face-selective regions to the upright orientation. Neuroimage, 34(4), 1708–1722. Pelphrey, K. A., Sasson, N. J., Reznick, J. S., Paul, G., Goldman, B. D., & Piven, J. (2002). Visual scanning of faces in autism. Journal Of Autism And Developmental Disorders, 32(4), 249–261.

222 Pierce, K., Conant, D., Hazin, R., Stoner, R., & Desmond, J. (2011). Preference for geometric patterns early in life as a risk factor for autism. Archives of General Psychiatry, 68, 101–109. Riby, D. M., & Hancock, P. J. B. (2008). Viewing it differently: Social scene perception in Williams syndrome and Autism. Neuropsychologia, 46(11), 2855–2860. Rizzolatti, G., & Sinigaglia, C. (2010). The functional role of the parieto-frontal mirror circuit: Interpretations and misinterpretations. Nature Reviews Neuroscience, 11(4), 264–274. Ronald, A., Happe, F., Bolton, P., Butcher, L. M., Price, T. S., Wheelwright, S., et al. (2006). Genetic heterogeneity between the three components of the autism spectrum: A twin study. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 691–699. Rutherford, M. D., & Towns, A. M. (2008). Scan path differences and similarities during emotion perception in those with and without autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(7), 1371–1381. doi:10.1007/s10803-007-0525-7. Schultz, R. T., Gauthier, I., Klin, A., Fulbright, R. K., Anderson, A. W., Volkmar, F., et al. (2000). Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and Asperger syndrome. Archives of General Psychiatry, 57(4), 331–340. Speer, L. L., Cook, A. E., McMahon, W. M., & Clark, E. (2007). Face processing in children with autism—Effects of

stimulus contents and type. Autism, 11(3), 265–277. doi:10.1177/1362361307076925. Spezio, M. L., Adolphs, R., Hurley, R. S. E., & Piven, J. (2007). Abnormal use of facial information in high-functioning autism. Journal of Autism and Developmental Disorders, 37 (5), 929–939. doi:10.1007/s10803-006-0232-9. Taylor, M. J., Edmonds, G. E., McCarthy, G., & Allison, T. (2001). Eyes first! Eye processing develops before face processing in children. NeuroReport, 12(8), 1671–1676. van der Geest, J. N., Kemner, C., Verbaten, M. N., & van Engeland, H. (2002). Gaze behavior of children with pervasive developmental disorder toward human faces: A fixation time study. Journal of Child Psychology and Psychiatry and Allied Disciplines, 43(5), 669–678. Vatikiotis-Bateson, E., Eigsti, I. M., Yano, S., & Munhall, K. G. (1998). Eye movement of perceivers during audiovisual speech perception. Perception and Psychophysics, 60(6), 926–940. von Hofsten, C., Uhlig, H., Adell, M., & Kochukhova, O. (2009). How children with autism look at events. Research in Autism Spectrum Disorders, 3(2), 556–569. Young, G. S., Merin, N., Rogers, S. J., & Ozonoff, S. (2009). Gaze behavior and affect at 6 months: Predicting clinical outcomes and language development in typically developing infants and infants at risk for autism. Developmental Science, 12(5), 798–814.