- Email: [email protected]
Camouflaged attention: covert attention is critical to social communication in natural settings
Camouflaged attention: Covert attention is critical to social communication in natural settings Kaitlin E.W. Laidlaw, Austin Rothwell, Alan Kingstone PII: DOI: Reference:
S1090-5138(16)30045-9 doi: 10.1016/j.evolhumbehav.2016.04.004 ENS 6047
To appear in:
Evolution and Human Behavior
Received date: Revised date: Accepted date:
27 October 2015 6 March 2016 7 April 2016
Please cite this article as: Laidlaw, K.E.W., Rothwell, A. & Kingstone, A., Camouflaged attention: Covert attention is critical to social communication in natural settings, Evolution and Human Behavior (2016), doi: 10.1016/j.evolhumbehav.2016.04.004
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
SC
RI P
T
ACCEPTED MANUSCRIPT
NU
Camouflaged attention: Covert attention is critical to social communication in natural
MA
settings
Kaitlin E.W. Laidlaw
ED
Austin Rothwell
AC
CE
PT
Alan Kingstone
Word count: 5738 including all but references Department of Psychology University of British Columbia 2136 West Mall Vancouver, British Columbia, Canada, V6T 1Z4 Corresponding Author: Kaitlin E.W. Laidlaw [email protected] (604) 822-0069
ACCEPTED MANUSCRIPT Abstract The evolution of the human eye's unique high contrast morphology allows people to
RI P
T
communicate with a simple look. Yet overt looking is not always preferred in social situations. Do covert shifts in attention – those that occur without a concomitant shift of the eyes or head –
SC
support this need? In the present field study, we discretely recorded pedestrians' looks to a confederate who performed an action - raising his hand to the side of his head and saying „Hey‟
NU
into a phone (private action), or raising his hand to the side of his head in greeting (i.e. a static
MA
wave) and saying „Hey‟ (public action). Critically, pedestrians were not looking at the confederate at the start of the action. Despite this, pedestrians looked more in response to the
ED
public action (wave) than private action (phone). We argue that the observed difference in looking responses must be due to pedestrians first attending to the confederate covertly in order
PT
to assess the intention of his action, and only signaling this attention with a look when socially
CE
appropriate (e.g. to respond to a public action). Though the functional utility of covert attention has rarely been considered outside of controlled laboratory tasks, these results provide the first
behaviour.
AC
demonstration that covert attention plays a critical role in guiding appropriate social looking
Keywords: covert attention, eye movements, social attention, social interaction, signaling & nonverbal communication
ACCEPTED MANUSCRIPT 1. Introduction The morphology of the human eye is unique in that the dark of the iris is surrounded by
RI P
T
the high-contrast of the white sclera. This is divergent from nonhuman primate eyes, whose darker sclera blends in with their iris, thereby camouflaging the direction of their gaze from
SC
predators or competitors (Kobayashi & Kohshima, 1997, 2001b; Perrett & Mistlin, 1990). Put simply, the evolution of the high-contrast human eye has facilitated communication by making
NU
eye position easier to discriminate (Ando, 2002; Ricciardelli, Baylis, & Driver, 2000), and
MA
evidence shows that people use their eyes as a powerful signal to communicate both a desire to initiate interactions (Cary, 1978b), as well as to facilitate ongoing interactions (Emery, 2000;
ED
Kleinke, Staneski, & Berger, 1975; Perrett & Emery, 1994; Saxe, 2006; Tomasello, Carpenter, Call, Behne, & Moll, 2005). In so doing, the heightened visibility of the human eye resulted in
PT
the sacrifice of gaze camouflage (Kobayashi & Kohshima, 2001a, 2001b).
CE
Though it is clear that eye movements are used to communicate to others within social
AC
scenarios, did this actually come at the cost of camouflage? While non-human primates rely on the low-contrast of their eyes to disguise their focus, rather than losing this ability altogether, humans may utilize an alternate method of discretely processing visual information. In fact, we take as a working assumption that effective communication with the eyes may actually require that humans first have some way to discretely assess their immediate social surroundings in order to determine whether or not subsequent signaling with the eyes would be necessary or appropriate. We propose that the covert attentional system is the ideal candidate for discretely processing social information in that it can provide observers with the attentional camouflage that was lost with the evolution of the high-contrast human eye.
ACCEPTED MANUSCRIPT Covert visual attention – that is, attending without a related eye movement – serves to bias perceptual and neural processes of salient or behaviorally relevant stimuli (e.g., see Moore,
T
Armstrong, & Fallah, 2003; Posner & Petersen, 1990). Covert attentional mechanisms are found
RI P
in human and nonhuman primates alike, and function similarly to improve visual acuity at attended as compared to unattended objects (Golla, Ignashchenkova, Haarmeier, & Thier, 2004).
SC
Traditionally, the function of covert attention has been considered to be relatively trivial, likely
NU
because it is often thought that under normal circumstances, people look to where they are attending. Indeed, it was once believed that covert attention was merely a byproduct of
MA
oculomotor planning (Rizzolatti, Riggio, Dascola, & Umiltà, 1987; Rizzolatti, Riggio, & Sheliga, 1994), with brain regions involved in overt attention, such as in fronto-parietal areas, also
ED
showing activation in covert attentional tasks (Beauchamp, Petit, Ellmore, Ingeholm, & Haxby,
PT
2001; Corbetta et al., 1998; de Haan, Morgan, & Rorden, 2008). While it has since been demonstrated that covert attention can be at least partially dissociated from oculomotor planning
CE
(Belopolsky & Theeuwes, 2012; Casarotti, Lisi, Umiltà, & Zorzi, 2012; Schall, 2004; Smith,
AC
Schenk, & Rorden, 2012; Smith & Schenk, 2012), its natural utility has not been seriously entertained. In recent years, the idea that human covert attention might serve a social function has been hinted at, for instance as a way of monitoring an aggressive other (Belopolsky & Theeuwes, 2012), or to hide one's own intentions from others (Klein, Shepherd, & Platt, 2009). However, to our knowledge, it has never been demonstrated that people use covert attention in order to monitor others in social settings. In the present study, we test the hypothesis that people covertly attend to others within social environments in order to facilitate appropriate overt looking behaviour. We did this by recording via hidden camera the responses of pedestrians to a simple action made by a naïve
ACCEPTED MANUSCRIPT confederate. Of interest were the responses from pedestrians who were not initially looking at the confederate, i.e. were not overtly attending to the confederate. As pedestrians approached, the
T
confederate either did nothing, or raised his hand to the side of his head and said 'Hey'. This
RI P
action could end in one of two ways. In the public action condition, the confederate held his empty hand up in a static wave position, as if to greet the oncoming pedestrian. Provided that
SC
pedestrians were attending to the confederate, it was anticipated that this public, social action
NU
would elicit a social response from the oncoming pedestrian: that is, they would acknowledge the greeting by (at minimum) looking at the confederate. In contrast, in the private action condition,
MA
the confederate performed the same action while holding a smart phone, thereby raising his hand to his ear and seemingly delivering the greeting to the phone rather than to the pedestrians. In
ED
this instance, it was anticipated that although the greeting and action may serve to shift their
PT
covert attention toward the confederate, they would avoid signalling their attentiveness in an effort to reduce the potential of engaging with the confederate (e.g. Wu, Bischof, & Kingstone,
CE
2014; Zuckerman, Miserandino, & Bernieri, 1983).
AC
In other words, it was predicted that looking rates would differ based on pedestrians' initial evaluation of the confederate's intention underlying his action. Without already looking at the confederate, covert attention would be the only viable mechanism for pedestrians to use in order to make this subtle peripheral discrimination (i.e. same hand raise with or without a phone). The no action condition served as a baseline by which to compare looking rates. Though we reasoned that the confederate's own looking behaviour could not be discriminated by the pedestrians using their peripheral vision at the viewing distances used in the present study (e.g., Loomis, Kelly, Pusch, Bailenson, & Beall, 2008), it was prudent to test this as laboratory tasks have often shown strong attentional effects of another‟s gaze (Frischen, Bayliss, & Tipper, 2007; Nummenmaa &
ACCEPTED MANUSCRIPT Calder, 2009; Senju & Johnson, 2009). Thus, either hand action was accompanied by the confederate looking at the pedestrian or staring straight ahead.
RI P
T
Actions were executed in a manner that minimized any differences across conditions except for the addition of a cell phone in the private action condition. This is not to suggest that all other
SC
changes were eliminated, as this is an unrealistic expectation for a naturalistic study such as this. In fact, small fluctuations in environmental variables such as crowding, lighting, or confederate
NU
posturing are better considered as a strength of field research: if changes in the signal –
MA
pedestrian looking behaviour – can be reliably measured despite the added 'noise' characteristic of naturalistic environments, then it supports the view that this behaviour is robust and not
ED
simply an artifact of a particular testing scenario. In sum, despite pedestrians not looking at the confederate at the start of the action and despite the minor changes present in each participant's
PT
immediate environment, we hypothesize that they will covertly attend to the confederate's action
CE
and, critically, tailor their response to his action such that they will look up at the confederate
2. Methods
AC
more frequently when his hand is raised to form a wave than to answer the phone.
2.1 Equipment and procedure Pedestrians were sampled from the two locations on the University of British Columbia's Vancouver campus - one indoor and one outdoor (Figure 1). A residence dining hall was used as the indoor location, where a walkway cut between a wall and a row of study rooms, leading to the building exit. The outdoor location was a path, bordered by garden on either side, which passed in between two buildings. Importantly, both locations had a linear flow of pedestrian traffic (i.e., where pedestrians could only walk straight, in two opposing directions) and
ACCEPTED MANUSCRIPT oncoming pedestrians were easily visible. In both locations, the confederate was situated in the flow of pedestrian traffic without obstructing pedestrian movement.
RI P
T
[FIGURE 1 ABOUT HERE]
The confederate was a male Caucasian undergraduate (20 yrs, 5'9") from the University
SC
of British Columbia, and was dressed in neutral clothing and wore nothing to obstruct his eyes
NU
(e.g., a hat, glasses, avoided squinting). The specific hypotheses related to pedestrian looking behavior were not disclosed to the confederate until after testing. The confederate stood casually,
MA
using a black smart phone down by his stomach/waist, and occasionally looked straight ahead to search for distant pedestrians that roughly met selection criteria (see below). When the
ED
confederate observed that a pedestrian met basic selection criteria, the confederate would re-
PT
engage with his phone at waist level until the pedestrian was within a pre-designated distance agreed upon prior to the start of the study, at which point he would initiate an action (described
CE
below). Representative distance measurements taken following pedestrian data collection
AC
estimated that the confederate initiated the action an average of 1.52 m (SD = .30 m) in front of the pedestrian. A relatively close distance was chosen for two reasons. First, research suggests that as pedestrians approach a stranger, they will divert their overt attention (both their head and eyes) away from the individual (Fotios, Yang, & Uttley, 2015; Goffman, 1963; Patterson, Webb, & Schwartz, 2002). Second, classic proxemics research suggests that the space around a person can be divided into areas of interpersonal space, with 1.2 m to 3.7 m representing 'social space' in which people are comfortable engaging in conversation and other casual interactions (Hall, 1968). Thus, the confederate aimed to initiate the action within a distance that is not only considered social space (i.e. in which a greeting might be made to another), but that overt attention is nevertheless diverted away from non-familiar others.
ACCEPTED MANUSCRIPT The action performed by the confederate was a greeting that involved hand, eye and verbal components, the specifics of which were as closely matched as possible so as to avoid any
T
differences in large motion signals across conditions. For the hand action, the confederate raised
RI P
his empty right hand near his ear as if to wave (i.e. palm forward), or raised the same hand and brought his cell phone to his ear (Figure 2). In all conditions, the confederate engaged with the
SC
cell phone prior to the start of the action. As such, the presence of the cell phone could not have
NU
served as a cue to the confederate‟s upcoming action. The arm motion in both cases mimicked the typical action of answering a phone, and the final position (wave or answering phone) was
MA
held static until the pedestrian passed (i.e. the wave condition did not include any additional movement of the hand). A forward facing head position was maintained throughout. In addition,
ED
the confederate either looked straight ahead (i.e. avoided eye contact), or fixated the pedestrian at
PT
the start of the action, and followed the pedestrian with his eyes until they approached too near to continue to do so. In all conditions, the confederate said „Hey‟ in a consistent and friendly tone
CE
as his hand action ended. Once the pedestrian passed, the confederate ended the action by
AC
withdrawing his hand and returning to using the phone near his waist. The confederate was instructed to avoid any additional interaction with the passersby, such as smiling or nodding, though in rare instances where this happened (e.g. the pedestrian stopped to ask a question), the pedestrian was removed from the analysis. In addition, a 'no action' baseline condition was culled from pre-recorded video sessions. Pedestrians were selected who met the selection criteria but passed the confederate while he was not performing any action. Thus, confederate's behaviour in these clips depicted natural waiting behaviour, and prominently featured the actor looking down at his phone, while occasionally looking up and away in the distance (i.e. not at nearby pedestrians).
ACCEPTED MANUSCRIPT [FIGURE 2 ABOUT HERE] Events were recorded using a high-definition (1080p) video camera placed so that it was
RI P
T
partially obstructed from the participant‟s view (i.e. stored in a backpack, with an opening for the lens). The camera was placed behind (outdoors: 15.21 m; indoors: 6.67 m) the confederate and to
SC
his left (outdoors: 6.70 m, indoors: 2.92 m), at a similar angle to the confederate (outdoors: 23.73˚; indoors: 23.66˚). An experimenter sat beside the hidden camera and recorded participant
NU
descriptives (e.g. sex, visual description) and which condition was run. Differences in camera-to-
MA
confederate distance were necessary to discretely position the camera and experimenter. Though locations differed in the distance of the camera from the confederate due to environmental
ED
constraints, confederate size was approximately equated through the use of camera zoom.
PT
2.2 Pedestrian sampling
CE
Data was collected over 12 sessions (7 Outside, 5 Inside). During each testing session, all conditions were run, in a pseudo-random order, documenting roughly ten pedestrians
AC
consecutively for each condition before moving to the next condition. Efforts were made to select only pedestrians who were walking alone, whose eyes were visible, and who were not wearing earphones or actively using a portable electronic device. In addition, the confederate delayed repeating an action immediately following a sampled pedestrian so as to avoid another pedestrian observing the behaviour from a distance. As participation in the study involved only observation in public space, no consent from selected pedestrians was required and no debriefing was provided. The study was approved by the University of British Columbia's Research Ethics Board. 3. Results
ACCEPTED MANUSCRIPT 3.1 Data handling The video recording was cut into short clips commencing with the participant entering
RI P
T
the frame and ending when the participant exited the frame. Any instance in which the pedestrian was looking at the confederate prior to his action were discarded, as were instances in which the
SC
confederate's action started before the participant entered the frame, as coders would be unable to ensure that participants were not overtly attending to the confederate just prior to the start of the
NU
action. Instances in which pedestrians were blocked during critical moments were also discarded,
MA
as were any instances caught with repeat pedestrians. In total, 483 videos were recorded that contained sufficient information to allow for subsequent coding. Two coders who were naïve to
ED
the hypotheses of the study coded all videos for pedestrian looking behaviour, timing, and environment details1. Coding was performed using software that enabled frame-by-frame
PT
viewing. As multiple looks were relatively infrequent, looking behaviour was coded in a binary
CE
fashion: following the confederate action, it was coded whether or not the pedestrian looked at the confederate. Looks were counted even if the pedestrian passed the confederate and turned
AC
back to look at him, though these instances were rare. Videos were excluded if at least one coder noted that the pedestrian or video did not meet criteria stated above, if gaze was indiscernible (e.g. by shadow), or if at least one coder noted that the pedestrian was looking at the confederate immediately prior to the start of the action (total: 100 videos, 20.70%). 1
Pedestrian ethnicity was also coded with good reliability (Krippendorff's Alpha = 0.83). The majority of pedestrians were either rated as Caucasian (42.64% of sample based on Coder 1's ratings) or East Asian (40.24% of the sample). Limiting analyses to only these two ethnic groups shows a small effect of ethnicity on pedestrian looking frequency, with East Asian pedestrians looking less than Caucasian, p < .05 (see Patterson et al., 2007 for a similar pattern). As this did not interact with our critical manipulations, ethnicity was not included as a factor in the reported analyses. Reliability was also good for pedestrian age group (Krippendorff's Alpha = 0.88) and environmental crowding (rated on a 10point scale, Krippendorff's Alpha = .77). Due to a relative lack of variability in these factors, however, they were excluded from analysis.
ACCEPTED MANUSCRIPT Coder reliability was assessed using Krippendorff's alpha and was good for all measures (Table 1). Coder reliability did not vary substantially based on location (e.g. Krippendorff‟s
T
Alpha for either location was never divergent from the reported value by more than .02),
RI P
suggesting that the use of camera zoom did not noticeably influence the coder‟s ability to detect pedestrian looks. If coders disagreed on whether the pedestrian looked, the pedestrian was
SC
excluded from analysis (n = 50), thus conservatively limiting analyses only to instances where
NU
both coders agreed on whether the pedestrian looked or not. Analyses were therefore based on
MA
333 pedestrians. Table 1. Coder reliability
ED
Krippendorff's Alpha 0.73 0.82 0.95
CE
PT
Variable Looks to confederate following action Time to pass confederate (from start of action) Time to exit frame following action initiation
AC
3.2 Pedestrian and video descriptives Of the 333 pedestrians, 168 (50.45%) were female and 172 (51.65%) were recorded from the outdoor location. The majority of the pedestrians sampled were judged to be between 16-29 (88.59%), with all other pedestrians being judged 30 or older (11.41%). The confederate raised his hand to wave for 136 (40.84%) of pedestrians. Of those, the confederate looked straight ahead for 63 (46.32%) cases while he looked directly at 73 (53.68%) of the passing pedestrians, The confederate raised his hand to answer his phone for 115 (34.53%) pedestrians, for which he
ACCEPTED MANUSCRIPT looked straight ahead for 57 (49.57%) cases or at the pedestrian for 58 (50.43%) cases. There were 82 pedestrians who passed the confederate while he performed no action (24.62%).
RI P
T
Average clip duration was 9.54 s (SD = 3.36 s). Based on the results from either coder, Time to pass the confederate (from the start of the action) and Total time to react (from start of
SC
action to exiting frame) did not vary as a function of confederate action or looking behaviour, all
NU
ps > .05.
MA
3.3 Did confederate actions influence overt looking frequency? The primary question was whether pedestrians would change looking behaviour in
ED
response to the confederate's action. As such, analyses were first conducted on only the 'Wave' and 'Phone' conditions. Owing to the dichotomous nature of the dependent variable, looking
PT
behaviour was analyzed using a loglinear analysis with location (outside, inside), confederate
CE
hand action (wave, phone), and confederate eye position action (looking straight ahead or at the pedestrian) as factors. Loglinear analysis can be thought of as an extension of Pearson's chi-
AC
square analysis when examining the relationship between more than two categorical variables. In a loglinear analysis, models are built and tested to find the least complex model (i.e. with the fewest factors) that best accounts for the variance in the observed frequencies. The only major assumptions of the analysis are that the observations are independent, and that the expected frequencies for each cell of the contingency table (i.e. each condition) should be large enough (greater than 5) to permit a reliable analysis. A stepwise backwards elimination procedure was used to determine which of the factors and their interactions significantly reduced model fit. In this way, a fully saturated model is first tested, and then interactions and factors are eliminated in a stepwise fashion (i.e. the highest order interaction effect is removed first, then if this does not
ACCEPTED MANUSCRIPT have a significant effect on model fit, the second highest-order effect is removed, etc) until such a point where eliminating a factor reduces the model fit significantly. Performing this analysis
T
revealed that only removing the interaction between of confederate hand action and pedestrian
RI P
looking behaviour had a significant effect on the model, χ2(1) = 38.87, p < .001. Pedestrians being more likely (odds ratio: 5.20) to look at the confederate following a wave than when he
SC
lifted his phone and said 'Hey'. Removing any other factors or their interactions had no
NU
significant effect, indicating that location and confederate looking behaviour (or their interactions) did not impact pedestrian looking responses. The final model with only pedestrian
MA
looking behaviour and confederate action resulted in a non-significant likelihood ratio of χ2(12) = 9.91, p = .62, which indicates that the model including only this interaction represents a good
ED
fit of the data2.
PT
3.4 How do overt looking frequencies compare against baseline?
CE
Though it is clear that overt looking rates change as a function of confederate action, it is
AC
unclear from the above data what 'normal' looking rates would be, and how those rates would relate to that observed when passing a confederate making an action. Of particular interest was how looks in response to the confederate raising his phone compared to baseline looking behaviour from the pedestrians. A chi-square analysis of looking behaviour with confederate
2
The confederate was blind to the specific hypotheses of the study until after data collection, though it may be argued that his performance of the arm action may have varied across conditions, thereby unduly attracting attention in one condition over another. To test this, we randomly selected a subset of trials from the Wave and Phone conditions (20 each; collapsing over Gaze & Location conditions), and determined the overall movement time from start of hand raise to final hand position. As peripheral vision is especially adept at detecting motion cues, this seemed the most likely candidate behaviour that may have inadvertently influenced looking responses, e.g., faster movement times would stand out more than if the same action were performed more slowly. However, analysis revealed no significant differences in motion time, t(38) = 0.83, p = .41.
ACCEPTED MANUSCRIPT action (wave, phone, no action) as a factor revealed a significant association, χ2(2) = 63.98, p < .001. To explore this further, the original 3x2 chi-square was partitioned to reveal that looking
T
behaviour was more likely (odds-ratio: 6.49) for the wave condition than in the phone and no-
RI P
action conditions combined, χ2(1) = 61.85, p < .001. Interestingly, looking frequencies in phone and no action conditions were comparable, χ2(1) = 2.81, p = .09 (odds-ratio: 1.80). Table 2
SC
reports the values for each condition; Figure 3 graphically presents the main effect of confederate
NU
action on looking responses.
PT
ED
Looked at pedestrian 17 41 58 29.31
Wave Looked straight ahead 40 23 63 63.49
Looked at pedestrian
No Action Looked down
52 21 73 71.23
15 67 82 18.29
AC
CE
Pedestrian looked Pedestrian did not look N Looking frequency (%)
Phone Looked straight ahead 16 41 57 28.07
MA
Table 2. Pedestrian looking frequency across confederate eye position and action conditions
[FIGURE 3 ABOUT HERE] 4. Discussion
In social environments, there may be no more important source of information to focus on than the actions of other people nearby, as they can dynamically alter one's own behaviour. Yet, monitoring the intentions of others by looking at them is not always appropriate or welcome. As such, it was hypothesized that humans have come to rely on covert attentional deployment to monitor nearby others, thereby enabling them to process other's actions while limiting their overt looking responses. To test this, the present study assessed whether a confederate's action – raising his hand to either wave or answer his phone – would elicit different
ACCEPTED MANUSCRIPT looking responses in passing pedestrians. Confederate eye gaze and testing locations were also varied. Though pedestrians were not overtly attending at the time that the action started, looking
T
responses to the confederate varied significantly based on the confederate's action: pedestrians
RI P
were five times more likely to look in response to a wave than to answering a phone. Confederate looking behaviour and testing location did not significantly influence responses. The
SC
difference in pedestrian looking response rates across action conditions required that pedestrians
NU
first covertly process the confederate's action in order to tailor their own response appropriately, thereby demonstrating for the first time that people use their covert attention as a way of
MA
observing and assessing other's intentions without overtly signaling this to others nearby. Through the use of covert attention, pedestrians in the present study were able to
ED
modulate their looking behaviour in order to signal attention in instances when it was socially
PT
appropriate. Looking in response to a wave may have served various, not independent, purposes, such as gaining more information about the other person's intentions (e.g. 'Is he waving at me?)
CE
or acknowledging the greeting by trying to establish mutual gaze. In support of the latter view,
AC
signaling one's acknowledgment of the other's greeting, or reciprocal recognition, is considered a standard social convention (Duranti, 1997), and serves to reinforce positive social relationships (Goffman, 1971), even in nonhuman primates (De Marco, Sanna, Cozzolino, & Thierry, 2014), and thus may be initiated even when pedestrians were uncertain that they were the target of the greeting. Though the exact intentionality of the look to the confederate is ambiguous, the critical point is that the pedestrians permitted themselves to react overtly only in the condition where the confederate's action could be interpreted as an invitation to interact. When the confederate's action was covertly determined to be a potential communicative signal, pedestrians responded in turn by using their looking behaviour as a communicative tool.
ACCEPTED MANUSCRIPT Although there is substantial literature detailing that motion (e.g., Abrams & Christ, 2003; Franconeri & Simons, 2003) and auditory stimuli (e.g., McDonald, Teder-Sälejärvi, &
T
Hillyard, 2000; Spence & Driver, 1997) attract visual attention, it is clear that in the present
RI P
study, looking behaviour was not simply an overt exogenous reaction to the act of raising the hand or hearing a verbal greeting. If that had been the case, pedestrians should have looked
SC
equally often in both wave and phone conditions, which both involved large arm motions and
NU
auditory cues, and least often when the confederate did nothing. However, it was only in the wave condition that pedestrians often looked towards the confederate; the phone condition,
MA
which was physically most similar to the wave condition, did not elicit pedestrian responses to nearly the same extent. Instead, looks were infrequent both when the confederate answered the
ED
phone and when he did nothing, despite these conditions varying dramatically in their
PT
involvement of attentional cues. It is extremely unlikely that the significant difference in looking behaviour between phone and wave conditions could be attributed to a „stronger‟ exogenous pull
CE
of one arm raise over the other, even if the final end position differed slightly; if the motion cue
AC
was insufficient to exogenously attract attention, so too would have been a small change in hand position between answering a phone and holding the hand near the face to „wave‟. However, the lack of looks in the phone condition should not be taken as evidence that pedestrians were not attending, as several recent findings suggest that people avoid looking at strangers when they believe their actions might be visible (and interpretable) by the other individual. For example, Gallup and colleagues (2012) have shown that pedestrians are more likely to follow another‟s gaze when walking behind or beside a group of lookers, as compared to when they walk in front (see also Gallup, Chong, & Couzin, 2012). Not only are people aware of the ability of their gaze to signal information, when in the presence of strangers they tend to
ACCEPTED MANUSCRIPT assume a gaze avoidance strategy. As such, the infrequency of looks to the confederate when he answered his phone is better described as pedestrians avoiding subsequent looking behaviour
T
following covert examination of the confederate. This inhibition was accomplished effectively,
RI P
so much so that looking rates in response to the confederate answering his phone did not
SC
significantly differ from when he performed no action at all.
As anticipated, looking behaviour in the present study was unaffected by whether the
NU
confederate stared straight ahead or if he focused on the pedestrian while they passed. While we
MA
will limit drawing many conclusions based on a non-significant effect, it bears mention in so much as it contrasts with many laboratory tasks suggesting that people are adept at detecting eye
ED
gaze directed towards them (Itier, Villate, & Ryan, 2007; Senju, Hasegawa, & Tojo, 2005; Senju & Hasegawa, 2005; von Grünau & Anston, 1995). As the confederate maintained head position
PT
across conditions, we suspect, as others have argued, that the visual angle subtended by the eyes
CE
was too small, and acuity too poor in peripheral vision to reliably detect changes in gaze direction (Loomis et al., 2008). Future investigations of the effects of eye gaze on attentional
AC
orienting may provide insight into whether this social cue is useful within different naturalistic conditions (e.g. when already attending near another‟s face). Not only do the present findings draw attention to the need for future research into the social influences on visual behaviour, they also lay the groundwork for better understanding of how covert attention is directed to people in everyday scenarios. For example, did pedestrians choose to covertly attend to the confederate (perhaps when passing or because of the initiation of an action), or was the confederate already prioritized within the attentional system owing to him being a social stimulus? There is little field work that speaks to this point, though laboratory based tasks would argue that people have an at least partially non-volitional attentional bias
ACCEPTED MANUSCRIPT towards social stimuli (Birmingham, Bischof, & Kingstone, 2009; Devue, Belopolsky, & Theeuwes, 2012; Laidlaw, Badiudeen, Zhu, & Kingstone, 2015; Laidlaw, Risko, & Kingstone,
T
2012). Several researchers have argued that attention may be proactively biased by highly
RI P
relevant or highly valuable stimuli (Anderson, Laurent, & Yantis, 2011; Anderson, 2013; Todd, Cunningham, Anderson, & Thompson, 2012; Wieser, McTeague, & Keil, 2011), and one could
SC
easily argue that nearby others reflect a perfect example of such a stimulus. What remains to be
NU
seen is whether these same automated biases are present within naturalistic environments, or if instead people strategically shift their covert attention to other people when convenient or
MA
necessary.
ED
Though it has long been known that attentional and oculomotor systems can operate independently (Eriksen & Yeh, 1985; Juan, Shorter-Jacobi, & Schall, 2004; Posner, 1980), there
PT
has been little emphasis placed on the role that covert attention plays within social environments.
CE
This omission is likely due in part to the manner in which social attention has so often been studied: using images of people who cannot look back. In real life, looking to another signals
AC
information to that person, who themselves can react, creating a cascade of responses from both parties. The awareness that one's looking behaviour can be interpreted by others in real life constrains the way in which people direct their overt attention around others (Cary, 1978b; Gallup, Chong, et al., 2012; Gallup, Hale, et al., 2012; Gobel, Kim, & Richardson, 2015; Laidlaw, Foulsham, Kuhn, & Kingstone, 2011; Wu et al., 2014), thereby creating a need for another mechanism – covert attention – to process important social information. In contrast, there are no social consequences of looking at images. In instances where looking serves no communicative role, as is the case in many traditional laboratory tasks of social attention, it may be that the decoupling of covert from overt attention is largely unnecessary, and as such, the
ACCEPTED MANUSCRIPT utility of covert social attention has gone largely unstudied. It is clear that in order to make generalized claims of the vital role of covert attention within naturalistic social settings, the
T
present results will need to be replicated and extended to new situations. Further, the present
RI P
results should not be taken to imply that the sole use of covert attention is socially-based, or that nonhuman primates do not have the ability to covertly attend. This should not diminish the
SC
impact of the current findings, however, as they serve as the first proof of concept: within a
NU
common social situation unaffected by unrealistic experimental manipulation, people direct covert attention to others in order to scaffold their subsequent looking behavior. As such, the
MA
present study represents an initial step toward a new line of research focused on how covert social attention operates in a naturalistic environment and guides overt looking responses within
ED
social environments.
PT
Finally, the present findings also highlight the importance not only of looking to
CE
communicate with others, but also the subtle but meaningful way in which people use the absence of looks as a communicative signal onto itself. While the value placed in averted gaze
AC
from other people varies depending on the culture, the appropriate use and withdrawal of looks to another emerges as an important social norm within many groups (Argyle & Cook, 1976; McCarthy, Lee, Itakura, & Muir, 2006, 2008; Rossano, Brown, & Levinson, 2009), and violations of these norms may be interpreted to signal developmental or neurological impairments (Moukheiber et al., 2010; Tanaka & Sung, 2013). As was observed in the present study, not looking overwhelmingly communicates a lack of attentiveness or signals an unwillingness to interact further; the use of gaze in this way is becoming a frequently observed social phenomenon within naturalistic environments (e.g. Cary, 1978a; Foulsham, Walker, &
ACCEPTED MANUSCRIPT Kingstone, 2011; Gallup, Chong, et al., 2012; Gallup, Hale, et al., 2012; Goffman, 1963; Laidlaw et al., 2011).
RI P
T
Acknowledgements & Funding
The authors thank Jalila Devji and Tara Luk for their work coding the videos. Funding for this
SC
project was provided by a Natural Sciences and Engineering Research Council of Canada
NU
Discovery Grant and a Social Sciences and Humanities Research Council of Canada Insight
AC
CE
PT
ED
MA
Grant.
ACCEPTED MANUSCRIPT References
T
Abrams, R. A., & Christ, S. E. (2003). Motion onset captures attention. Psychological Science, 14(5), 427–432. http://doi.org/10.1111/1467-9280.01458
RI P
Anderson, B. A. (2013). A value-driven mechanism of attentional selection. Journal of Vision, 13(3), 1–16. http://doi.org/10.1167/13.3.7.doi
SC
Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences of the U.S.A., 108(25), 10367–10371. http://doi.org/10.1073/pnas.1104047108
NU
Ando, S. (2002). Luminance-induced shift in the apparent direction of gaze. Perception, 31(6), 657–674. http://doi.org/10.1167/1.3.287
MA
Argyle, M., & Cook, M. (1976). Gaze and mutual gaze. Oxford, England, England: Cambridge University Press. Beauchamp, M. S., Petit, L., Ellmore, T. M., Ingeholm, J., & Haxby, J. V. (2001). A parametric fMRI study of overt and covert shifts of visuospatial attention. NeuroImage, 14, 310–321. http://doi.org/10.1006/nimg.2001.0788
PT
ED
Belopolsky, A. V, & Theeuwes, J. (2012). Updating the premotor theory: The allocation of attention is not always accompanied by saccade preparation. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 902–914. http://doi.org/10.1037/a0028662
CE
Birmingham, E., Bischof, W. F., & Kingstone, A. (2009). Saliency does not account for fixations to eyes within social scenes. Vision Research, 49(24), 2992–3000. http://doi.org/10.1016/j.visres.2009.09.014
AC
Cary, M. (1978a). Does civil inattention exist in pedestrian passing? Journal of Personality and Social Psychology, 36(11), 1185–1193. http://doi.org/10.1037/0022-3514.36.11.1185 Cary, M. (1978b). The role of gaze in the initiation of conversation. Social Psychology, 41(3), 269–271. http://doi.org/10.2307/3033565 Casarotti, M., Lisi, M., Umiltà, C., & Zorzi, M. (2012). Paying attention through eye movements: A computational investigation of the premotor theory of spatial attention. Journal of Cognitive Neuroscience, 24(7), 1519–1531. http://doi.org/10.1162/jocn_a_00231 Corbetta, M., Akbudak, E., Conturo, T. E., Snyder, A. Z., Ollinger, J. M., Drury, H. A., … Shulman, G. L. (1998). A common network of functional areas for attention and eye movements. Neuron, 21, 761–773. http://doi.org/10.1016/S0896-6273(00)80593-0 de Haan, B., Morgan, P. S., & Rorden, C. (2008). Covert orienting of attention and overt eye movements activate identical brain regions. Brain Research, 1204, 102–111. http://doi.org/10.1016/j.brainres.2008.01.105 De Marco, A., Sanna, A., Cozzolino, R., & Thierry, B. (2014). The function of greetings in male Tonkean macaques. American Journal of Primatology, 76(10), 989–998. http://doi.org/10.1002/ajp.22288
ACCEPTED MANUSCRIPT Devue, C., Belopolsky, A. V, & Theeuwes, J. (2012). Oculomotor guidance and capture by irrelevant faces. PloS One, 7(4), e34598. http://doi.org/10.1371/journal.pone.0034598
T
Duranti, A. (1997). Universal and culture-specific properties of greetings. Linguistic Anthropology, 7(1), 63–97. http://doi.org/10.1525/jlin.1997.7.1.63
RI P
Emery, N. J. (2000). The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24(6), 581–604. http://doi.org/10.1016/S01497634(00)00025-7
SC
Eriksen, C. W., & Yeh, Y. Y. (1985). Allocation of attention in the visual field. Journal of Experimental Psychology: Human Perception & Performance, 11(5), 583–597.
NU
Fotios, S., Yang, B., & Uttley, J. (2015). Observing other pedestrians: Investigating the typical distance and duration of fixation. Lighting Research and Technology, 47(5), 548–564. http://doi.org/10.1177/1477153514529299
MA
Foulsham, T., Walker, E., & Kingstone, A. (2011). The where, what and when of gaze allocation in the lab and the natural environment. Vision Research, 51(17), 1920–1931. http://doi.org/10.1016/j.visres.2011.07.002
ED
Franconeri, S. L., & Simons, D. J. (2003). Moving and looming stimuli capture attention. Perception & Psychophysics, 65(7), 999–1010. http://doi.org/10.3758/BF03194829
PT
Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007). Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133(4), 694–724. http://doi.org/10.1037/0033-2909.133.4.694
CE
Gallup, A. C., Chong, A., & Couzin, I. D. (2012). The directional flow of visual information transfer between pedestrians. Biology Letters, 8(4), 520–522. http://doi.org/10.1098/rsbl.2012.0160
AC
Gallup, A. C., Hale, J. J., Sumpter, D. J. T., Garnier, S., Kacelnik, A., Krebs, J. R., & Couzin, I. D. (2012). Visual attention and the acquisition of information in human crowds. Proceedings of the National Academy of Sciences of the United States of America, 109(19), 7245–7250. http://doi.org/10.1073/pnas.1116141109 Gobel, M. S., Kim, H. S., & Richardson, D. C. (2015). The dual function of social gaze. Cognition, 136, 359–364. http://doi.org/10.1016/j.cognition.2014.11.040 Goffman, E. (1963). Behavior in public places: Notes on the social organization of gatherings. New York: The Free Press. Goffman, E. (1971). Relations in public: Microstudies of the public order. New York: Basic Books. Golla, H., Ignashchenkova, A., Haarmeier, T., & Thier, P. (2004). Improvement of visual acuity by spatial cueing: A comparative study in human and non-human primates. Vision Research, 44(13), 1589–1600. http://doi.org/10.1016/j.visres.2004.01.009 Hall, E. T. (1968). Proxemics. Current Anthropology, 9(2/3), 83–108. http://doi.org/10.1086/200975 Itier, R. J., Villate, C., & Ryan, J. D. (2007). Eyes always attract attention but gaze orienting is
ACCEPTED MANUSCRIPT task-dependent: Evidence from eye movement monitoring. Neuropsychologia, 45(5), 1019– 1028. http://doi.org/10.1016/j.neuropsychologia.2006.09.004
T
Juan, C.-H., Shorter-Jacobi, S. M., & Schall, J. D. (2004). Dissociation of spatial attention and saccade preparation. Proceedings of the National Academy of Sciences of the United States of America, 101(43), 15541–15544. http://doi.org/10.1073/pnas.0403507101
RI P
Klein, J. T., Shepherd, S. V, & Platt, M. L. (2009). Social attention and the brain. Current Biology, 19(20), R958–R962. http://doi.org/10.1016/j.cub.2009.08.010
SC
Kleinke, C. L., Staneski, R. A., & Berger, D. E. (1975). Evaluation of an interviewer as a function of interviewer gaze, reinforcement of subject gaze, and interviewer attractiveness. Journal of Personality and Social Psychology, 31(1), 115–122. http://doi.org/10.1037/h0076244
NU
Kobayashi, H., & Kohshima, S. (1997). Unique morphology of the human eye. Nature, 387(6635), 767–768. http://doi.org/10.1038/42842
MA
Kobayashi, H., & Kohshima, S. (2001a). Evolution of the human eye as a device for communication. In T. Matsuzawa (Ed.), Primate Origins of Human Cognition and Behavior (pp. 383–401). Japan: Springer.
ED
Kobayashi, H., & Kohshima, S. (2001b). Unique morphology of the human eye and its adaptive meaning: Comparative studies on external morphology of the primate eye. Journal of Human Evolution, 40(5), 419–435. http://doi.org/10.1006/jhev.2001.0468
PT
Laidlaw, K. E. W., Badiudeen, T. A., Zhu, M. J. H., & Kingstone, A. (2015). A fresh look at saccadic trajectories and task irrelevant stimuli: Social relevance matters. Vision Research, 111(Part A), 82–90. http://doi.org/10.1016/j.visres.2015.03.024
AC
CE
Laidlaw, K. E. W., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences of the United States of America, 108(14), 5548–5553. http://doi.org/10.1073/pnas.1017022108 Laidlaw, K. E. W., Risko, E. F., & Kingstone, A. (2012). A new look at social attention: Orienting to the eyes is not (entirely) under volitional control. Journal of Experimental Psychology: Human Perception and Performance, 38(5), 1132–1143. http://doi.org/10.1037/a0027075 Loomis, J. M., Kelly, J. W., Pusch, M., Bailenson, J. N., & Beall, A. C. (2008). Psychophysics of perceiving eye-gaze and head direction with peripheral vision: Implications for the dynamics of eye-gaze behavior. Perception, 37(9), 1443–1457. http://doi.org/10.1068/p5896 McCarthy, A., Lee, K., Itakura, S., & Muir, D. W. (2006). Cultural display rules drive eye gaze during thinking. Journal of Cross-Cultural Psychology, 37(6), 717–722. http://doi.org/10.1177/0022022106292079 McCarthy, A., Lee, K., Itakura, S., & Muir, D. W. (2008). Gaze display when thinking depends on culture and context. Journal of Cross-Cultural Psychology, 39(6), 716–729. http://doi.org/10.1177/0022022108323807 McDonald, J. J., Teder-Sälejärvi, W. a, & Hillyard, S. a. (2000). Involuntary orienting to sound
ACCEPTED MANUSCRIPT improves visual perception. Nature, 407(6806), 906–908. http://doi.org/10.1038/35038085 Moore, T., Armstrong, K. M., & Fallah, M. (2003). Visuomotor origins of covert spatial attention. Neuron, 40(4), 671–683. http://doi.org/10.1016/S0896-6273(03)00716-5
RI P
T
Moukheiber, A., Rautureau, G., Perez-Diaz, F., Soussignan, R., Dubal, S., Jouvent, R., & Pelissolo, A. (2010). Gaze avoidance in social phobia: Objective measure and correlates. Behaviour Research and Therapy, 48(2), 147–151. http://doi.org/10.1016/j.brat.2009.09.012
SC
Nummenmaa, L., & Calder, A. J. (2009). Neural mechanisms of social attention. Trends in Cognitive Sciences, 13(3), 135–143. http://doi.org/10.1016/j.tics.2008.12.006
NU
Patterson, M. L., Iizuka, Y., Tubbs, M. E., Ansel, J., Tsutsumi, M., & Anson, J. (2007). Passing encounters East and West: Comparing Japanese and American pedestrian interactions. Journal of Nonverbal Behavior, 31(3), 155–166. http://doi.org/10.1007/s10919-007-0028-4
MA
Patterson, M. L., Webb, A., & Schwartz, W. (2002). Passing encounters: Patterns of recognition and avoidance in pedestrians. Basic and Applied Social Psychology, 24(1), 57–66. http://doi.org/10.1207/S15324834BASP2401_5
ED
Perrett, D. I., & Emery, N. J. (1994). Understanding the intentions of others from visual signals: Neurophysiological evidence. Current Psychology of Cognition, 13(5), 683–694. Retrieved from http://doi.apa.org/psycinfo/1995-24608-001
PT
Perrett, D. I., & Mistlin, A. J. (1990). Perception of facial characteristics by monkeys. In W. C. Stebbins & M. A. Berkley (Eds.), Comparative perception, Vol. 2: Complex signals. (Vol 2, pp. 187–215). Oxford: John Wiley & Sons.
CE
Posner, M. I. (1980). Orienting of attention. The Quarterly Journal Of Experimental Psychology, 32(1), 3–25. http://doi.org/10.1080/00335558008248231
AC
Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, 25–42. http://doi.org/10.1146/annurev.ne.13.030190.000325 Ricciardelli, P., Baylis, G., & Driver, J. (2000). The positive and negative of human expertise in gaze perception. Cognition, 77(1), B1–B14. http://doi.org/10.1016/S0010-0277(00)00092-5 Rizzolatti, G., Riggio, L., Dascola, I., & Umiltà, C. (1987). Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention. Neuropsychologia, 25(1), 31–40. http://doi.org/10.1016/0028-3932(87)90041-8 Rizzolatti, G., Riggio, L., & Sheliga, B. M. (1994). Space and selective attention. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV: Conscious and Nonconscious Information Processing (Vol. 3 Suppl, pp. 232–265). Boston: MIT Press. Rossano, F., Brown, P., & Levinson, S. C. (2009). Gaze, questioning, and culture. In J. Sidnell (Ed.), Conversation Analysis: Comparative Perspectives (pp. 187–249). New York: Cambridge University Press. Saxe, R. (2006). Uniquely human social cognition. Current Opinion in Neurobiology, 16(2), 235–239. http://doi.org/10.1016/j.conb.2006.03.001 Schall, J. D. (2004). On the role of frontal eye field in guiding attention and saccades. Vision
ACCEPTED MANUSCRIPT Research, 44(12), 1453–1467. http://doi.org/10.1016/j.visres.2003.10.025 Senju, A., & Hasegawa, T. (2005). Direct gaze captures visuospatial attention. Visual Cognition, 12(1), 127–144. http://doi.org/10.1080/13506280444000157
RI P
T
Senju, A., Hasegawa, T., & Tojo, Y. (2005). Does perceived direct gaze boost detection in adults and children with and without autism? The stare-in-the-crowd effect revisited. Visual Cognition, 12(8), 1474–1496. http://doi.org/10.1080/13506280444000797 Senju, A., & Johnson, M. H. (2009). The eye contact effect: Mechanisms and development. Trends in Cognitive Sciences, 13(3), 127–134. http://doi.org/10.1016/j.tics.2008.11.009
NU
SC
Smith, D. T., & Schenk, T. (2012). The premotor theory of attention: Time to move on? Neuropsychologia, 50(6), 1104–1114. http://doi.org/10.1016/j.neuropsychologia.2012.01.025
MA
Smith, D. T., Schenk, T., & Rorden, C. (2012). Saccade preparation is required for exogenous attention but not endogenous attention or IOR. Journal of Experimental Psychology: Human Perception & Performance, 38(6), 1438–1447. http://doi.org/http://dx.doi.org/10.1037/a0027794
ED
Spence, C., & Driver, J. (1997). Audiovisual links in exogenous covert spatial orienting. Perception & Psychophysics, 59(1), 1–22. http://doi.org/10.3758/BF03206843 Tanaka, J. W., & Sung, A. (2013). The “eye avoidance” hypothesis of autism face processing. Journal of Autism and Developmental Disorders. http://doi.org/10.1007/s10803-013-1976-7
CE
PT
Todd, R. M., Cunningham, W. A., Anderson, A. K., & Thompson, E. (2012). Affect-biased attention as emotion regulation. Trends in Cognitive Sciences, 16(7), 365–372. http://doi.org/10.1016/j.tics.2012.06.003
AC
Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005). Understanding and sharing intentions: The origins of cultural cognition. The Behavioral and Brain Sciences, 28(5), 675–691. http://doi.org/10.1017/S0140525X05000129 von Grünau, M., & Anston, C. (1995). The detection of gaze direction: A stare-in-the-crowd effect. Perception, 24(11), 1297–1313. http://doi.org/10.1068/p241297 Wieser, M. J., McTeague, L. M., & Keil, A. (2011). Sustained preferential processing of social threat cues: bias without competition? Journal of Cognitive Neuroscience, 23(8), 1973– 1986. http://doi.org/10.1162/jocn.2010.21566 Wu, D. W.-L., Bischof, W. F., & Kingstone, A. (2014). Natural gaze signaling in a social context. Evolution and Human Behavior, 35(3), 211–218. http://doi.org/10.1016/j.evolhumbehav.2014.01.005 Zuckerman, M., Miserandino, M., & Bernieri, F. (1983). Civil inattention exists - in elevators. Personality and Social Psychology Bulletin, 9(4), 578–586. http://doi.org/http://dx.doi.org/10.1177/0146167283094007
ACCEPTED MANUSCRIPT Figure Captions Fig 1. Example camera angles from the outside (A) and indoor (B) locations. Confederate is grey
SC
RI P
T
shaded figure, approaching pedestrians are represented as white shaded figures.
Fig 2. Illustrative confederate actions. Final hand positions for the conditions in which the
NU
confederate raised his hand to answer a phone (top) or wave (bottom), with eyes positioned
MA
either straight ahead (left) or tracking the passing pedestrians (right).
ED
Fig 3. Frequency of pedestrian looks made to the confederate as a function of confederate hand
PT
actions, compared against a no action condition. Only a confederate wave elicited greater
AC
CE
looking than when no action was performed.
AC
CE
PT
ED
MA
NU
SC
RI P
T
ACCEPTED MANUSCRIPT
Fig. 1
CE AC
Fig. 2
PT
ED
MA
NU
SC
RI P
T
ACCEPTED MANUSCRIPT
MA
NU
SC
RI P
T
ACCEPTED MANUSCRIPT
AC
CE
PT
ED
Fig. 3
S1090-5138(16)30045-9 doi: 10.1016/j.evolhumbehav.2016.04.004 ENS 6047
To appear in:
Evolution and Human Behavior
Received date: Revised date: Accepted date:
27 October 2015 6 March 2016 7 April 2016
Please cite this article as: Laidlaw, K.E.W., Rothwell, A. & Kingstone, A., Camouflaged attention: Covert attention is critical to social communication in natural settings, Evolution and Human Behavior (2016), doi: 10.1016/j.evolhumbehav.2016.04.004
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
SC
RI P
T
ACCEPTED MANUSCRIPT
NU
Camouflaged attention: Covert attention is critical to social communication in natural
MA
settings
Kaitlin E.W. Laidlaw
ED
Austin Rothwell
AC
CE
PT
Alan Kingstone
Word count: 5738 including all but references Department of Psychology University of British Columbia 2136 West Mall Vancouver, British Columbia, Canada, V6T 1Z4 Corresponding Author: Kaitlin E.W. Laidlaw [email protected] (604) 822-0069
ACCEPTED MANUSCRIPT Abstract The evolution of the human eye's unique high contrast morphology allows people to
RI P
T
communicate with a simple look. Yet overt looking is not always preferred in social situations. Do covert shifts in attention – those that occur without a concomitant shift of the eyes or head –
SC
support this need? In the present field study, we discretely recorded pedestrians' looks to a confederate who performed an action - raising his hand to the side of his head and saying „Hey‟
NU
into a phone (private action), or raising his hand to the side of his head in greeting (i.e. a static
MA
wave) and saying „Hey‟ (public action). Critically, pedestrians were not looking at the confederate at the start of the action. Despite this, pedestrians looked more in response to the
ED
public action (wave) than private action (phone). We argue that the observed difference in looking responses must be due to pedestrians first attending to the confederate covertly in order
PT
to assess the intention of his action, and only signaling this attention with a look when socially
CE
appropriate (e.g. to respond to a public action). Though the functional utility of covert attention has rarely been considered outside of controlled laboratory tasks, these results provide the first
behaviour.
AC
demonstration that covert attention plays a critical role in guiding appropriate social looking
Keywords: covert attention, eye movements, social attention, social interaction, signaling & nonverbal communication
ACCEPTED MANUSCRIPT 1. Introduction The morphology of the human eye is unique in that the dark of the iris is surrounded by
RI P
T
the high-contrast of the white sclera. This is divergent from nonhuman primate eyes, whose darker sclera blends in with their iris, thereby camouflaging the direction of their gaze from
SC
predators or competitors (Kobayashi & Kohshima, 1997, 2001b; Perrett & Mistlin, 1990). Put simply, the evolution of the high-contrast human eye has facilitated communication by making
NU
eye position easier to discriminate (Ando, 2002; Ricciardelli, Baylis, & Driver, 2000), and
MA
evidence shows that people use their eyes as a powerful signal to communicate both a desire to initiate interactions (Cary, 1978b), as well as to facilitate ongoing interactions (Emery, 2000;
ED
Kleinke, Staneski, & Berger, 1975; Perrett & Emery, 1994; Saxe, 2006; Tomasello, Carpenter, Call, Behne, & Moll, 2005). In so doing, the heightened visibility of the human eye resulted in
PT
the sacrifice of gaze camouflage (Kobayashi & Kohshima, 2001a, 2001b).
CE
Though it is clear that eye movements are used to communicate to others within social
AC
scenarios, did this actually come at the cost of camouflage? While non-human primates rely on the low-contrast of their eyes to disguise their focus, rather than losing this ability altogether, humans may utilize an alternate method of discretely processing visual information. In fact, we take as a working assumption that effective communication with the eyes may actually require that humans first have some way to discretely assess their immediate social surroundings in order to determine whether or not subsequent signaling with the eyes would be necessary or appropriate. We propose that the covert attentional system is the ideal candidate for discretely processing social information in that it can provide observers with the attentional camouflage that was lost with the evolution of the high-contrast human eye.
ACCEPTED MANUSCRIPT Covert visual attention – that is, attending without a related eye movement – serves to bias perceptual and neural processes of salient or behaviorally relevant stimuli (e.g., see Moore,
T
Armstrong, & Fallah, 2003; Posner & Petersen, 1990). Covert attentional mechanisms are found
RI P
in human and nonhuman primates alike, and function similarly to improve visual acuity at attended as compared to unattended objects (Golla, Ignashchenkova, Haarmeier, & Thier, 2004).
SC
Traditionally, the function of covert attention has been considered to be relatively trivial, likely
NU
because it is often thought that under normal circumstances, people look to where they are attending. Indeed, it was once believed that covert attention was merely a byproduct of
MA
oculomotor planning (Rizzolatti, Riggio, Dascola, & Umiltà, 1987; Rizzolatti, Riggio, & Sheliga, 1994), with brain regions involved in overt attention, such as in fronto-parietal areas, also
ED
showing activation in covert attentional tasks (Beauchamp, Petit, Ellmore, Ingeholm, & Haxby,
PT
2001; Corbetta et al., 1998; de Haan, Morgan, & Rorden, 2008). While it has since been demonstrated that covert attention can be at least partially dissociated from oculomotor planning
CE
(Belopolsky & Theeuwes, 2012; Casarotti, Lisi, Umiltà, & Zorzi, 2012; Schall, 2004; Smith,
AC
Schenk, & Rorden, 2012; Smith & Schenk, 2012), its natural utility has not been seriously entertained. In recent years, the idea that human covert attention might serve a social function has been hinted at, for instance as a way of monitoring an aggressive other (Belopolsky & Theeuwes, 2012), or to hide one's own intentions from others (Klein, Shepherd, & Platt, 2009). However, to our knowledge, it has never been demonstrated that people use covert attention in order to monitor others in social settings. In the present study, we test the hypothesis that people covertly attend to others within social environments in order to facilitate appropriate overt looking behaviour. We did this by recording via hidden camera the responses of pedestrians to a simple action made by a naïve
ACCEPTED MANUSCRIPT confederate. Of interest were the responses from pedestrians who were not initially looking at the confederate, i.e. were not overtly attending to the confederate. As pedestrians approached, the
T
confederate either did nothing, or raised his hand to the side of his head and said 'Hey'. This
RI P
action could end in one of two ways. In the public action condition, the confederate held his empty hand up in a static wave position, as if to greet the oncoming pedestrian. Provided that
SC
pedestrians were attending to the confederate, it was anticipated that this public, social action
NU
would elicit a social response from the oncoming pedestrian: that is, they would acknowledge the greeting by (at minimum) looking at the confederate. In contrast, in the private action condition,
MA
the confederate performed the same action while holding a smart phone, thereby raising his hand to his ear and seemingly delivering the greeting to the phone rather than to the pedestrians. In
ED
this instance, it was anticipated that although the greeting and action may serve to shift their
PT
covert attention toward the confederate, they would avoid signalling their attentiveness in an effort to reduce the potential of engaging with the confederate (e.g. Wu, Bischof, & Kingstone,
CE
2014; Zuckerman, Miserandino, & Bernieri, 1983).
AC
In other words, it was predicted that looking rates would differ based on pedestrians' initial evaluation of the confederate's intention underlying his action. Without already looking at the confederate, covert attention would be the only viable mechanism for pedestrians to use in order to make this subtle peripheral discrimination (i.e. same hand raise with or without a phone). The no action condition served as a baseline by which to compare looking rates. Though we reasoned that the confederate's own looking behaviour could not be discriminated by the pedestrians using their peripheral vision at the viewing distances used in the present study (e.g., Loomis, Kelly, Pusch, Bailenson, & Beall, 2008), it was prudent to test this as laboratory tasks have often shown strong attentional effects of another‟s gaze (Frischen, Bayliss, & Tipper, 2007; Nummenmaa &
ACCEPTED MANUSCRIPT Calder, 2009; Senju & Johnson, 2009). Thus, either hand action was accompanied by the confederate looking at the pedestrian or staring straight ahead.
RI P
T
Actions were executed in a manner that minimized any differences across conditions except for the addition of a cell phone in the private action condition. This is not to suggest that all other
SC
changes were eliminated, as this is an unrealistic expectation for a naturalistic study such as this. In fact, small fluctuations in environmental variables such as crowding, lighting, or confederate
NU
posturing are better considered as a strength of field research: if changes in the signal –
MA
pedestrian looking behaviour – can be reliably measured despite the added 'noise' characteristic of naturalistic environments, then it supports the view that this behaviour is robust and not
ED
simply an artifact of a particular testing scenario. In sum, despite pedestrians not looking at the confederate at the start of the action and despite the minor changes present in each participant's
PT
immediate environment, we hypothesize that they will covertly attend to the confederate's action
CE
and, critically, tailor their response to his action such that they will look up at the confederate
2. Methods
AC
more frequently when his hand is raised to form a wave than to answer the phone.
2.1 Equipment and procedure Pedestrians were sampled from the two locations on the University of British Columbia's Vancouver campus - one indoor and one outdoor (Figure 1). A residence dining hall was used as the indoor location, where a walkway cut between a wall and a row of study rooms, leading to the building exit. The outdoor location was a path, bordered by garden on either side, which passed in between two buildings. Importantly, both locations had a linear flow of pedestrian traffic (i.e., where pedestrians could only walk straight, in two opposing directions) and
ACCEPTED MANUSCRIPT oncoming pedestrians were easily visible. In both locations, the confederate was situated in the flow of pedestrian traffic without obstructing pedestrian movement.
RI P
T
[FIGURE 1 ABOUT HERE]
The confederate was a male Caucasian undergraduate (20 yrs, 5'9") from the University
SC
of British Columbia, and was dressed in neutral clothing and wore nothing to obstruct his eyes
NU
(e.g., a hat, glasses, avoided squinting). The specific hypotheses related to pedestrian looking behavior were not disclosed to the confederate until after testing. The confederate stood casually,
MA
using a black smart phone down by his stomach/waist, and occasionally looked straight ahead to search for distant pedestrians that roughly met selection criteria (see below). When the
ED
confederate observed that a pedestrian met basic selection criteria, the confederate would re-
PT
engage with his phone at waist level until the pedestrian was within a pre-designated distance agreed upon prior to the start of the study, at which point he would initiate an action (described
CE
below). Representative distance measurements taken following pedestrian data collection
AC
estimated that the confederate initiated the action an average of 1.52 m (SD = .30 m) in front of the pedestrian. A relatively close distance was chosen for two reasons. First, research suggests that as pedestrians approach a stranger, they will divert their overt attention (both their head and eyes) away from the individual (Fotios, Yang, & Uttley, 2015; Goffman, 1963; Patterson, Webb, & Schwartz, 2002). Second, classic proxemics research suggests that the space around a person can be divided into areas of interpersonal space, with 1.2 m to 3.7 m representing 'social space' in which people are comfortable engaging in conversation and other casual interactions (Hall, 1968). Thus, the confederate aimed to initiate the action within a distance that is not only considered social space (i.e. in which a greeting might be made to another), but that overt attention is nevertheless diverted away from non-familiar others.
ACCEPTED MANUSCRIPT The action performed by the confederate was a greeting that involved hand, eye and verbal components, the specifics of which were as closely matched as possible so as to avoid any
T
differences in large motion signals across conditions. For the hand action, the confederate raised
RI P
his empty right hand near his ear as if to wave (i.e. palm forward), or raised the same hand and brought his cell phone to his ear (Figure 2). In all conditions, the confederate engaged with the
SC
cell phone prior to the start of the action. As such, the presence of the cell phone could not have
NU
served as a cue to the confederate‟s upcoming action. The arm motion in both cases mimicked the typical action of answering a phone, and the final position (wave or answering phone) was
MA
held static until the pedestrian passed (i.e. the wave condition did not include any additional movement of the hand). A forward facing head position was maintained throughout. In addition,
ED
the confederate either looked straight ahead (i.e. avoided eye contact), or fixated the pedestrian at
PT
the start of the action, and followed the pedestrian with his eyes until they approached too near to continue to do so. In all conditions, the confederate said „Hey‟ in a consistent and friendly tone
CE
as his hand action ended. Once the pedestrian passed, the confederate ended the action by
AC
withdrawing his hand and returning to using the phone near his waist. The confederate was instructed to avoid any additional interaction with the passersby, such as smiling or nodding, though in rare instances where this happened (e.g. the pedestrian stopped to ask a question), the pedestrian was removed from the analysis. In addition, a 'no action' baseline condition was culled from pre-recorded video sessions. Pedestrians were selected who met the selection criteria but passed the confederate while he was not performing any action. Thus, confederate's behaviour in these clips depicted natural waiting behaviour, and prominently featured the actor looking down at his phone, while occasionally looking up and away in the distance (i.e. not at nearby pedestrians).
ACCEPTED MANUSCRIPT [FIGURE 2 ABOUT HERE] Events were recorded using a high-definition (1080p) video camera placed so that it was
RI P
T
partially obstructed from the participant‟s view (i.e. stored in a backpack, with an opening for the lens). The camera was placed behind (outdoors: 15.21 m; indoors: 6.67 m) the confederate and to
SC
his left (outdoors: 6.70 m, indoors: 2.92 m), at a similar angle to the confederate (outdoors: 23.73˚; indoors: 23.66˚). An experimenter sat beside the hidden camera and recorded participant
NU
descriptives (e.g. sex, visual description) and which condition was run. Differences in camera-to-
MA
confederate distance were necessary to discretely position the camera and experimenter. Though locations differed in the distance of the camera from the confederate due to environmental
ED
constraints, confederate size was approximately equated through the use of camera zoom.
PT
2.2 Pedestrian sampling
CE
Data was collected over 12 sessions (7 Outside, 5 Inside). During each testing session, all conditions were run, in a pseudo-random order, documenting roughly ten pedestrians
AC
consecutively for each condition before moving to the next condition. Efforts were made to select only pedestrians who were walking alone, whose eyes were visible, and who were not wearing earphones or actively using a portable electronic device. In addition, the confederate delayed repeating an action immediately following a sampled pedestrian so as to avoid another pedestrian observing the behaviour from a distance. As participation in the study involved only observation in public space, no consent from selected pedestrians was required and no debriefing was provided. The study was approved by the University of British Columbia's Research Ethics Board. 3. Results
ACCEPTED MANUSCRIPT 3.1 Data handling The video recording was cut into short clips commencing with the participant entering
RI P
T
the frame and ending when the participant exited the frame. Any instance in which the pedestrian was looking at the confederate prior to his action were discarded, as were instances in which the
SC
confederate's action started before the participant entered the frame, as coders would be unable to ensure that participants were not overtly attending to the confederate just prior to the start of the
NU
action. Instances in which pedestrians were blocked during critical moments were also discarded,
MA
as were any instances caught with repeat pedestrians. In total, 483 videos were recorded that contained sufficient information to allow for subsequent coding. Two coders who were naïve to
ED
the hypotheses of the study coded all videos for pedestrian looking behaviour, timing, and environment details1. Coding was performed using software that enabled frame-by-frame
PT
viewing. As multiple looks were relatively infrequent, looking behaviour was coded in a binary
CE
fashion: following the confederate action, it was coded whether or not the pedestrian looked at the confederate. Looks were counted even if the pedestrian passed the confederate and turned
AC
back to look at him, though these instances were rare. Videos were excluded if at least one coder noted that the pedestrian or video did not meet criteria stated above, if gaze was indiscernible (e.g. by shadow), or if at least one coder noted that the pedestrian was looking at the confederate immediately prior to the start of the action (total: 100 videos, 20.70%). 1
Pedestrian ethnicity was also coded with good reliability (Krippendorff's Alpha = 0.83). The majority of pedestrians were either rated as Caucasian (42.64% of sample based on Coder 1's ratings) or East Asian (40.24% of the sample). Limiting analyses to only these two ethnic groups shows a small effect of ethnicity on pedestrian looking frequency, with East Asian pedestrians looking less than Caucasian, p < .05 (see Patterson et al., 2007 for a similar pattern). As this did not interact with our critical manipulations, ethnicity was not included as a factor in the reported analyses. Reliability was also good for pedestrian age group (Krippendorff's Alpha = 0.88) and environmental crowding (rated on a 10point scale, Krippendorff's Alpha = .77). Due to a relative lack of variability in these factors, however, they were excluded from analysis.
ACCEPTED MANUSCRIPT Coder reliability was assessed using Krippendorff's alpha and was good for all measures (Table 1). Coder reliability did not vary substantially based on location (e.g. Krippendorff‟s
T
Alpha for either location was never divergent from the reported value by more than .02),
RI P
suggesting that the use of camera zoom did not noticeably influence the coder‟s ability to detect pedestrian looks. If coders disagreed on whether the pedestrian looked, the pedestrian was
SC
excluded from analysis (n = 50), thus conservatively limiting analyses only to instances where
NU
both coders agreed on whether the pedestrian looked or not. Analyses were therefore based on
MA
333 pedestrians. Table 1. Coder reliability
ED
Krippendorff's Alpha 0.73 0.82 0.95
CE
PT
Variable Looks to confederate following action Time to pass confederate (from start of action) Time to exit frame following action initiation
AC
3.2 Pedestrian and video descriptives Of the 333 pedestrians, 168 (50.45%) were female and 172 (51.65%) were recorded from the outdoor location. The majority of the pedestrians sampled were judged to be between 16-29 (88.59%), with all other pedestrians being judged 30 or older (11.41%). The confederate raised his hand to wave for 136 (40.84%) of pedestrians. Of those, the confederate looked straight ahead for 63 (46.32%) cases while he looked directly at 73 (53.68%) of the passing pedestrians, The confederate raised his hand to answer his phone for 115 (34.53%) pedestrians, for which he
ACCEPTED MANUSCRIPT looked straight ahead for 57 (49.57%) cases or at the pedestrian for 58 (50.43%) cases. There were 82 pedestrians who passed the confederate while he performed no action (24.62%).
RI P
T
Average clip duration was 9.54 s (SD = 3.36 s). Based on the results from either coder, Time to pass the confederate (from the start of the action) and Total time to react (from start of
SC
action to exiting frame) did not vary as a function of confederate action or looking behaviour, all
NU
ps > .05.
MA
3.3 Did confederate actions influence overt looking frequency? The primary question was whether pedestrians would change looking behaviour in
ED
response to the confederate's action. As such, analyses were first conducted on only the 'Wave' and 'Phone' conditions. Owing to the dichotomous nature of the dependent variable, looking
PT
behaviour was analyzed using a loglinear analysis with location (outside, inside), confederate
CE
hand action (wave, phone), and confederate eye position action (looking straight ahead or at the pedestrian) as factors. Loglinear analysis can be thought of as an extension of Pearson's chi-
AC
square analysis when examining the relationship between more than two categorical variables. In a loglinear analysis, models are built and tested to find the least complex model (i.e. with the fewest factors) that best accounts for the variance in the observed frequencies. The only major assumptions of the analysis are that the observations are independent, and that the expected frequencies for each cell of the contingency table (i.e. each condition) should be large enough (greater than 5) to permit a reliable analysis. A stepwise backwards elimination procedure was used to determine which of the factors and their interactions significantly reduced model fit. In this way, a fully saturated model is first tested, and then interactions and factors are eliminated in a stepwise fashion (i.e. the highest order interaction effect is removed first, then if this does not
ACCEPTED MANUSCRIPT have a significant effect on model fit, the second highest-order effect is removed, etc) until such a point where eliminating a factor reduces the model fit significantly. Performing this analysis
T
revealed that only removing the interaction between of confederate hand action and pedestrian
RI P
looking behaviour had a significant effect on the model, χ2(1) = 38.87, p < .001. Pedestrians being more likely (odds ratio: 5.20) to look at the confederate following a wave than when he
SC
lifted his phone and said 'Hey'. Removing any other factors or their interactions had no
NU
significant effect, indicating that location and confederate looking behaviour (or their interactions) did not impact pedestrian looking responses. The final model with only pedestrian
MA
looking behaviour and confederate action resulted in a non-significant likelihood ratio of χ2(12) = 9.91, p = .62, which indicates that the model including only this interaction represents a good
ED
fit of the data2.
PT
3.4 How do overt looking frequencies compare against baseline?
CE
Though it is clear that overt looking rates change as a function of confederate action, it is
AC
unclear from the above data what 'normal' looking rates would be, and how those rates would relate to that observed when passing a confederate making an action. Of particular interest was how looks in response to the confederate raising his phone compared to baseline looking behaviour from the pedestrians. A chi-square analysis of looking behaviour with confederate
2
The confederate was blind to the specific hypotheses of the study until after data collection, though it may be argued that his performance of the arm action may have varied across conditions, thereby unduly attracting attention in one condition over another. To test this, we randomly selected a subset of trials from the Wave and Phone conditions (20 each; collapsing over Gaze & Location conditions), and determined the overall movement time from start of hand raise to final hand position. As peripheral vision is especially adept at detecting motion cues, this seemed the most likely candidate behaviour that may have inadvertently influenced looking responses, e.g., faster movement times would stand out more than if the same action were performed more slowly. However, analysis revealed no significant differences in motion time, t(38) = 0.83, p = .41.
ACCEPTED MANUSCRIPT action (wave, phone, no action) as a factor revealed a significant association, χ2(2) = 63.98, p < .001. To explore this further, the original 3x2 chi-square was partitioned to reveal that looking
T
behaviour was more likely (odds-ratio: 6.49) for the wave condition than in the phone and no-
RI P
action conditions combined, χ2(1) = 61.85, p < .001. Interestingly, looking frequencies in phone and no action conditions were comparable, χ2(1) = 2.81, p = .09 (odds-ratio: 1.80). Table 2
SC
reports the values for each condition; Figure 3 graphically presents the main effect of confederate
NU
action on looking responses.
PT
ED
Looked at pedestrian 17 41 58 29.31
Wave Looked straight ahead 40 23 63 63.49
Looked at pedestrian
No Action Looked down
52 21 73 71.23
15 67 82 18.29
AC
CE
Pedestrian looked Pedestrian did not look N Looking frequency (%)
Phone Looked straight ahead 16 41 57 28.07
MA
Table 2. Pedestrian looking frequency across confederate eye position and action conditions
[FIGURE 3 ABOUT HERE] 4. Discussion
In social environments, there may be no more important source of information to focus on than the actions of other people nearby, as they can dynamically alter one's own behaviour. Yet, monitoring the intentions of others by looking at them is not always appropriate or welcome. As such, it was hypothesized that humans have come to rely on covert attentional deployment to monitor nearby others, thereby enabling them to process other's actions while limiting their overt looking responses. To test this, the present study assessed whether a confederate's action – raising his hand to either wave or answer his phone – would elicit different
ACCEPTED MANUSCRIPT looking responses in passing pedestrians. Confederate eye gaze and testing locations were also varied. Though pedestrians were not overtly attending at the time that the action started, looking
T
responses to the confederate varied significantly based on the confederate's action: pedestrians
RI P
were five times more likely to look in response to a wave than to answering a phone. Confederate looking behaviour and testing location did not significantly influence responses. The
SC
difference in pedestrian looking response rates across action conditions required that pedestrians
NU
first covertly process the confederate's action in order to tailor their own response appropriately, thereby demonstrating for the first time that people use their covert attention as a way of
MA
observing and assessing other's intentions without overtly signaling this to others nearby. Through the use of covert attention, pedestrians in the present study were able to
ED
modulate their looking behaviour in order to signal attention in instances when it was socially
PT
appropriate. Looking in response to a wave may have served various, not independent, purposes, such as gaining more information about the other person's intentions (e.g. 'Is he waving at me?)
CE
or acknowledging the greeting by trying to establish mutual gaze. In support of the latter view,
AC
signaling one's acknowledgment of the other's greeting, or reciprocal recognition, is considered a standard social convention (Duranti, 1997), and serves to reinforce positive social relationships (Goffman, 1971), even in nonhuman primates (De Marco, Sanna, Cozzolino, & Thierry, 2014), and thus may be initiated even when pedestrians were uncertain that they were the target of the greeting. Though the exact intentionality of the look to the confederate is ambiguous, the critical point is that the pedestrians permitted themselves to react overtly only in the condition where the confederate's action could be interpreted as an invitation to interact. When the confederate's action was covertly determined to be a potential communicative signal, pedestrians responded in turn by using their looking behaviour as a communicative tool.
ACCEPTED MANUSCRIPT Although there is substantial literature detailing that motion (e.g., Abrams & Christ, 2003; Franconeri & Simons, 2003) and auditory stimuli (e.g., McDonald, Teder-Sälejärvi, &
T
Hillyard, 2000; Spence & Driver, 1997) attract visual attention, it is clear that in the present
RI P
study, looking behaviour was not simply an overt exogenous reaction to the act of raising the hand or hearing a verbal greeting. If that had been the case, pedestrians should have looked
SC
equally often in both wave and phone conditions, which both involved large arm motions and
NU
auditory cues, and least often when the confederate did nothing. However, it was only in the wave condition that pedestrians often looked towards the confederate; the phone condition,
MA
which was physically most similar to the wave condition, did not elicit pedestrian responses to nearly the same extent. Instead, looks were infrequent both when the confederate answered the
ED
phone and when he did nothing, despite these conditions varying dramatically in their
PT
involvement of attentional cues. It is extremely unlikely that the significant difference in looking behaviour between phone and wave conditions could be attributed to a „stronger‟ exogenous pull
CE
of one arm raise over the other, even if the final end position differed slightly; if the motion cue
AC
was insufficient to exogenously attract attention, so too would have been a small change in hand position between answering a phone and holding the hand near the face to „wave‟. However, the lack of looks in the phone condition should not be taken as evidence that pedestrians were not attending, as several recent findings suggest that people avoid looking at strangers when they believe their actions might be visible (and interpretable) by the other individual. For example, Gallup and colleagues (2012) have shown that pedestrians are more likely to follow another‟s gaze when walking behind or beside a group of lookers, as compared to when they walk in front (see also Gallup, Chong, & Couzin, 2012). Not only are people aware of the ability of their gaze to signal information, when in the presence of strangers they tend to
ACCEPTED MANUSCRIPT assume a gaze avoidance strategy. As such, the infrequency of looks to the confederate when he answered his phone is better described as pedestrians avoiding subsequent looking behaviour
T
following covert examination of the confederate. This inhibition was accomplished effectively,
RI P
so much so that looking rates in response to the confederate answering his phone did not
SC
significantly differ from when he performed no action at all.
As anticipated, looking behaviour in the present study was unaffected by whether the
NU
confederate stared straight ahead or if he focused on the pedestrian while they passed. While we
MA
will limit drawing many conclusions based on a non-significant effect, it bears mention in so much as it contrasts with many laboratory tasks suggesting that people are adept at detecting eye
ED
gaze directed towards them (Itier, Villate, & Ryan, 2007; Senju, Hasegawa, & Tojo, 2005; Senju & Hasegawa, 2005; von Grünau & Anston, 1995). As the confederate maintained head position
PT
across conditions, we suspect, as others have argued, that the visual angle subtended by the eyes
CE
was too small, and acuity too poor in peripheral vision to reliably detect changes in gaze direction (Loomis et al., 2008). Future investigations of the effects of eye gaze on attentional
AC
orienting may provide insight into whether this social cue is useful within different naturalistic conditions (e.g. when already attending near another‟s face). Not only do the present findings draw attention to the need for future research into the social influences on visual behaviour, they also lay the groundwork for better understanding of how covert attention is directed to people in everyday scenarios. For example, did pedestrians choose to covertly attend to the confederate (perhaps when passing or because of the initiation of an action), or was the confederate already prioritized within the attentional system owing to him being a social stimulus? There is little field work that speaks to this point, though laboratory based tasks would argue that people have an at least partially non-volitional attentional bias
ACCEPTED MANUSCRIPT towards social stimuli (Birmingham, Bischof, & Kingstone, 2009; Devue, Belopolsky, & Theeuwes, 2012; Laidlaw, Badiudeen, Zhu, & Kingstone, 2015; Laidlaw, Risko, & Kingstone,
T
2012). Several researchers have argued that attention may be proactively biased by highly
RI P
relevant or highly valuable stimuli (Anderson, Laurent, & Yantis, 2011; Anderson, 2013; Todd, Cunningham, Anderson, & Thompson, 2012; Wieser, McTeague, & Keil, 2011), and one could
SC
easily argue that nearby others reflect a perfect example of such a stimulus. What remains to be
NU
seen is whether these same automated biases are present within naturalistic environments, or if instead people strategically shift their covert attention to other people when convenient or
MA
necessary.
ED
Though it has long been known that attentional and oculomotor systems can operate independently (Eriksen & Yeh, 1985; Juan, Shorter-Jacobi, & Schall, 2004; Posner, 1980), there
PT
has been little emphasis placed on the role that covert attention plays within social environments.
CE
This omission is likely due in part to the manner in which social attention has so often been studied: using images of people who cannot look back. In real life, looking to another signals
AC
information to that person, who themselves can react, creating a cascade of responses from both parties. The awareness that one's looking behaviour can be interpreted by others in real life constrains the way in which people direct their overt attention around others (Cary, 1978b; Gallup, Chong, et al., 2012; Gallup, Hale, et al., 2012; Gobel, Kim, & Richardson, 2015; Laidlaw, Foulsham, Kuhn, & Kingstone, 2011; Wu et al., 2014), thereby creating a need for another mechanism – covert attention – to process important social information. In contrast, there are no social consequences of looking at images. In instances where looking serves no communicative role, as is the case in many traditional laboratory tasks of social attention, it may be that the decoupling of covert from overt attention is largely unnecessary, and as such, the
ACCEPTED MANUSCRIPT utility of covert social attention has gone largely unstudied. It is clear that in order to make generalized claims of the vital role of covert attention within naturalistic social settings, the
T
present results will need to be replicated and extended to new situations. Further, the present
RI P
results should not be taken to imply that the sole use of covert attention is socially-based, or that nonhuman primates do not have the ability to covertly attend. This should not diminish the
SC
impact of the current findings, however, as they serve as the first proof of concept: within a
NU
common social situation unaffected by unrealistic experimental manipulation, people direct covert attention to others in order to scaffold their subsequent looking behavior. As such, the
MA
present study represents an initial step toward a new line of research focused on how covert social attention operates in a naturalistic environment and guides overt looking responses within
ED
social environments.
PT
Finally, the present findings also highlight the importance not only of looking to
CE
communicate with others, but also the subtle but meaningful way in which people use the absence of looks as a communicative signal onto itself. While the value placed in averted gaze
AC
from other people varies depending on the culture, the appropriate use and withdrawal of looks to another emerges as an important social norm within many groups (Argyle & Cook, 1976; McCarthy, Lee, Itakura, & Muir, 2006, 2008; Rossano, Brown, & Levinson, 2009), and violations of these norms may be interpreted to signal developmental or neurological impairments (Moukheiber et al., 2010; Tanaka & Sung, 2013). As was observed in the present study, not looking overwhelmingly communicates a lack of attentiveness or signals an unwillingness to interact further; the use of gaze in this way is becoming a frequently observed social phenomenon within naturalistic environments (e.g. Cary, 1978a; Foulsham, Walker, &
ACCEPTED MANUSCRIPT Kingstone, 2011; Gallup, Chong, et al., 2012; Gallup, Hale, et al., 2012; Goffman, 1963; Laidlaw et al., 2011).
RI P
T
Acknowledgements & Funding
The authors thank Jalila Devji and Tara Luk for their work coding the videos. Funding for this
SC
project was provided by a Natural Sciences and Engineering Research Council of Canada
NU
Discovery Grant and a Social Sciences and Humanities Research Council of Canada Insight
AC
CE
PT
ED
MA
Grant.
ACCEPTED MANUSCRIPT References
T
Abrams, R. A., & Christ, S. E. (2003). Motion onset captures attention. Psychological Science, 14(5), 427–432. http://doi.org/10.1111/1467-9280.01458
RI P
Anderson, B. A. (2013). A value-driven mechanism of attentional selection. Journal of Vision, 13(3), 1–16. http://doi.org/10.1167/13.3.7.doi
SC
Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences of the U.S.A., 108(25), 10367–10371. http://doi.org/10.1073/pnas.1104047108
NU
Ando, S. (2002). Luminance-induced shift in the apparent direction of gaze. Perception, 31(6), 657–674. http://doi.org/10.1167/1.3.287
MA
Argyle, M., & Cook, M. (1976). Gaze and mutual gaze. Oxford, England, England: Cambridge University Press. Beauchamp, M. S., Petit, L., Ellmore, T. M., Ingeholm, J., & Haxby, J. V. (2001). A parametric fMRI study of overt and covert shifts of visuospatial attention. NeuroImage, 14, 310–321. http://doi.org/10.1006/nimg.2001.0788
PT
ED
Belopolsky, A. V, & Theeuwes, J. (2012). Updating the premotor theory: The allocation of attention is not always accompanied by saccade preparation. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 902–914. http://doi.org/10.1037/a0028662
CE
Birmingham, E., Bischof, W. F., & Kingstone, A. (2009). Saliency does not account for fixations to eyes within social scenes. Vision Research, 49(24), 2992–3000. http://doi.org/10.1016/j.visres.2009.09.014
AC
Cary, M. (1978a). Does civil inattention exist in pedestrian passing? Journal of Personality and Social Psychology, 36(11), 1185–1193. http://doi.org/10.1037/0022-3514.36.11.1185 Cary, M. (1978b). The role of gaze in the initiation of conversation. Social Psychology, 41(3), 269–271. http://doi.org/10.2307/3033565 Casarotti, M., Lisi, M., Umiltà, C., & Zorzi, M. (2012). Paying attention through eye movements: A computational investigation of the premotor theory of spatial attention. Journal of Cognitive Neuroscience, 24(7), 1519–1531. http://doi.org/10.1162/jocn_a_00231 Corbetta, M., Akbudak, E., Conturo, T. E., Snyder, A. Z., Ollinger, J. M., Drury, H. A., … Shulman, G. L. (1998). A common network of functional areas for attention and eye movements. Neuron, 21, 761–773. http://doi.org/10.1016/S0896-6273(00)80593-0 de Haan, B., Morgan, P. S., & Rorden, C. (2008). Covert orienting of attention and overt eye movements activate identical brain regions. Brain Research, 1204, 102–111. http://doi.org/10.1016/j.brainres.2008.01.105 De Marco, A., Sanna, A., Cozzolino, R., & Thierry, B. (2014). The function of greetings in male Tonkean macaques. American Journal of Primatology, 76(10), 989–998. http://doi.org/10.1002/ajp.22288
ACCEPTED MANUSCRIPT Devue, C., Belopolsky, A. V, & Theeuwes, J. (2012). Oculomotor guidance and capture by irrelevant faces. PloS One, 7(4), e34598. http://doi.org/10.1371/journal.pone.0034598
T
Duranti, A. (1997). Universal and culture-specific properties of greetings. Linguistic Anthropology, 7(1), 63–97. http://doi.org/10.1525/jlin.1997.7.1.63
RI P
Emery, N. J. (2000). The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24(6), 581–604. http://doi.org/10.1016/S01497634(00)00025-7
SC
Eriksen, C. W., & Yeh, Y. Y. (1985). Allocation of attention in the visual field. Journal of Experimental Psychology: Human Perception & Performance, 11(5), 583–597.
NU
Fotios, S., Yang, B., & Uttley, J. (2015). Observing other pedestrians: Investigating the typical distance and duration of fixation. Lighting Research and Technology, 47(5), 548–564. http://doi.org/10.1177/1477153514529299
MA
Foulsham, T., Walker, E., & Kingstone, A. (2011). The where, what and when of gaze allocation in the lab and the natural environment. Vision Research, 51(17), 1920–1931. http://doi.org/10.1016/j.visres.2011.07.002
ED
Franconeri, S. L., & Simons, D. J. (2003). Moving and looming stimuli capture attention. Perception & Psychophysics, 65(7), 999–1010. http://doi.org/10.3758/BF03194829
PT
Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007). Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133(4), 694–724. http://doi.org/10.1037/0033-2909.133.4.694
CE
Gallup, A. C., Chong, A., & Couzin, I. D. (2012). The directional flow of visual information transfer between pedestrians. Biology Letters, 8(4), 520–522. http://doi.org/10.1098/rsbl.2012.0160
AC
Gallup, A. C., Hale, J. J., Sumpter, D. J. T., Garnier, S., Kacelnik, A., Krebs, J. R., & Couzin, I. D. (2012). Visual attention and the acquisition of information in human crowds. Proceedings of the National Academy of Sciences of the United States of America, 109(19), 7245–7250. http://doi.org/10.1073/pnas.1116141109 Gobel, M. S., Kim, H. S., & Richardson, D. C. (2015). The dual function of social gaze. Cognition, 136, 359–364. http://doi.org/10.1016/j.cognition.2014.11.040 Goffman, E. (1963). Behavior in public places: Notes on the social organization of gatherings. New York: The Free Press. Goffman, E. (1971). Relations in public: Microstudies of the public order. New York: Basic Books. Golla, H., Ignashchenkova, A., Haarmeier, T., & Thier, P. (2004). Improvement of visual acuity by spatial cueing: A comparative study in human and non-human primates. Vision Research, 44(13), 1589–1600. http://doi.org/10.1016/j.visres.2004.01.009 Hall, E. T. (1968). Proxemics. Current Anthropology, 9(2/3), 83–108. http://doi.org/10.1086/200975 Itier, R. J., Villate, C., & Ryan, J. D. (2007). Eyes always attract attention but gaze orienting is
ACCEPTED MANUSCRIPT task-dependent: Evidence from eye movement monitoring. Neuropsychologia, 45(5), 1019– 1028. http://doi.org/10.1016/j.neuropsychologia.2006.09.004
T
Juan, C.-H., Shorter-Jacobi, S. M., & Schall, J. D. (2004). Dissociation of spatial attention and saccade preparation. Proceedings of the National Academy of Sciences of the United States of America, 101(43), 15541–15544. http://doi.org/10.1073/pnas.0403507101
RI P
Klein, J. T., Shepherd, S. V, & Platt, M. L. (2009). Social attention and the brain. Current Biology, 19(20), R958–R962. http://doi.org/10.1016/j.cub.2009.08.010
SC
Kleinke, C. L., Staneski, R. A., & Berger, D. E. (1975). Evaluation of an interviewer as a function of interviewer gaze, reinforcement of subject gaze, and interviewer attractiveness. Journal of Personality and Social Psychology, 31(1), 115–122. http://doi.org/10.1037/h0076244
NU
Kobayashi, H., & Kohshima, S. (1997). Unique morphology of the human eye. Nature, 387(6635), 767–768. http://doi.org/10.1038/42842
MA
Kobayashi, H., & Kohshima, S. (2001a). Evolution of the human eye as a device for communication. In T. Matsuzawa (Ed.), Primate Origins of Human Cognition and Behavior (pp. 383–401). Japan: Springer.
ED
Kobayashi, H., & Kohshima, S. (2001b). Unique morphology of the human eye and its adaptive meaning: Comparative studies on external morphology of the primate eye. Journal of Human Evolution, 40(5), 419–435. http://doi.org/10.1006/jhev.2001.0468
PT
Laidlaw, K. E. W., Badiudeen, T. A., Zhu, M. J. H., & Kingstone, A. (2015). A fresh look at saccadic trajectories and task irrelevant stimuli: Social relevance matters. Vision Research, 111(Part A), 82–90. http://doi.org/10.1016/j.visres.2015.03.024
AC
CE
Laidlaw, K. E. W., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences of the United States of America, 108(14), 5548–5553. http://doi.org/10.1073/pnas.1017022108 Laidlaw, K. E. W., Risko, E. F., & Kingstone, A. (2012). A new look at social attention: Orienting to the eyes is not (entirely) under volitional control. Journal of Experimental Psychology: Human Perception and Performance, 38(5), 1132–1143. http://doi.org/10.1037/a0027075 Loomis, J. M., Kelly, J. W., Pusch, M., Bailenson, J. N., & Beall, A. C. (2008). Psychophysics of perceiving eye-gaze and head direction with peripheral vision: Implications for the dynamics of eye-gaze behavior. Perception, 37(9), 1443–1457. http://doi.org/10.1068/p5896 McCarthy, A., Lee, K., Itakura, S., & Muir, D. W. (2006). Cultural display rules drive eye gaze during thinking. Journal of Cross-Cultural Psychology, 37(6), 717–722. http://doi.org/10.1177/0022022106292079 McCarthy, A., Lee, K., Itakura, S., & Muir, D. W. (2008). Gaze display when thinking depends on culture and context. Journal of Cross-Cultural Psychology, 39(6), 716–729. http://doi.org/10.1177/0022022108323807 McDonald, J. J., Teder-Sälejärvi, W. a, & Hillyard, S. a. (2000). Involuntary orienting to sound
ACCEPTED MANUSCRIPT improves visual perception. Nature, 407(6806), 906–908. http://doi.org/10.1038/35038085 Moore, T., Armstrong, K. M., & Fallah, M. (2003). Visuomotor origins of covert spatial attention. Neuron, 40(4), 671–683. http://doi.org/10.1016/S0896-6273(03)00716-5
RI P
T
Moukheiber, A., Rautureau, G., Perez-Diaz, F., Soussignan, R., Dubal, S., Jouvent, R., & Pelissolo, A. (2010). Gaze avoidance in social phobia: Objective measure and correlates. Behaviour Research and Therapy, 48(2), 147–151. http://doi.org/10.1016/j.brat.2009.09.012
SC
Nummenmaa, L., & Calder, A. J. (2009). Neural mechanisms of social attention. Trends in Cognitive Sciences, 13(3), 135–143. http://doi.org/10.1016/j.tics.2008.12.006
NU
Patterson, M. L., Iizuka, Y., Tubbs, M. E., Ansel, J., Tsutsumi, M., & Anson, J. (2007). Passing encounters East and West: Comparing Japanese and American pedestrian interactions. Journal of Nonverbal Behavior, 31(3), 155–166. http://doi.org/10.1007/s10919-007-0028-4
MA
Patterson, M. L., Webb, A., & Schwartz, W. (2002). Passing encounters: Patterns of recognition and avoidance in pedestrians. Basic and Applied Social Psychology, 24(1), 57–66. http://doi.org/10.1207/S15324834BASP2401_5
ED
Perrett, D. I., & Emery, N. J. (1994). Understanding the intentions of others from visual signals: Neurophysiological evidence. Current Psychology of Cognition, 13(5), 683–694. Retrieved from http://doi.apa.org/psycinfo/1995-24608-001
PT
Perrett, D. I., & Mistlin, A. J. (1990). Perception of facial characteristics by monkeys. In W. C. Stebbins & M. A. Berkley (Eds.), Comparative perception, Vol. 2: Complex signals. (Vol 2, pp. 187–215). Oxford: John Wiley & Sons.
CE
Posner, M. I. (1980). Orienting of attention. The Quarterly Journal Of Experimental Psychology, 32(1), 3–25. http://doi.org/10.1080/00335558008248231
AC
Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, 25–42. http://doi.org/10.1146/annurev.ne.13.030190.000325 Ricciardelli, P., Baylis, G., & Driver, J. (2000). The positive and negative of human expertise in gaze perception. Cognition, 77(1), B1–B14. http://doi.org/10.1016/S0010-0277(00)00092-5 Rizzolatti, G., Riggio, L., Dascola, I., & Umiltà, C. (1987). Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention. Neuropsychologia, 25(1), 31–40. http://doi.org/10.1016/0028-3932(87)90041-8 Rizzolatti, G., Riggio, L., & Sheliga, B. M. (1994). Space and selective attention. In C. Umiltà & M. Moscovitch (Eds.), Attention and performance XV: Conscious and Nonconscious Information Processing (Vol. 3 Suppl, pp. 232–265). Boston: MIT Press. Rossano, F., Brown, P., & Levinson, S. C. (2009). Gaze, questioning, and culture. In J. Sidnell (Ed.), Conversation Analysis: Comparative Perspectives (pp. 187–249). New York: Cambridge University Press. Saxe, R. (2006). Uniquely human social cognition. Current Opinion in Neurobiology, 16(2), 235–239. http://doi.org/10.1016/j.conb.2006.03.001 Schall, J. D. (2004). On the role of frontal eye field in guiding attention and saccades. Vision
ACCEPTED MANUSCRIPT Research, 44(12), 1453–1467. http://doi.org/10.1016/j.visres.2003.10.025 Senju, A., & Hasegawa, T. (2005). Direct gaze captures visuospatial attention. Visual Cognition, 12(1), 127–144. http://doi.org/10.1080/13506280444000157
RI P
T
Senju, A., Hasegawa, T., & Tojo, Y. (2005). Does perceived direct gaze boost detection in adults and children with and without autism? The stare-in-the-crowd effect revisited. Visual Cognition, 12(8), 1474–1496. http://doi.org/10.1080/13506280444000797 Senju, A., & Johnson, M. H. (2009). The eye contact effect: Mechanisms and development. Trends in Cognitive Sciences, 13(3), 127–134. http://doi.org/10.1016/j.tics.2008.11.009
NU
SC
Smith, D. T., & Schenk, T. (2012). The premotor theory of attention: Time to move on? Neuropsychologia, 50(6), 1104–1114. http://doi.org/10.1016/j.neuropsychologia.2012.01.025
MA
Smith, D. T., Schenk, T., & Rorden, C. (2012). Saccade preparation is required for exogenous attention but not endogenous attention or IOR. Journal of Experimental Psychology: Human Perception & Performance, 38(6), 1438–1447. http://doi.org/http://dx.doi.org/10.1037/a0027794
ED
Spence, C., & Driver, J. (1997). Audiovisual links in exogenous covert spatial orienting. Perception & Psychophysics, 59(1), 1–22. http://doi.org/10.3758/BF03206843 Tanaka, J. W., & Sung, A. (2013). The “eye avoidance” hypothesis of autism face processing. Journal of Autism and Developmental Disorders. http://doi.org/10.1007/s10803-013-1976-7
CE
PT
Todd, R. M., Cunningham, W. A., Anderson, A. K., & Thompson, E. (2012). Affect-biased attention as emotion regulation. Trends in Cognitive Sciences, 16(7), 365–372. http://doi.org/10.1016/j.tics.2012.06.003
AC
Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005). Understanding and sharing intentions: The origins of cultural cognition. The Behavioral and Brain Sciences, 28(5), 675–691. http://doi.org/10.1017/S0140525X05000129 von Grünau, M., & Anston, C. (1995). The detection of gaze direction: A stare-in-the-crowd effect. Perception, 24(11), 1297–1313. http://doi.org/10.1068/p241297 Wieser, M. J., McTeague, L. M., & Keil, A. (2011). Sustained preferential processing of social threat cues: bias without competition? Journal of Cognitive Neuroscience, 23(8), 1973– 1986. http://doi.org/10.1162/jocn.2010.21566 Wu, D. W.-L., Bischof, W. F., & Kingstone, A. (2014). Natural gaze signaling in a social context. Evolution and Human Behavior, 35(3), 211–218. http://doi.org/10.1016/j.evolhumbehav.2014.01.005 Zuckerman, M., Miserandino, M., & Bernieri, F. (1983). Civil inattention exists - in elevators. Personality and Social Psychology Bulletin, 9(4), 578–586. http://doi.org/http://dx.doi.org/10.1177/0146167283094007
ACCEPTED MANUSCRIPT Figure Captions Fig 1. Example camera angles from the outside (A) and indoor (B) locations. Confederate is grey
SC
RI P
T
shaded figure, approaching pedestrians are represented as white shaded figures.
Fig 2. Illustrative confederate actions. Final hand positions for the conditions in which the
NU
confederate raised his hand to answer a phone (top) or wave (bottom), with eyes positioned
MA
either straight ahead (left) or tracking the passing pedestrians (right).
ED
Fig 3. Frequency of pedestrian looks made to the confederate as a function of confederate hand
PT
actions, compared against a no action condition. Only a confederate wave elicited greater
AC
CE
looking than when no action was performed.
AC
CE
PT
ED
MA
NU
SC
RI P
T
ACCEPTED MANUSCRIPT
Fig. 1
CE AC
Fig. 2
PT
ED
MA
NU
SC
RI P
T
ACCEPTED MANUSCRIPT
MA
NU
SC
RI P
T
ACCEPTED MANUSCRIPT
AC
CE
PT
ED
Fig. 3