- Email: [email protected]
Seeing other minds: attributed mental states influence perception
Opinion
Seeing other minds: attributed mental states influence perception Christoph Teufel1, Paul C. Fletcher1 and Greg Davis2 1 2
Brain Mapping Unit, Department of Psychiatry, University of Cambridge, Downing Site, Cambridge CB2 3EB, UK Department of Experimental Psychology, University of Cambridge, Downing Site, Cambridge CB2 3EB, UK
A current consensus views social perception as a bottom-up process in which the human brain uses social signals to make inferences about another’s mental state. Here we propose that, contrary to this model, even the most basic perceptual processing of a social stimulus and closely associated automatic responses are modulated by mental-state attribution. We suggest that social perception is subserved by an interactive bidirectional relationship between the neural mechanisms supporting basic sensory processing of social information and the theory-of-mind system. Consequently, processing of a social stimulus cannot be divorced from its representation in terms of mental states. This hypothesis has farreaching implications for our understanding of both the healthy social brain and characteristic social failures in psychopathology. Social perception and theory of mind In social primates, selective pressures have forged brains that devote extensive neural resources to the perceptual analysis of conspecifics [1]. In humans, the outputs of these processes provide substrates for higher-level social cognition (Box 1). In particular, dynamic cues of human faces and bodies typically trigger the attribution of mental states to others, a process termed theory of mind (ToM) [2,3]. For instance, facial expressions carry information about another’s emotions, head and eye-gaze orientation signal attentional focus, and the specific kinematics of another’s movements often indicate whether an action has been executed intentionally or not. The causal relationship between perception and such mental-state attributions has conventionally been viewed as a unidirectional bottom-up flow of information from the former to the latter [4– 7]. Indeed, it is typically considered that ToM processes operate on completed sensory analysis, with any suggestion that ToM might feedback to alter perception remaining the subject of occasional conjecture. Here, we review recent evidence that even basic perceptual processing of another person’s physical characteristics and automatic behavioural responses closely linked to perception are influenced by what the observer believes about the other’s mental state, which we subsume under the term ‘perceptual mentalizing’. We argue that such evidence should motivate fundamental revision of the conventional views of social perception, incorporating a greater emphasis on prior knowledge and expectations in perception, similar to Corresponding author: Teufel, C. ([email protected]).
376
models in non-social domains [8–13]. This alternative perspective has implications for our understanding of social perception in neurotypical individuals and in psychopathology. Top-down modulation by ToM The most direct evidence of a top-down influence of mentalstate attribution on social perception concerns perception of another’s gaze direction [14]. Seminal electrophysiological work by David Perrett and colleagues identified single cells in the superior temporal sulcus (STS) of the macaque brain that are specifically tuned to different gaze directions Glossary Adaptation: a decrease in the responsiveness of neurons selective for a specific stimulus property after prolonged exposure to that property, often measurable as aftereffects (i.e. specific perceptual distortions). Because of its specificity, adaptation can be used to functionally isolate the contribution of distinct populations of neurons to perception. Autism: developmental disorder defined by repetitive behaviours and characteristic impairments of social cognition and imagination. Bottom-up processing: processing that takes as its starting point information that is contained in the stimulation of the sensory receptors (e.g. retina). Gaze perception and following: gaze perception refers to analysis of the perceptual features of a gaze cue. By contrast, gaze following refers to the tendency of observers to shift attention to locations looked at by others. The latter process thus relies on the former. Imitation: copying by an observer of a feature of another’s body movements. In the current article, we mainly refer to studies that focus on so-called automatic imitation, which relies on a very close association between perception of another person’s movements and execution of the same action. Mirror system: mirror neurons discharge when an individual performs an action and when they observe another individual execute the same action. They were initially described in area F5 of the interior frontal cortex and in the inferior parietal lobule in the macaque brain using single-cell recordings. More recently, neuroimaging studies have demonstrated mirroring characteristics of areas in the human insula and the premotor, parietal, and secondary somatosensory cortices. It is thought that the mirror system plays an important role in ToM by translating perceived behaviour of others into the observer’s ‘neural language’. Schizophrenia: chronic and debilitating syndrome with a lifetime prevalence of just under 1%. It is characterized by, among other features, delusions, which are bizarre beliefs arising without good evidence and held tenaciously even in the face of strongly contradictory evidence. Delusions often involve the attribution of persecutory intentions to others. Social perception: sensory processing of social stimuli such as observed gaze, actions or facial expressions. It is noteworthy that some researchers also use this term to describe higher-level sociocognitive, emotional and evaluative processes based on the perception of such signals. We believe that this inflationary use of the term dilutes its meaning and – in accordance with vision research in other areas – we therefore adopt the narrow definition. Theory of mind: understanding of the behaviour of others in terms of underlying bodily sensations, emotions and mental states such as perceptions, desires and beliefs. This ability involves a diverse collection of different neurocognitive processes. A consistent terminology of the various components is currently lacking. Top-down processing: processing that takes as its starting point knowledge and expectation that the observer’s cognitive system brings to a situation.
1364-6613/$ – see front matter ß 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2010.05.005 Trends in Cognitive Sciences 14 (2010) 376–382
Opinion
Trends in Cognitive Sciences
Vol.14 No.8
Box 1. Neurobiology of social perception and ToM
Box 2. Adaptation in gaze perception and ToM
Some non-primary sensory areas are tuned to specific social features enabling us to perceive complex social stimuli. Electrophysiological, lesioning and neuroimaging studies in humans and non-human primates have identified the STS and adjacent areas as crucial in the initial sensory analysis of social signals (for reviews see [4,6,7,40]). For instance, this region is involved in processing the sensory characteristics of biological, animate and intentional motion [40,41], representing a key processing stage in the understanding of other people’s actions [5,31]. Furthermore, together with the fusiform gyrus (FFG), the STS is responsible for perceptual analysis of faces [42]. It is thought that the FFA processes invariant features such as face identity and the STS processes variant characteristics such as gaze direction and facial expressions. Sensory information about social signals processed in the STS and surrounding areas is relayed to various cortical and subcortical areas associated with ToM. It is important to emphasize that ToM is not a unitary concept and involves a loose collection of dissociable neurocognitive systems, currently without a consistent terminology. Any simplified generalization will cut across categories, but two broad distinctions can be discerned from the literature [2,43]. The classical areas associated with ToM involve a distributed network including the mPFC [44,45] and the TPJ [46,47]. This system is related to visual perspective taking, higher-order belief reasoning and the attribution of representational mental states (perceptions, knowledge, beliefs), all potentially explicit, inferential and deliberate processes. The specific contributions and the extent to which these areas are domain-specific are still strongly debated. When we discuss top-down modulation by ToM in the main text of this article, we refer to this frontotemporoparietal network of the ToM system, which supports explicit mental-state attribution. More recently, the mirror system has been discussed as a second component of the ToM system, subserving a more implicit understanding of other people. It spans a wide range of cortical areas including the insula for the understanding of others’ emotions [48], the secondary somatosensory cortex for the understanding of bodily sensations [49], and the premotor and parietal areas for the understanding of actions [31]. A characteristic of all of these areas is that they have shared circuits for processing of one’s own and other people’s emotions, bodily states and actions. It is thought that this mirroring is important in ToM because it provides a description of observed behaviour in terms of the observers’ own ‘neural language’, thereby allowing them to intuitively place themselves in another person’s mental shoes. Both systems involved in ToM – the frontotemporoparietal and the mirror systems – complement and interact with each other [50].
Following extended exposure to a stimulus, many neural mechanisms coding the properties of that stimulus adapt, becoming less responsive. This adaptation biases the responses of populations of neurons recruited to perceive subsequently presented stimuli, and thereby yields measurable effects on perception. The specificity of these aftereffects makes adaptation ideal for characterizing specialized neural circuits and the methodology is thus referred to as the psychologist’s micro-electrode [51]. In the gaze-adaptation paradigm (Figure 1a) [52,53], observers are exposed to pictures of faces gazing in a specific direction. After this adaptation, the observers’ gaze-direction perception is biased towards the opposite side. In other words, adaptation to leftward gazing faces leads to a subsequent bias in gaze perception to the right and adaptation to a rightward gaze leads to a subsequent leftward bias. These gaze-direction aftereffects indicate the existence of distinct populations of neurons in the human brain that are specifically tuned to different gaze directions. Moreover, a recent study combining adaptation and functional magnetic resonance imaging (fMRI) localized these neurons in the human STS [54], lending support to the notion that the gaze-adaptation paradigm constitutes a non-invasive human analogue to the single-unit recording used in macaques [15]. One recent study modified this paradigm to assess the role of ToM in gaze perception [14]. The static pictures of previous studies were exchanged for short video clips (Figure 1b). An elaborate deception procedure convinced observers that these clips showed a real person on-line via a live camera-link to an adjoining room. Moreover, the specific mental states that observers attributed to this person were directly manipulated using two pairs of goggles, an idea originally borrowed from comparative and developmental psychology [55, M.S. Novey, unpublished doctoral dissertation, Harvard University, 1975]. The lenses of these goggles were highly mirrored and therefore seemed identical when looked at from outside. From the perspective of the person wearing them, however, one was transparent so that the person could see and the other was opaque, thus blindfolding the wearer. Prior to the experiment, observers experienced these visual properties, so that when they saw another person wearing them, one pair signalled that the other could see and the second pair signalled that the other could not see. Observers were adapted to short video clips of a person looking in a specific direction (as indicated by head orientation), believing that this pre-recorded video showed a real person who was either able or unable to see. Gaze-direction aftereffects were subsequently measured in terms of the observer’s bias in perceiving another person’s eye-orientation.
[15]. Functionally similar organization in the human STS has been demonstrated by studies of gaze adaptation, in which distortions in perception were used to infer the activity of gaze-sensitive neurons in the STS (Figure 1a and Box 2). Teufel and colleagues [14] used the gazeadaptation paradigm to demonstrate that ToM processes influence the basic gaze-perception system in the STS (Figure 1b and Box 2). Specifically, when observers believed that another person was able to see, this mental-state attribution facilitated gaze processing relative to when they believed that the person was not able to see despite identical gaze stimuli. This top-down effect was observed even though observers’ allocation of spatial attention to the stimuli was not measurably different across conditions, providing evidence of direct ToM influences on perception. These findings were subsequently extended to gaze following, the tendency of observers to shift their attention towards locations fixated by others. This behaviour constitutes a key developmental building block and regulates adult social interactions in both human and non-human
primates [16–19]. Some authors claim that we follow another person’s gaze because we attribute to them a ‘seeing’ mental state (for a review see [18]), whereas others explicitly dismiss ToM influences, arguing that gaze following is an automatic response triggered directly by the perceptual properties of a gaze stimulus [16,17]. Teufel and colleagues provided a means to integrate these perspectives by demonstrating that the automatic tendency to shift one’s attention towards locations looked at by others is topdown modulated by ToM [20]. They found that the tendency of observers to follow another’s gaze was automatic (in the sense that it could not be entirely suppressed) when observers believed another person could see. However, when they attributed a non-seeing mental state, gaze following could be overridden voluntarily. Thus, ToM seems to specifically influence automatic components of the gaze-following response. One intriguing aspect of this finding is that the automaticity of gaze-following seems to resist direct, voluntary, top-down control (i.e. the participant’s intention to suppress the response) but is susceptible to top-down control by ToM information. Although we 377
(Figure_1)TD$IG][ Opinion
Trends in Cognitive Sciences Vol.14 No.8
Figure 1. Key paradigms in social perception research. (a) In the standard version of the gaze-adaptation paradigm, pictures of faces are used to adapt observers to a specific gaze direction. In a subsequent test phase, in which observers judge another person’s gaze direction, their gaze perception is biased in the non-adapted direction (gaze-direction aftereffect), revealing a specific contribution of adapted neurons to gaze perception. fMRI adaptation indicates distinct populations of neurons in the STS specifically tuned to different gaze directions [54] [a top-up adaptation stimulus precedes every test stimulus (probe) to maintain adaptation to a specific gaze direction]. Adapted from [54] with permission from Elsevier. (b) A recent gaze-adaptation study used short video clips instead of pictures and an elaborate deception to convince observers that they interacted with two real people via a live camera linked to adjoining rooms [14]. The person whose gaze the observers adapted to faced left or right and wore one of two pairs of highly reflective goggles. Observers believed that the person could see through one pair of goggles but not the other. Despite the fact that perceptual bottom-up information was identical in both conditions, attribution of a ‘seeing’ mental state increased gaze processing as measured by gaze-direction aftereffects relative to attribution of a ‘non-seeing’ mental state. Adapted from [14] with permission from Elsevier (c) Automatic imitation is tested in an interference paradigm in which observers have to execute a manual action (a number displayed on the screen indicates which movements observers are asked to perform). The observers’ action is either congruent or incongruent with an action they see being performed simultaneously on the screen [23]. Interference by an incongruent observed action as measured by reaction times (RT) – a marker for automatic imitation – is increased when participants believe that an action has been performed intentionally rather than when they believe that an apparatus forced the movement on the other person. Again, perceptual bottom-up information remains constant across conditions. Copyright ß 2008 by the American Psychological Association. Adapted with permission. The official citation that should be used in referencing this material is [23]. The use of APA information does not imply endorsement by APA.
currently have no clear explanation of the source of this difference, the finding seems to indicate some sort of privileged control of the behaviour by social information (Box 4). Strikingly similar effects have been found in studies investigating imitation, in which a participant copies the body movements of another individual [21]. Despite the fact that imitation plays a pivotal role in socialization, it is partly based on a relatively low-level perception–action coupling: observation of another’s actions automatically triggers preparation of the same action by the observer, sometimes spilling over into overt imitation [22]. Similar to gaze following, in this context automaticity means that the response cannot be entirely suppressed, not that it is exclusively stimulus-driven or informationally-encapsulated. Thus, a role for top-down influences by ToM is not precluded. Indeed, the automaticity of imitation increases when observers believe that finger movements reflect an intentional action [23] (Figure 1c), directly mirroring findings for gaze following [20]. Beliefs about the animacy of observed movements show similar effects [24–26]. Note that in both imitation and gaze-following 378
studies, top-down modulation of social behaviour by mental-state attribution occurred despite the fact that the bottom-up perceptual input remained identical across conditions. The ‘where’, ‘what’ and ‘how’ of top-down modulation Currently, the neural mechanisms of perceptual modulation by ToM are uncertain. Based on our knowledge of the gaze-processing system (Boxes 1 and 2), we can be fairly sure that the site of such influences is the STS, but their source remains a matter of speculation. Teufel and colleagues [14] hypothesized that areas implicated in explicit mental-state attribution such as the medial prefrontal cortex (mPFC) and the temporoparietal junction (TPJ) determine STS activity (Figure 2). In this context, the impact of ToM on automaticity of gaze following [20] is most readily explained not de novo as a direct influence of mental-state attribution on attention independent of perception, but rather as a knock-on effect of this modulation of sensory processing in the STS (in combination with strong connections between the STS and the cortical attention network [6]).
(Figure_2)TD$IG][ Opinion
Trends in Cognitive Sciences
Vol.14 No.8
Figure 2. Schematic representation of our model of social perception. The critical aspect of this perceptual mentalizing model is the influence of the explicit ToM system (mPFC and TPJ) on perceptual processing in the STS and, via this route, on the mirror system (implicit ToM). Predominantly cognitive processes are indicated by light blue, predominantly perceptual processes by grey; red indicates behavioural responses that are very closely associated with perception. Context evaluation, person knowledge and previous interactions are examples of factors that influence the explicit attribution of mental states.
The imitation studies discussed above demonstrate influences of mental-state attribution on automatic components of behaviour, but might, in common with the gaze studies, simply reflect influences of ToM on perception of another’s actions. Indeed, there is already evidence of such modulation [27,28]. An observer’s perception of the temporal onset of another individual’s action changes depending on whether they believe that the same movement is intentional or is achieved via an external motorized apparatus, suggesting a top-down influence of intention attribution on perception. This is even true for intentionality one attributes to one’s own actions, an intriguing perceptual distortion termed the intentional binding effect [29,30]. Our speculation that modulatory effects of ToM on imitation might be a consequence of top-down influences on perceptual coding of actions in STS might seem inconsistent with the suggestion by Liepelt and colleagues [23,24] that intention attribution influences perception–action coupling in the mirror system of the brain [5,31]. Note, however, that the mirror system needs information about the sensory characteristics of another person’s actions to initiate perception–action coupling [5]. Intriguingly, it is the STS that codes this information (Box 1) [5,32]. In other words, the STS is tuned to the sensory characteristics of biological, animate and intentional motion and only observed movements that are perceived as exhibiting these sensory characteristics engage the mirror neuron system via the STS, a proposal that is also suggested by neurophysiological evidence from single-unit recordings [33]. We hypothesize that the top-down influence of explicit beliefs (processed in the mPFC and TPJ) about the intentionality of a movement shapes the sensory processing of another person’s action in the STS in a way that gates access to the strong perception–action coupling in the mirror system (Figure 2). A further key question concerns the nature of the proposed top-down modulation. It seems unlikely that differences in spatial attention allocation play any major role [14], but two further candidate interpretations, illustrated here with reference to gaze perception, remain. First,
mental-state attribution might generally facilitate sensory processing of stimuli to which the mental property ‘seeing’ has been attributed and/or suppress processing of a ‘nonseeing’ stimulus. Neurophysiologically, this should manifest as modulation of activity within a single set of neurons in the STS, and psychologically as increased or decreased perceptual salience of the physical characteristics of a face. In other words, ToM might simply act as an enabler and/or disabler of sensory processing (which in turn might lead to more or less strongly automatic responses). To some extent, this hypothesis has features in common with the perceptual (or intentional) weighting component of the theory of event coding (TEC) [23,34] (although this account mainly focuses on perception–action interactions) in that more or less abstract features are differentially highlighted for perceptual processing. Alternatively, mental-state attribution might switch recruitment between distinct populations of neurons coding stimuli with different mental properties (e.g. those coding ‘seeing’ versus ‘non-seeing’ faces). According to this view, attribution of different mental states should manifest psychologically as a qualitative shift in the observer’s perceptual representations. The latter view is perhaps more congruent with our conscious perception, but the two cannot yet be empirically distinguished. Functional and clinical implications Everyday social interactions are highly complex and the visual system is continuously inundated by important, rapidly-changing and underspecified information. Under these challenging circumstances, we argue that top-down modulation by ToM, a cognitive process specific to the social domain, achieves two key goals: prioritization of perception of the most important social stimuli and disambiguation of informationally noisy perceptual signals. In the latter sense, such top-down control is compatible with emerging Bayesian interpretations of the influence of prior knowledge or expectation on visual processing [35] and sensorimotor control [36]. More generally, this view is consistent with the notion that bidirectional interactions 379
Opinion
Trends in Cognitive Sciences Vol.14 No.8
Box 3. What makes a social stimulus social? Social stimuli can be conceptualized at various levels. At the most fundamental level, people extend in external space and are objects like any others. Stimuli emanating from others thus share fundamental aspects with non-social stimuli and similar bottom-up and top-down perceptual processes apply to both. Second, some social stimuli have perceptual properties that set them apart from other objects. For instance, the kinematic velocity profiles of biological and non-biological motion differ [40] and faces represent a special stimulus for the human visual system, with dedicated neural processes [42]. Finally, the main focus of this article is a level that is inherently social and has no analogue in the non-social realm: the representation of social stimuli in terms of their underlying mental states. Complex dependences between these different levels pose a challenge for experimental design. For instance, at a mental-state level, a person with open eyes can see and a person with closed eyes cannot. Humans have a strong tendency to interpret such stimuli purely in terms of mental states. From a logical perspective, however, equating open and closed eyes with the ability and inability to see means confounding the mental state itself with its possible but not necessary, external manifestations (e.g. [56]). In other words, when using such stimuli it is impossible to determine whether the perceptual differences or the different mental states
between social representations at varying levels of abstraction pick out the socially most relevant information to guide behaviour (Box 3). The adaptive value of top-down influences is illustrated by a situation in which another person’s head and eyes are oriented toward an object (perceptual-properties level) but do not attend to or see the object (mental-state level). This person’s gaze direction might be of little if any social relevance and automatic following of the gaze of this person would be maladaptive. Similarly, whereas fast and automatic imitation of intentional action has a high adaptive value in development and cooperation, automatic imitation of another person’s accidental behaviour might have disastrous consequences. However, perceptually available information is often insufficient to adequately specify particular mental states underlying behaviour. Accordingly, a range of variables such as identity of the social agent, sociocultural context and previous interactions must be used to supplement perceptual information. In our view, top-down influences of ToM construct perceptual representations that encapsulate an integration of these various types of information, enabling an observer to act on the deeper social dimension of another’s behaviour. Although of clear adaptive value, the proposed influence of ToM on perception comes with inherent risks. If ToM beliefs distort ‘the evidence of our senses’, they effectively preclude any unbiased input to social cognition. Given neurochemical disruption, abnormalities of any of the model’s components might lead to deficits in social functioning. The nature of these deficits critically depends on the specifics of how mental-state attribution modulates perceptual processing, which, as outlined above, remains unclear. Accordingly, the specifics of how abnormalities in this complex feedback architecture might contribute to psychopathology are a matter of speculation. These uncertainties aside, if ToM pervades our conscious perception, as it seems to, abnormalities of this interactive process should 380
indicated by the perceptual differences are responsible for differential processing. Innovative studies have recently adopted a very different route to circumvent this problem by keeping stimulus material constant and manipulating only the observers’ interpretation in terms of mental states or animacy [14,20,23–28]. Further complications exist. Most studies of social cognition present pictures of other people to participants, stimuli that exhibit similar perceptual properties as real people. Whereas such pictures are appropriate for testing social perception at the level of perceptual properties, the degree to which they can trigger ToM processes is unclear. Observers clearly know that pictures are objects that cannot see, feel or think. At best, pictures or animations of others therefore represent ambiguous stimuli on the mental-state level, and at worst they trigger qualitatively different sociocognitive processes (e.g. [57]), a finding that is supported by neuroimaging data [58,59]. This methodological problem has recently been overcome by the use of a deception procedure convincing observers that short video clips showing another person was a live video feed to an adjoining room via a camera link [14,20]. The observers thus believed that they were interacting with a real person, while full stimulus control was retained. Studies using pictures and those adopting this novel approach both contribute important complementary information to our understanding of social perception, albeit at different levels.
drastically alter how an observer perceptually experiences the social world, potentially accounting for some puzzling characteristics of psychiatric disorders including autism and schizophrenia (Box 4). First, many high-functioning individuals with autism demonstrate the cognitive capacity for ToM in laboratory tests (e.g. [37]) but do not seem to apply these skills in social interactions. That is, such individuals can attribute mental states to others and use them to explain others’ behaviour, but do not. Our model of social perception might provide a possible explanation for this puzzling finding: A disruption of the feedback by ToM on perception might deplete interactions of the immediate experience of their deeper social dimension. Yet, this immediacy might be crucial for the fast, complex and flexible social responses typical of healthy adults. As a second example, consider the delusional state. Previous frameworks have conceptualized delusions as perturbations of prediction-error-dependent evaluative systems [8,38]. Persistent and inappropriate predictionerror signals induce a compelling sense of novelty and Box 4. Outstanding questions What is the neurochemical basis of the top-down effects of ToM on social perception? Are there individual differences in the balance between bottom-up and top-down control in social perception? To what extent do such differences help to further our understanding of characteristic breakdowns in autism and schizophrenia? Does social information about another person’s mental state really have privileged control over (automatic) social responses that are largely resistant to other types of top-down control? If so, what processes mediate this special type of control? It has been demonstrated that the interpretation of another person’s facial expressions can change depending on contextual variables [60]. This provides an opportunity for future work to investigate the question of whether top-down influences of ToM extend to the sensory processing of emotional expression.
Opinion salience leading to erroneous interpretation and updating of learned associations. In this context, if faulty mentalstate attributions form the basis for perceptual expectations about another’s behaviour, such disturbances would result in perception of this behaviour as unpredictable, strange and perhaps portentous of unusual intent in another person, with time accounting for the emergence of a delusion. Indeed, delusions very frequently centre around other’s intentions. Critically, once strong beliefs have been formed, our framework might also account for their striking persistence in the face of strong evidence to the contrary. Consider a patient who falsely attributes malicious intentions to his partner and believes she wants to poison him. Why does the partner’s loving behaviour not dispel this belief? The finding that beliefs about another person’s mental states pervade even our basic perception suggests that a patient’s unusual belief about their partner’s intentions changes the way in which subsequent social signals are processed. In other words, the patient might not actually perceive the sensory evidence that contradicts their delusional beliefs. Conclusion: a new model of social perception One remarkable feature of social perception is that mental properties we attribute to other humans pervade not only our cognition, but also our conscious perception. The impression that we directly see others’ emotions, intentions and attention was famously noted by the philosopher Ludwig Wittgenstein [39] (although his purposes were different from ours) and now finds parallels in evidence from social cognitive neuroscience that perception of others’ physical characteristics is influenced by processing of their mental states (Figure 2), a phenomenon referred to as perceptual mentalizing. We argue that the functional significance of this interactive process is to ensure that the resulting percept encapsulates socially relevant information that is not (and cannot be) directly provided by the stimulus itself. In other words, the percept is an integration of bottom-up information provided by the stimulus and top-down influences by various context variables channelled by ToM. This interactive model of social perception has important implications for the experience of the social world by both healthy observers and psychiatric patients. Acknowledgments We thank Ian Apperly, Eva Loth, Julia Fischer, Scott Stevens, James Moore and two anonymous reviewers for comments on a previous version of this manuscript. We thank Lisa Ronan for help with editing of the figures. C.T. is supported by the Isaac Newton Trust.
References 1 Ghazanfar, A.A. and Santos, L.R. (2004) Primate brains in the wild: the sensory bases for social interactions. Nat. Rev. Neurosci. 5, 603–616 2 Adolphs, R. (2009) The social brain: neural basis of social knowledge. Annu. Rev. Psychol. 60, 693–716 3 Apperly, I.A. and Butterfill, S.A. (2009) Do humans have two systems to track beliefs and belief-like states? Psychol. Rev. 116, 953–970 4 Allison, T. et al. (2000) Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278 5 Blakemore, S.J. and Decety, J. (2001) From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2, 561–567 6 Nummenmaa, L. and Calder, A.J. (2009) Neural mechanisms of social attention. Trends Cogn. Sci. 13, 135–143
Trends in Cognitive Sciences
Vol.14 No.8
7 Zilbovicius, M. et al. (2006) Autism, the superior temporal sulcus and social perception. Trends Neurosci. 29, 359–366 8 Fletcher, P.C. and Frith, C.D. (2009) Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat. Rev. Neurosci. 10, 48–58 9 Friston, K. (2005) A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815–836 10 Gregory, R.L. (1997) Knowledge in perception and illusion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 352, 1121–1127 11 von Helmholtz, H. (1896) Handbuch der physiologischen Optik, Leopold Voss 12 Sterzer, P. et al. (2008) Believing is seeing: expectations alter visual awareness. Curr. Biol. 18, R697–R698 13 Summerfield, C. and Egner, T. (2009) Expectation (and attention) in visual cognition. Trends Cogn. Sci. 13, 403–409 14 Teufel, C. et al. (2009) Social cognition modulates the sensory coding of observed gaze direction. Curr. Biol. 19, 1274–1277 15 Perrett, D.I. et al. (1992) Organization and functions of cells responsive to faces in the temporal cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 335, 23–30 16 Driver, J. et al. (1999) Gaze perception triggers reflexive visuospatial orienting. Vis. Cogn. 6, 509–540 17 Fernandez-Duque, D. and Baird, J.A. (2005) Is there a ‘‘social brain’’? Lessons from eye-gaze following, joint attention, and autism. In Other Minds: How Humans Bridge the Divide between Self and Others (Malle, B.F. and Hodges, S.D., eds), pp. 75–90, The Guilford Press 18 Meltzoff, A.N. and Brooks, R. (2007) Eyes wide shut: the importance of eyes in infant gaze-following and understanding other minds. In GazeFollowing: Its Development and Significance (Flom, R. et al., eds), pp. 217–241, Lawrence Erlbaum Associates 19 Teufel, C. et al. (2010) Facial expressions modulate the ontogenetic trajectory of gaze-following among monkeys. Dev. Sci. DOI: 10.1111/ j.1467-7687.2010.00956.x 20 Teufel, C. et al. (2010) Mental attribution drives rapid, reflexive gazefollowing in human adults. Atten. Percept. Psychophys. 72, 695–705 21 Heyes, C. (2001) Causes and consequences of imitation. Trends Cogn. Sci. 5, 253–261 22 Brass, M. et al. (2001) Movement observation affects movement execution in a simple response task. Acta Psychol. 106, 3–22 23 Liepelt, R. et al. (2008) What is matched in direct matching? Intention attribution modulates motor priming. J. Exp. Psychol. Hum. Percept. Perform. 34, 578–591 24 Liepelt, R. and Brass, M. (2010) Top-down modulation of motor priming by belief about animacy. Exp. Psychol. 57, 221–227 25 Longo, M.R. and Bertenthal, B.I. (2009) Attention modulates the specificity of automatic imitation to human actors. Exp. Brain Res. 192, 739–744 26 Stanley, J. et al. (2007) Effects of agency on movement interference during observation of a moving dot stimulus. J. Exp. Psychol. Hum. Percept. Perform. 33, 915–926 27 Wohlschla¨ger, A. et al. (2003) Intentionality as a constituting condition for the own self-and other selves. Conscious. Cogn. 12, 708–716 28 Wohlschla¨ger, A. et al. (2003) The perceived onset time of self- and other-generated actions. Psychol. Sci. 14, 586–591 29 Haggard, P. et al. (2002) Voluntary action and conscious awareness. Nat. Neurosci. 5, 382–385 30 Moore, J.W. et al. (2009) Feelings of control: contingency determines experience of action. Cognition 110, 279–283 31 Blakemore, S.J. and Frith, C. (2005) The role of motor contagion in the prediction of action. Neuropsychologia 43, 260–267 32 Rizzolatti, G. et al. (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2, 661– 670 33 Umilta, M.A. et al. (2001) I know what you are doing: a neurophysiological study. Neuron 31, 155–165 34 Hommel, B. et al. (2001) The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–937 35 Yuille, A. and Kersten, D. (2006) Vision as Bayesian inference: analysis by synthesis? Trends Cogn. Sci. 10, 301–308 36 Ko¨rding, K.P. and Wolpert, D.M. (2006) Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 37 Bowler, D.M. (1992) ‘‘Theory of mind’’ in Asperger’s syndrome. J. Child Psychol. Psychiatry 33, 877–893
381
Opinion 38 Corlett, P.R. et al. (2007) Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain 130, 2387–2400 39 Wittgenstein, L. (1953) Philosophische Untersuchungen, Blackwell Publishing 40 Puce, A. and Perrett, D. (2003) Electrophysiology and brain imaging of biological motion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 358, 435– 445 41 Schultz, J. et al. (2004) Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. J. Cogn. Neurosci. 16, 1695–1705 42 Haxby, J.V. et al. (2000) The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223–233 43 Keysers, C. and Gazzola, V. (2007) Integrating simulation and theory of mind: from self to social cognition. Trends Cogn. Sci. 11, 194–196 44 Amodio, D.M. and Frith, C.D. (2006) Meeting of minds: the medial frontal cortex and social cognition. Nat. Rev. Neurosci. 7, 268–277 45 Fletcher, P.C. et al. (1995) Other minds in the brain: a functional imaging study of ‘‘theory of mind’’ in story comprehension. Cognition 57, 109–128 46 Samson, D. et al. (2004) Left temporoparietal junction is necessary for representing someone else’s belief. Nat. Neurosci. 7, 499–500 47 Saxe, R. and Kanwisher, N. (2003) People thinking about thinking people: the role of the temporo-parietal junction in ‘‘theory of mind’’. Neuroimage 19, 1835–1842 48 Singer, T. et al. (2004) Empathy for pain involves the affective but not sensory components of pain. Science 303, 1157–1162
382
Trends in Cognitive Sciences Vol.14 No.8 49 Keysers, C. et al. (2004) A touching sight: SII/PV activation during the observation and experience of touch. Neuron 42, 335–346 50 Brass, M. et al. (2007) Investigating action understanding: Inferential processes versus action simulation. Curr. Biol. 17, 2117–2121 51 Frisby, J.P. (1979) Seeing: Illusion, Brain and Mind, Oxford University Press 52 Calder, A.J. et al. (2008) Visual representation of eye gaze is coded by a nonopponent multichannel system. J. Exp. Psychol. Gen. 137, 244– 261 53 Jenkins, R. et al. (2006) I thought you were looking at me: directionspecific aftereffects in gaze perception. Psychol. Sci. 17, 506–513 54 Calder, A.J. et al. (2007) Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule. Curr. Biol. 17, 20–25 55 Heyes, C.M. (1998) Theory of mind in nonhuman primates. Behav. Brain Sci. 21, 101–148 56 Nuku, P. and Bekkering, H. (2008) Joint attention: inferring what others perceive (and don’t perceive). Conscious. Cogn. 17, 339–349 57 Skarratt, P.A. et al. (2010) Social inhibition of return. Acta Psychol. 134, 48–54 58 Perani, D. et al. (2001) Different brain correlates for watching real and virtual hand actions. Neuroimage 14, 749–758 59 Redcay, E. et al. (2010) Live face-to-face interaction during fMRI: a new tool for social cognitive neuroscience. Neuroimage 50, 1639–1647 60 Mobbs, D. et al. (2006) The Kuleshov effect: the influence of contextual framing on emotional attributions. Soc. Cogn. Affect. Neurosci. 1, 95–106
Seeing other minds: attributed mental states influence perception Christoph Teufel1, Paul C. Fletcher1 and Greg Davis2 1 2
Brain Mapping Unit, Department of Psychiatry, University of Cambridge, Downing Site, Cambridge CB2 3EB, UK Department of Experimental Psychology, University of Cambridge, Downing Site, Cambridge CB2 3EB, UK
A current consensus views social perception as a bottom-up process in which the human brain uses social signals to make inferences about another’s mental state. Here we propose that, contrary to this model, even the most basic perceptual processing of a social stimulus and closely associated automatic responses are modulated by mental-state attribution. We suggest that social perception is subserved by an interactive bidirectional relationship between the neural mechanisms supporting basic sensory processing of social information and the theory-of-mind system. Consequently, processing of a social stimulus cannot be divorced from its representation in terms of mental states. This hypothesis has farreaching implications for our understanding of both the healthy social brain and characteristic social failures in psychopathology. Social perception and theory of mind In social primates, selective pressures have forged brains that devote extensive neural resources to the perceptual analysis of conspecifics [1]. In humans, the outputs of these processes provide substrates for higher-level social cognition (Box 1). In particular, dynamic cues of human faces and bodies typically trigger the attribution of mental states to others, a process termed theory of mind (ToM) [2,3]. For instance, facial expressions carry information about another’s emotions, head and eye-gaze orientation signal attentional focus, and the specific kinematics of another’s movements often indicate whether an action has been executed intentionally or not. The causal relationship between perception and such mental-state attributions has conventionally been viewed as a unidirectional bottom-up flow of information from the former to the latter [4– 7]. Indeed, it is typically considered that ToM processes operate on completed sensory analysis, with any suggestion that ToM might feedback to alter perception remaining the subject of occasional conjecture. Here, we review recent evidence that even basic perceptual processing of another person’s physical characteristics and automatic behavioural responses closely linked to perception are influenced by what the observer believes about the other’s mental state, which we subsume under the term ‘perceptual mentalizing’. We argue that such evidence should motivate fundamental revision of the conventional views of social perception, incorporating a greater emphasis on prior knowledge and expectations in perception, similar to Corresponding author: Teufel, C. ([email protected]).
376
models in non-social domains [8–13]. This alternative perspective has implications for our understanding of social perception in neurotypical individuals and in psychopathology. Top-down modulation by ToM The most direct evidence of a top-down influence of mentalstate attribution on social perception concerns perception of another’s gaze direction [14]. Seminal electrophysiological work by David Perrett and colleagues identified single cells in the superior temporal sulcus (STS) of the macaque brain that are specifically tuned to different gaze directions Glossary Adaptation: a decrease in the responsiveness of neurons selective for a specific stimulus property after prolonged exposure to that property, often measurable as aftereffects (i.e. specific perceptual distortions). Because of its specificity, adaptation can be used to functionally isolate the contribution of distinct populations of neurons to perception. Autism: developmental disorder defined by repetitive behaviours and characteristic impairments of social cognition and imagination. Bottom-up processing: processing that takes as its starting point information that is contained in the stimulation of the sensory receptors (e.g. retina). Gaze perception and following: gaze perception refers to analysis of the perceptual features of a gaze cue. By contrast, gaze following refers to the tendency of observers to shift attention to locations looked at by others. The latter process thus relies on the former. Imitation: copying by an observer of a feature of another’s body movements. In the current article, we mainly refer to studies that focus on so-called automatic imitation, which relies on a very close association between perception of another person’s movements and execution of the same action. Mirror system: mirror neurons discharge when an individual performs an action and when they observe another individual execute the same action. They were initially described in area F5 of the interior frontal cortex and in the inferior parietal lobule in the macaque brain using single-cell recordings. More recently, neuroimaging studies have demonstrated mirroring characteristics of areas in the human insula and the premotor, parietal, and secondary somatosensory cortices. It is thought that the mirror system plays an important role in ToM by translating perceived behaviour of others into the observer’s ‘neural language’. Schizophrenia: chronic and debilitating syndrome with a lifetime prevalence of just under 1%. It is characterized by, among other features, delusions, which are bizarre beliefs arising without good evidence and held tenaciously even in the face of strongly contradictory evidence. Delusions often involve the attribution of persecutory intentions to others. Social perception: sensory processing of social stimuli such as observed gaze, actions or facial expressions. It is noteworthy that some researchers also use this term to describe higher-level sociocognitive, emotional and evaluative processes based on the perception of such signals. We believe that this inflationary use of the term dilutes its meaning and – in accordance with vision research in other areas – we therefore adopt the narrow definition. Theory of mind: understanding of the behaviour of others in terms of underlying bodily sensations, emotions and mental states such as perceptions, desires and beliefs. This ability involves a diverse collection of different neurocognitive processes. A consistent terminology of the various components is currently lacking. Top-down processing: processing that takes as its starting point knowledge and expectation that the observer’s cognitive system brings to a situation.
1364-6613/$ – see front matter ß 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2010.05.005 Trends in Cognitive Sciences 14 (2010) 376–382
Opinion
Trends in Cognitive Sciences
Vol.14 No.8
Box 1. Neurobiology of social perception and ToM
Box 2. Adaptation in gaze perception and ToM
Some non-primary sensory areas are tuned to specific social features enabling us to perceive complex social stimuli. Electrophysiological, lesioning and neuroimaging studies in humans and non-human primates have identified the STS and adjacent areas as crucial in the initial sensory analysis of social signals (for reviews see [4,6,7,40]). For instance, this region is involved in processing the sensory characteristics of biological, animate and intentional motion [40,41], representing a key processing stage in the understanding of other people’s actions [5,31]. Furthermore, together with the fusiform gyrus (FFG), the STS is responsible for perceptual analysis of faces [42]. It is thought that the FFA processes invariant features such as face identity and the STS processes variant characteristics such as gaze direction and facial expressions. Sensory information about social signals processed in the STS and surrounding areas is relayed to various cortical and subcortical areas associated with ToM. It is important to emphasize that ToM is not a unitary concept and involves a loose collection of dissociable neurocognitive systems, currently without a consistent terminology. Any simplified generalization will cut across categories, but two broad distinctions can be discerned from the literature [2,43]. The classical areas associated with ToM involve a distributed network including the mPFC [44,45] and the TPJ [46,47]. This system is related to visual perspective taking, higher-order belief reasoning and the attribution of representational mental states (perceptions, knowledge, beliefs), all potentially explicit, inferential and deliberate processes. The specific contributions and the extent to which these areas are domain-specific are still strongly debated. When we discuss top-down modulation by ToM in the main text of this article, we refer to this frontotemporoparietal network of the ToM system, which supports explicit mental-state attribution. More recently, the mirror system has been discussed as a second component of the ToM system, subserving a more implicit understanding of other people. It spans a wide range of cortical areas including the insula for the understanding of others’ emotions [48], the secondary somatosensory cortex for the understanding of bodily sensations [49], and the premotor and parietal areas for the understanding of actions [31]. A characteristic of all of these areas is that they have shared circuits for processing of one’s own and other people’s emotions, bodily states and actions. It is thought that this mirroring is important in ToM because it provides a description of observed behaviour in terms of the observers’ own ‘neural language’, thereby allowing them to intuitively place themselves in another person’s mental shoes. Both systems involved in ToM – the frontotemporoparietal and the mirror systems – complement and interact with each other [50].
Following extended exposure to a stimulus, many neural mechanisms coding the properties of that stimulus adapt, becoming less responsive. This adaptation biases the responses of populations of neurons recruited to perceive subsequently presented stimuli, and thereby yields measurable effects on perception. The specificity of these aftereffects makes adaptation ideal for characterizing specialized neural circuits and the methodology is thus referred to as the psychologist’s micro-electrode [51]. In the gaze-adaptation paradigm (Figure 1a) [52,53], observers are exposed to pictures of faces gazing in a specific direction. After this adaptation, the observers’ gaze-direction perception is biased towards the opposite side. In other words, adaptation to leftward gazing faces leads to a subsequent bias in gaze perception to the right and adaptation to a rightward gaze leads to a subsequent leftward bias. These gaze-direction aftereffects indicate the existence of distinct populations of neurons in the human brain that are specifically tuned to different gaze directions. Moreover, a recent study combining adaptation and functional magnetic resonance imaging (fMRI) localized these neurons in the human STS [54], lending support to the notion that the gaze-adaptation paradigm constitutes a non-invasive human analogue to the single-unit recording used in macaques [15]. One recent study modified this paradigm to assess the role of ToM in gaze perception [14]. The static pictures of previous studies were exchanged for short video clips (Figure 1b). An elaborate deception procedure convinced observers that these clips showed a real person on-line via a live camera-link to an adjoining room. Moreover, the specific mental states that observers attributed to this person were directly manipulated using two pairs of goggles, an idea originally borrowed from comparative and developmental psychology [55, M.S. Novey, unpublished doctoral dissertation, Harvard University, 1975]. The lenses of these goggles were highly mirrored and therefore seemed identical when looked at from outside. From the perspective of the person wearing them, however, one was transparent so that the person could see and the other was opaque, thus blindfolding the wearer. Prior to the experiment, observers experienced these visual properties, so that when they saw another person wearing them, one pair signalled that the other could see and the second pair signalled that the other could not see. Observers were adapted to short video clips of a person looking in a specific direction (as indicated by head orientation), believing that this pre-recorded video showed a real person who was either able or unable to see. Gaze-direction aftereffects were subsequently measured in terms of the observer’s bias in perceiving another person’s eye-orientation.
[15]. Functionally similar organization in the human STS has been demonstrated by studies of gaze adaptation, in which distortions in perception were used to infer the activity of gaze-sensitive neurons in the STS (Figure 1a and Box 2). Teufel and colleagues [14] used the gazeadaptation paradigm to demonstrate that ToM processes influence the basic gaze-perception system in the STS (Figure 1b and Box 2). Specifically, when observers believed that another person was able to see, this mental-state attribution facilitated gaze processing relative to when they believed that the person was not able to see despite identical gaze stimuli. This top-down effect was observed even though observers’ allocation of spatial attention to the stimuli was not measurably different across conditions, providing evidence of direct ToM influences on perception. These findings were subsequently extended to gaze following, the tendency of observers to shift their attention towards locations fixated by others. This behaviour constitutes a key developmental building block and regulates adult social interactions in both human and non-human
primates [16–19]. Some authors claim that we follow another person’s gaze because we attribute to them a ‘seeing’ mental state (for a review see [18]), whereas others explicitly dismiss ToM influences, arguing that gaze following is an automatic response triggered directly by the perceptual properties of a gaze stimulus [16,17]. Teufel and colleagues provided a means to integrate these perspectives by demonstrating that the automatic tendency to shift one’s attention towards locations looked at by others is topdown modulated by ToM [20]. They found that the tendency of observers to follow another’s gaze was automatic (in the sense that it could not be entirely suppressed) when observers believed another person could see. However, when they attributed a non-seeing mental state, gaze following could be overridden voluntarily. Thus, ToM seems to specifically influence automatic components of the gaze-following response. One intriguing aspect of this finding is that the automaticity of gaze-following seems to resist direct, voluntary, top-down control (i.e. the participant’s intention to suppress the response) but is susceptible to top-down control by ToM information. Although we 377
(Figure_1)TD$IG][ Opinion
Trends in Cognitive Sciences Vol.14 No.8
Figure 1. Key paradigms in social perception research. (a) In the standard version of the gaze-adaptation paradigm, pictures of faces are used to adapt observers to a specific gaze direction. In a subsequent test phase, in which observers judge another person’s gaze direction, their gaze perception is biased in the non-adapted direction (gaze-direction aftereffect), revealing a specific contribution of adapted neurons to gaze perception. fMRI adaptation indicates distinct populations of neurons in the STS specifically tuned to different gaze directions [54] [a top-up adaptation stimulus precedes every test stimulus (probe) to maintain adaptation to a specific gaze direction]. Adapted from [54] with permission from Elsevier. (b) A recent gaze-adaptation study used short video clips instead of pictures and an elaborate deception to convince observers that they interacted with two real people via a live camera linked to adjoining rooms [14]. The person whose gaze the observers adapted to faced left or right and wore one of two pairs of highly reflective goggles. Observers believed that the person could see through one pair of goggles but not the other. Despite the fact that perceptual bottom-up information was identical in both conditions, attribution of a ‘seeing’ mental state increased gaze processing as measured by gaze-direction aftereffects relative to attribution of a ‘non-seeing’ mental state. Adapted from [14] with permission from Elsevier (c) Automatic imitation is tested in an interference paradigm in which observers have to execute a manual action (a number displayed on the screen indicates which movements observers are asked to perform). The observers’ action is either congruent or incongruent with an action they see being performed simultaneously on the screen [23]. Interference by an incongruent observed action as measured by reaction times (RT) – a marker for automatic imitation – is increased when participants believe that an action has been performed intentionally rather than when they believe that an apparatus forced the movement on the other person. Again, perceptual bottom-up information remains constant across conditions. Copyright ß 2008 by the American Psychological Association. Adapted with permission. The official citation that should be used in referencing this material is [23]. The use of APA information does not imply endorsement by APA.
currently have no clear explanation of the source of this difference, the finding seems to indicate some sort of privileged control of the behaviour by social information (Box 4). Strikingly similar effects have been found in studies investigating imitation, in which a participant copies the body movements of another individual [21]. Despite the fact that imitation plays a pivotal role in socialization, it is partly based on a relatively low-level perception–action coupling: observation of another’s actions automatically triggers preparation of the same action by the observer, sometimes spilling over into overt imitation [22]. Similar to gaze following, in this context automaticity means that the response cannot be entirely suppressed, not that it is exclusively stimulus-driven or informationally-encapsulated. Thus, a role for top-down influences by ToM is not precluded. Indeed, the automaticity of imitation increases when observers believe that finger movements reflect an intentional action [23] (Figure 1c), directly mirroring findings for gaze following [20]. Beliefs about the animacy of observed movements show similar effects [24–26]. Note that in both imitation and gaze-following 378
studies, top-down modulation of social behaviour by mental-state attribution occurred despite the fact that the bottom-up perceptual input remained identical across conditions. The ‘where’, ‘what’ and ‘how’ of top-down modulation Currently, the neural mechanisms of perceptual modulation by ToM are uncertain. Based on our knowledge of the gaze-processing system (Boxes 1 and 2), we can be fairly sure that the site of such influences is the STS, but their source remains a matter of speculation. Teufel and colleagues [14] hypothesized that areas implicated in explicit mental-state attribution such as the medial prefrontal cortex (mPFC) and the temporoparietal junction (TPJ) determine STS activity (Figure 2). In this context, the impact of ToM on automaticity of gaze following [20] is most readily explained not de novo as a direct influence of mental-state attribution on attention independent of perception, but rather as a knock-on effect of this modulation of sensory processing in the STS (in combination with strong connections between the STS and the cortical attention network [6]).
(Figure_2)TD$IG][ Opinion
Trends in Cognitive Sciences
Vol.14 No.8
Figure 2. Schematic representation of our model of social perception. The critical aspect of this perceptual mentalizing model is the influence of the explicit ToM system (mPFC and TPJ) on perceptual processing in the STS and, via this route, on the mirror system (implicit ToM). Predominantly cognitive processes are indicated by light blue, predominantly perceptual processes by grey; red indicates behavioural responses that are very closely associated with perception. Context evaluation, person knowledge and previous interactions are examples of factors that influence the explicit attribution of mental states.
The imitation studies discussed above demonstrate influences of mental-state attribution on automatic components of behaviour, but might, in common with the gaze studies, simply reflect influences of ToM on perception of another’s actions. Indeed, there is already evidence of such modulation [27,28]. An observer’s perception of the temporal onset of another individual’s action changes depending on whether they believe that the same movement is intentional or is achieved via an external motorized apparatus, suggesting a top-down influence of intention attribution on perception. This is even true for intentionality one attributes to one’s own actions, an intriguing perceptual distortion termed the intentional binding effect [29,30]. Our speculation that modulatory effects of ToM on imitation might be a consequence of top-down influences on perceptual coding of actions in STS might seem inconsistent with the suggestion by Liepelt and colleagues [23,24] that intention attribution influences perception–action coupling in the mirror system of the brain [5,31]. Note, however, that the mirror system needs information about the sensory characteristics of another person’s actions to initiate perception–action coupling [5]. Intriguingly, it is the STS that codes this information (Box 1) [5,32]. In other words, the STS is tuned to the sensory characteristics of biological, animate and intentional motion and only observed movements that are perceived as exhibiting these sensory characteristics engage the mirror neuron system via the STS, a proposal that is also suggested by neurophysiological evidence from single-unit recordings [33]. We hypothesize that the top-down influence of explicit beliefs (processed in the mPFC and TPJ) about the intentionality of a movement shapes the sensory processing of another person’s action in the STS in a way that gates access to the strong perception–action coupling in the mirror system (Figure 2). A further key question concerns the nature of the proposed top-down modulation. It seems unlikely that differences in spatial attention allocation play any major role [14], but two further candidate interpretations, illustrated here with reference to gaze perception, remain. First,
mental-state attribution might generally facilitate sensory processing of stimuli to which the mental property ‘seeing’ has been attributed and/or suppress processing of a ‘nonseeing’ stimulus. Neurophysiologically, this should manifest as modulation of activity within a single set of neurons in the STS, and psychologically as increased or decreased perceptual salience of the physical characteristics of a face. In other words, ToM might simply act as an enabler and/or disabler of sensory processing (which in turn might lead to more or less strongly automatic responses). To some extent, this hypothesis has features in common with the perceptual (or intentional) weighting component of the theory of event coding (TEC) [23,34] (although this account mainly focuses on perception–action interactions) in that more or less abstract features are differentially highlighted for perceptual processing. Alternatively, mental-state attribution might switch recruitment between distinct populations of neurons coding stimuli with different mental properties (e.g. those coding ‘seeing’ versus ‘non-seeing’ faces). According to this view, attribution of different mental states should manifest psychologically as a qualitative shift in the observer’s perceptual representations. The latter view is perhaps more congruent with our conscious perception, but the two cannot yet be empirically distinguished. Functional and clinical implications Everyday social interactions are highly complex and the visual system is continuously inundated by important, rapidly-changing and underspecified information. Under these challenging circumstances, we argue that top-down modulation by ToM, a cognitive process specific to the social domain, achieves two key goals: prioritization of perception of the most important social stimuli and disambiguation of informationally noisy perceptual signals. In the latter sense, such top-down control is compatible with emerging Bayesian interpretations of the influence of prior knowledge or expectation on visual processing [35] and sensorimotor control [36]. More generally, this view is consistent with the notion that bidirectional interactions 379
Opinion
Trends in Cognitive Sciences Vol.14 No.8
Box 3. What makes a social stimulus social? Social stimuli can be conceptualized at various levels. At the most fundamental level, people extend in external space and are objects like any others. Stimuli emanating from others thus share fundamental aspects with non-social stimuli and similar bottom-up and top-down perceptual processes apply to both. Second, some social stimuli have perceptual properties that set them apart from other objects. For instance, the kinematic velocity profiles of biological and non-biological motion differ [40] and faces represent a special stimulus for the human visual system, with dedicated neural processes [42]. Finally, the main focus of this article is a level that is inherently social and has no analogue in the non-social realm: the representation of social stimuli in terms of their underlying mental states. Complex dependences between these different levels pose a challenge for experimental design. For instance, at a mental-state level, a person with open eyes can see and a person with closed eyes cannot. Humans have a strong tendency to interpret such stimuli purely in terms of mental states. From a logical perspective, however, equating open and closed eyes with the ability and inability to see means confounding the mental state itself with its possible but not necessary, external manifestations (e.g. [56]). In other words, when using such stimuli it is impossible to determine whether the perceptual differences or the different mental states
between social representations at varying levels of abstraction pick out the socially most relevant information to guide behaviour (Box 3). The adaptive value of top-down influences is illustrated by a situation in which another person’s head and eyes are oriented toward an object (perceptual-properties level) but do not attend to or see the object (mental-state level). This person’s gaze direction might be of little if any social relevance and automatic following of the gaze of this person would be maladaptive. Similarly, whereas fast and automatic imitation of intentional action has a high adaptive value in development and cooperation, automatic imitation of another person’s accidental behaviour might have disastrous consequences. However, perceptually available information is often insufficient to adequately specify particular mental states underlying behaviour. Accordingly, a range of variables such as identity of the social agent, sociocultural context and previous interactions must be used to supplement perceptual information. In our view, top-down influences of ToM construct perceptual representations that encapsulate an integration of these various types of information, enabling an observer to act on the deeper social dimension of another’s behaviour. Although of clear adaptive value, the proposed influence of ToM on perception comes with inherent risks. If ToM beliefs distort ‘the evidence of our senses’, they effectively preclude any unbiased input to social cognition. Given neurochemical disruption, abnormalities of any of the model’s components might lead to deficits in social functioning. The nature of these deficits critically depends on the specifics of how mental-state attribution modulates perceptual processing, which, as outlined above, remains unclear. Accordingly, the specifics of how abnormalities in this complex feedback architecture might contribute to psychopathology are a matter of speculation. These uncertainties aside, if ToM pervades our conscious perception, as it seems to, abnormalities of this interactive process should 380
indicated by the perceptual differences are responsible for differential processing. Innovative studies have recently adopted a very different route to circumvent this problem by keeping stimulus material constant and manipulating only the observers’ interpretation in terms of mental states or animacy [14,20,23–28]. Further complications exist. Most studies of social cognition present pictures of other people to participants, stimuli that exhibit similar perceptual properties as real people. Whereas such pictures are appropriate for testing social perception at the level of perceptual properties, the degree to which they can trigger ToM processes is unclear. Observers clearly know that pictures are objects that cannot see, feel or think. At best, pictures or animations of others therefore represent ambiguous stimuli on the mental-state level, and at worst they trigger qualitatively different sociocognitive processes (e.g. [57]), a finding that is supported by neuroimaging data [58,59]. This methodological problem has recently been overcome by the use of a deception procedure convincing observers that short video clips showing another person was a live video feed to an adjoining room via a camera link [14,20]. The observers thus believed that they were interacting with a real person, while full stimulus control was retained. Studies using pictures and those adopting this novel approach both contribute important complementary information to our understanding of social perception, albeit at different levels.
drastically alter how an observer perceptually experiences the social world, potentially accounting for some puzzling characteristics of psychiatric disorders including autism and schizophrenia (Box 4). First, many high-functioning individuals with autism demonstrate the cognitive capacity for ToM in laboratory tests (e.g. [37]) but do not seem to apply these skills in social interactions. That is, such individuals can attribute mental states to others and use them to explain others’ behaviour, but do not. Our model of social perception might provide a possible explanation for this puzzling finding: A disruption of the feedback by ToM on perception might deplete interactions of the immediate experience of their deeper social dimension. Yet, this immediacy might be crucial for the fast, complex and flexible social responses typical of healthy adults. As a second example, consider the delusional state. Previous frameworks have conceptualized delusions as perturbations of prediction-error-dependent evaluative systems [8,38]. Persistent and inappropriate predictionerror signals induce a compelling sense of novelty and Box 4. Outstanding questions What is the neurochemical basis of the top-down effects of ToM on social perception? Are there individual differences in the balance between bottom-up and top-down control in social perception? To what extent do such differences help to further our understanding of characteristic breakdowns in autism and schizophrenia? Does social information about another person’s mental state really have privileged control over (automatic) social responses that are largely resistant to other types of top-down control? If so, what processes mediate this special type of control? It has been demonstrated that the interpretation of another person’s facial expressions can change depending on contextual variables [60]. This provides an opportunity for future work to investigate the question of whether top-down influences of ToM extend to the sensory processing of emotional expression.
Opinion salience leading to erroneous interpretation and updating of learned associations. In this context, if faulty mentalstate attributions form the basis for perceptual expectations about another’s behaviour, such disturbances would result in perception of this behaviour as unpredictable, strange and perhaps portentous of unusual intent in another person, with time accounting for the emergence of a delusion. Indeed, delusions very frequently centre around other’s intentions. Critically, once strong beliefs have been formed, our framework might also account for their striking persistence in the face of strong evidence to the contrary. Consider a patient who falsely attributes malicious intentions to his partner and believes she wants to poison him. Why does the partner’s loving behaviour not dispel this belief? The finding that beliefs about another person’s mental states pervade even our basic perception suggests that a patient’s unusual belief about their partner’s intentions changes the way in which subsequent social signals are processed. In other words, the patient might not actually perceive the sensory evidence that contradicts their delusional beliefs. Conclusion: a new model of social perception One remarkable feature of social perception is that mental properties we attribute to other humans pervade not only our cognition, but also our conscious perception. The impression that we directly see others’ emotions, intentions and attention was famously noted by the philosopher Ludwig Wittgenstein [39] (although his purposes were different from ours) and now finds parallels in evidence from social cognitive neuroscience that perception of others’ physical characteristics is influenced by processing of their mental states (Figure 2), a phenomenon referred to as perceptual mentalizing. We argue that the functional significance of this interactive process is to ensure that the resulting percept encapsulates socially relevant information that is not (and cannot be) directly provided by the stimulus itself. In other words, the percept is an integration of bottom-up information provided by the stimulus and top-down influences by various context variables channelled by ToM. This interactive model of social perception has important implications for the experience of the social world by both healthy observers and psychiatric patients. Acknowledgments We thank Ian Apperly, Eva Loth, Julia Fischer, Scott Stevens, James Moore and two anonymous reviewers for comments on a previous version of this manuscript. We thank Lisa Ronan for help with editing of the figures. C.T. is supported by the Isaac Newton Trust.
References 1 Ghazanfar, A.A. and Santos, L.R. (2004) Primate brains in the wild: the sensory bases for social interactions. Nat. Rev. Neurosci. 5, 603–616 2 Adolphs, R. (2009) The social brain: neural basis of social knowledge. Annu. Rev. Psychol. 60, 693–716 3 Apperly, I.A. and Butterfill, S.A. (2009) Do humans have two systems to track beliefs and belief-like states? Psychol. Rev. 116, 953–970 4 Allison, T. et al. (2000) Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278 5 Blakemore, S.J. and Decety, J. (2001) From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2, 561–567 6 Nummenmaa, L. and Calder, A.J. (2009) Neural mechanisms of social attention. Trends Cogn. Sci. 13, 135–143
Trends in Cognitive Sciences
Vol.14 No.8
7 Zilbovicius, M. et al. (2006) Autism, the superior temporal sulcus and social perception. Trends Neurosci. 29, 359–366 8 Fletcher, P.C. and Frith, C.D. (2009) Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat. Rev. Neurosci. 10, 48–58 9 Friston, K. (2005) A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815–836 10 Gregory, R.L. (1997) Knowledge in perception and illusion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 352, 1121–1127 11 von Helmholtz, H. (1896) Handbuch der physiologischen Optik, Leopold Voss 12 Sterzer, P. et al. (2008) Believing is seeing: expectations alter visual awareness. Curr. Biol. 18, R697–R698 13 Summerfield, C. and Egner, T. (2009) Expectation (and attention) in visual cognition. Trends Cogn. Sci. 13, 403–409 14 Teufel, C. et al. (2009) Social cognition modulates the sensory coding of observed gaze direction. Curr. Biol. 19, 1274–1277 15 Perrett, D.I. et al. (1992) Organization and functions of cells responsive to faces in the temporal cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 335, 23–30 16 Driver, J. et al. (1999) Gaze perception triggers reflexive visuospatial orienting. Vis. Cogn. 6, 509–540 17 Fernandez-Duque, D. and Baird, J.A. (2005) Is there a ‘‘social brain’’? Lessons from eye-gaze following, joint attention, and autism. In Other Minds: How Humans Bridge the Divide between Self and Others (Malle, B.F. and Hodges, S.D., eds), pp. 75–90, The Guilford Press 18 Meltzoff, A.N. and Brooks, R. (2007) Eyes wide shut: the importance of eyes in infant gaze-following and understanding other minds. In GazeFollowing: Its Development and Significance (Flom, R. et al., eds), pp. 217–241, Lawrence Erlbaum Associates 19 Teufel, C. et al. (2010) Facial expressions modulate the ontogenetic trajectory of gaze-following among monkeys. Dev. Sci. DOI: 10.1111/ j.1467-7687.2010.00956.x 20 Teufel, C. et al. (2010) Mental attribution drives rapid, reflexive gazefollowing in human adults. Atten. Percept. Psychophys. 72, 695–705 21 Heyes, C. (2001) Causes and consequences of imitation. Trends Cogn. Sci. 5, 253–261 22 Brass, M. et al. (2001) Movement observation affects movement execution in a simple response task. Acta Psychol. 106, 3–22 23 Liepelt, R. et al. (2008) What is matched in direct matching? Intention attribution modulates motor priming. J. Exp. Psychol. Hum. Percept. Perform. 34, 578–591 24 Liepelt, R. and Brass, M. (2010) Top-down modulation of motor priming by belief about animacy. Exp. Psychol. 57, 221–227 25 Longo, M.R. and Bertenthal, B.I. (2009) Attention modulates the specificity of automatic imitation to human actors. Exp. Brain Res. 192, 739–744 26 Stanley, J. et al. (2007) Effects of agency on movement interference during observation of a moving dot stimulus. J. Exp. Psychol. Hum. Percept. Perform. 33, 915–926 27 Wohlschla¨ger, A. et al. (2003) Intentionality as a constituting condition for the own self-and other selves. Conscious. Cogn. 12, 708–716 28 Wohlschla¨ger, A. et al. (2003) The perceived onset time of self- and other-generated actions. Psychol. Sci. 14, 586–591 29 Haggard, P. et al. (2002) Voluntary action and conscious awareness. Nat. Neurosci. 5, 382–385 30 Moore, J.W. et al. (2009) Feelings of control: contingency determines experience of action. Cognition 110, 279–283 31 Blakemore, S.J. and Frith, C. (2005) The role of motor contagion in the prediction of action. Neuropsychologia 43, 260–267 32 Rizzolatti, G. et al. (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2, 661– 670 33 Umilta, M.A. et al. (2001) I know what you are doing: a neurophysiological study. Neuron 31, 155–165 34 Hommel, B. et al. (2001) The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–937 35 Yuille, A. and Kersten, D. (2006) Vision as Bayesian inference: analysis by synthesis? Trends Cogn. Sci. 10, 301–308 36 Ko¨rding, K.P. and Wolpert, D.M. (2006) Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 37 Bowler, D.M. (1992) ‘‘Theory of mind’’ in Asperger’s syndrome. J. Child Psychol. Psychiatry 33, 877–893
381
Opinion 38 Corlett, P.R. et al. (2007) Disrupted prediction-error signal in psychosis: evidence for an associative account of delusions. Brain 130, 2387–2400 39 Wittgenstein, L. (1953) Philosophische Untersuchungen, Blackwell Publishing 40 Puce, A. and Perrett, D. (2003) Electrophysiology and brain imaging of biological motion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 358, 435– 445 41 Schultz, J. et al. (2004) Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. J. Cogn. Neurosci. 16, 1695–1705 42 Haxby, J.V. et al. (2000) The distributed human neural system for face perception. Trends Cogn. Sci. 4, 223–233 43 Keysers, C. and Gazzola, V. (2007) Integrating simulation and theory of mind: from self to social cognition. Trends Cogn. Sci. 11, 194–196 44 Amodio, D.M. and Frith, C.D. (2006) Meeting of minds: the medial frontal cortex and social cognition. Nat. Rev. Neurosci. 7, 268–277 45 Fletcher, P.C. et al. (1995) Other minds in the brain: a functional imaging study of ‘‘theory of mind’’ in story comprehension. Cognition 57, 109–128 46 Samson, D. et al. (2004) Left temporoparietal junction is necessary for representing someone else’s belief. Nat. Neurosci. 7, 499–500 47 Saxe, R. and Kanwisher, N. (2003) People thinking about thinking people: the role of the temporo-parietal junction in ‘‘theory of mind’’. Neuroimage 19, 1835–1842 48 Singer, T. et al. (2004) Empathy for pain involves the affective but not sensory components of pain. Science 303, 1157–1162
382
Trends in Cognitive Sciences Vol.14 No.8 49 Keysers, C. et al. (2004) A touching sight: SII/PV activation during the observation and experience of touch. Neuron 42, 335–346 50 Brass, M. et al. (2007) Investigating action understanding: Inferential processes versus action simulation. Curr. Biol. 17, 2117–2121 51 Frisby, J.P. (1979) Seeing: Illusion, Brain and Mind, Oxford University Press 52 Calder, A.J. et al. (2008) Visual representation of eye gaze is coded by a nonopponent multichannel system. J. Exp. Psychol. Gen. 137, 244– 261 53 Jenkins, R. et al. (2006) I thought you were looking at me: directionspecific aftereffects in gaze perception. Psychol. Sci. 17, 506–513 54 Calder, A.J. et al. (2007) Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule. Curr. Biol. 17, 20–25 55 Heyes, C.M. (1998) Theory of mind in nonhuman primates. Behav. Brain Sci. 21, 101–148 56 Nuku, P. and Bekkering, H. (2008) Joint attention: inferring what others perceive (and don’t perceive). Conscious. Cogn. 17, 339–349 57 Skarratt, P.A. et al. (2010) Social inhibition of return. Acta Psychol. 134, 48–54 58 Perani, D. et al. (2001) Different brain correlates for watching real and virtual hand actions. Neuroimage 14, 749–758 59 Redcay, E. et al. (2010) Live face-to-face interaction during fMRI: a new tool for social cognitive neuroscience. Neuroimage 50, 1639–1647 60 Mobbs, D. et al. (2006) The Kuleshov effect: the influence of contextual framing on emotional attributions. Soc. Cogn. Affect. Neurosci. 1, 95–106