Attitudinal prosody: What we know and directions for future study

Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

Contents lists available at SciVerse ScienceDirect

Neuroscience and Biobehavioral Reviews journal homepage: www.elsevier.com/locate/neubiorev

Review

Attitudinal prosody: What we know and directions for future study Rachel L.C. Mitchell a,∗ , Elliott D. Ross b a b

Department of Psychology, Durham University, Durham DH1 3LE, UK Department of Neurology, University of Oklahoma Health Sciences Centre, Oklahoma City, OK 73104, USA

a r t i c l e

i n f o

Article history: Received 4 December 2012 Received in revised form 16 January 2013 Accepted 28 January 2013 Keywords: Attitudes Prosody Voice Neurology Affective fMRI

a b s t r a c t Prosodic aspects of speech such as pitch, duration and amplitude constitute nonverbal cues that supplement or modify the meaning of the spoken word, to provide valuable clues as to a speakers’ state of mind. It can thus indicate what emotion a person is feeling (emotional prosody), or their attitude towards an event, person or object (attitudinal prosody). Whilst the study of emotional prosody has gathered pace, attitudinal prosody now deserves equal attention. In social cognition, understanding attitudinal prosody is important in its own right, since it can convey powerful constructs such as confidence, persuasion, sarcasm and superiority. In this review, it is examined what prosody is, how it conveys attitudes, and which attitudes prosody can convey. The review finishes by considering the neuroanatomy associated with attitudinal prosody, and put forward the hypothesis that this cognition is mediated by the right cerebral hemisphere, particularly posterior superior lateral temporal cortex, with an additional role for the basal ganglia, and limbic regions such as the medial prefrontal cortex and amygdala. It is suggested that further exploration of its functional neuroanatomy is greatly needed, since it could provide valuable clues about the value of current prosody nomenclature and its separability from other types of prosody at the behavioural level. © 2013 Elsevier Ltd. All rights reserved.

Contents 1. 2. 3. 4. 5. 6. 7.

What is prosody? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conceptual overlap and disparity between attitudinal and emotional prosody . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How does prosody convey attitudes? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Which attitudes can be conveyed by prosody? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The neuroanatomy of attitudinal prosody and its overlap with emotional prosody: clinical and lesion studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The neuroanatomy of attitudinal prosody and its overlap with emotional prosody: neuroimaging studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1. What is prosody? Nonverbal aspects of communication add value to verbal communication (Schul and Lamb, 1982), and may be the way we have been equipped by evolution to signal our authentic feelings (Turner, 2002). Thus, auditory nonverbal cues may be just as important as visual ones (Borod, 1993). Indeed in certain situations (e.g. telephone calls) facial cues are not available (Barath and Cannell, 1976). In the auditory modality, manipulation of communication partners’ perception can be achieved with carefully chosen words but,

∗ Corresponding author. Tel.: +44 191 334 3272; fax: +44 191 334 3241. E-mail addresses: [email protected], [email protected] (R.L.C. Mitchell). 0149-7634/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.neubiorev.2013.01.027

471 472 472 473 474 476 476 477

equally important may be the way in which these words are spoken, i.e. the tone of voice used. Tone of voice is a pragmatic language function; an umbrella term that concerns the ways in which context contributes to meaning. Grice thus distinguishes between ‘sentence meaning’ (semantic properties of a communication assigned by grammar) and ‘speaker’s meaning’ (what the speaker intended to communicate by their utterance) (Grice, 1989). Understanding sentence meaning simply requires one to decode the utterance, but understanding speaker meaning involves attributing the intention of producing a cognitive effect, by causing the audience to recognise that intention (Noveck and Reboul, 2008). In research literature, tone of voice is typically referred to as ‘prosody’, a term that refers to suprasegmental aspects of speech that are paralinguistic. These features encompass changes in intonation, amplitude envelope,

472

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

tempo, rhythm and voice quality (Grandjean et al., 2006), although dynamic change in pitch is the most important (Balaguer-Ballester et al., 2009; Ross et al., 1986, 1988). According to Monrad-Kohn, spoken narratives thus contain three main elements: the spoken words (vocabulary), their declination, conjugation and correct placing in phrases (grammar), and their variations in pitch, rhythm and stress of pronunciation (prosody) (Monrad-Kohn, 1963). Prosody is that aspect of communication that creates emotional tone, subtle grades of meaning and slightly varied emphasis in spoken word (Boss, 1996). One of the most well-known groupings suggests that four different types of prosody can be distinguished (Monrad-Kohn, 1947). Intrinsic prosody includes linguistic uses of prosody e.g. the demarcation between a question or statement, and also creates dialectical (regional) and idiosyncratic differences in speech quality that give rise to voice patterns and qualities that are unique to an individual. The speaker’s attitude about the information being conveyed (e.g. scepticism, doubt, enthusiasm, boredom etc.) is intellectual prosody. Intellectual prosody encodes attitudinal information in discourse that may drastically alter meaning. The emotion conveyed by tone of voice in the communication (happiness, sadness, fear etc.) is known as emotional prosody. Finally, inarticulate prosody refers to non-linguistic sounds such as sighs, groans or cries that are often used to embellish discourse (Boss, 1996).

2. Conceptual overlap and disparity between attitudinal and emotional prosody More recently, attitudinal or intellectual prosody is often grouped together with emotional prosody under the superordinate term ‘affective prosody’ (Ross, 2000), but most prior affective prosody research has focussed on emotional prosody (Fichten et al., 1992). One reason for this grouping is that attitudes and emotions are expressed by partially overlapping prosodic elements (Pell, 2006a). Following measurement of the acoustic properties amplitude, pitch and spectral profile for simple emotions such as sadness and more attitudinal states such as relief, discriminant analyses established that these acoustic measures provide sufficient discrimination between expressions to permit accurate statistical classification of both types of prosody (Scott et al., 2010). Of these acoustic properties, the key element that conveys emotional prosody is changes in pitch i.e. fundamental frequency (Bulut and Narayanan, 2008), at least for non-tone languages, such as English (Ross et al., 1986). Research on the acoustic correlates of attitudinal prosody has come to a similar conclusion, showing that fundamental frequency provides artificial neural networks with enough information to correctly discriminate prosodic attitudes with 83% success (Blanc and Dominey, 2003). Elsewhere, there are important differences in the acoustic correlates of emotional vs. attitudinal prosody; prosodic contour is particularly important for the perception of attitudes, whilst voice quality is important for the perception of emotions (Grichkovtsova et al., 2012). However, depending on context, the same prosodic structures can convey either an attitude or an emotion. For example, the rise-fall structure (falling tone with delayed fundamental frequency peak) can either convey the attitude ‘impressed’ or the emotion ‘surprised’, depending on the context provided by type of clause and the proposition contained (Cruttenden, 1986). Given that a key way in which advancing our knowledge of attitudinal prosody has been hampered is the inconsistent way in which attitudinal prosody is conceptualised and defined, further work on psychoacoustic overlap may prove fruitful. Whilst getting the scientific community to agree and accept a definitive concept may take some time, a simple way to rule out the possibility that attitudinal prosody and emotional prosody are not independent of each other would be to record

actors speaking nonsense words or numbers in designated emotional tones of voice. Two groups of healthy young adults could then listen to these stimuli, one group being instructed to choose the emotion conveyed by intonation. The other group would listen to the intonation of the same stimuli, but would be given a list of attitudes to choose from instead. If there was a large degree of overlap between certain attitudes and emotions (e.g. the emotion disgust and the attitude contemptuous), one may have to conclude that the two are not wholly independent. At the theoretical level, one way in which (basic) prosodic emotions and attitudes have been distinguished is that ‘emotions’ are the involuntary expressions of affects, whilst intentionally controlled prosodic expressions are the so-called ‘social affect’ or ‘attitudes’ (Aubergé and Gestalt, 2002; Crystal, 1979). According to Scherer, emotions are usually expressed in an intense way in response to a highly significant event, and the identification of emotions is largely universal. In contrast, attitudes are more enduring and concern affectively charged beliefs and predispositions. They are less intense and more socially and culturally controlled than emotions (Scherer, 2003; Scherer et al., 2001). However, the terms attitudinal prosody and emotional prosody are sometimes used interchangeably (Blanc and Dominey, 2003; Schmitt et al., 1997; Tompkins and Mateer, 1985), and it has even been commented that “there is no compelling theoretical base for a distinction between attitudes such as indignation and emotions such as fear” (Mozziconacci, 2001). This confusion may partly result from the problematic definitions of ‘attitude’ as applied to intonation though, some of which partially conflate the terms ‘emotion’ and ‘attitude’. Indeed, to paraphrase O’Connor ‘attitudinal prosody is bedevilled by the lack of agreed categories and terms for dealing with attitudes; they are characterised more with an eye to identifying them to the reader than to classifying them in an orderly scheme, and until some method of dealing with attitudes is developed along very much more scientific lines than is possible at present, we shall not be able to tell whether this language and that are similar or different in the number or nature of attitudes they mark (O’Connor, 1973). In this review, we attempt to shed light on the linguistic, psychological and neuroanatomical characteristics of attitudinal prosody and suggest that it is deserved of greater recognition and more focussed research in its own right, at least until our understanding of it is on more of a par with emotional prosody.

3. How does prosody convey attitudes? In the previous section, we saw that an important function of prosody is to convey attitudes. Attitudes are a multi-dimensional construct though, and are often vaguely defined (Altmann, 2008). For the purposes of this review, ‘attitude’ refers to a disposition towards or against a specified phenomenon, person or thing, i.e. the interpersonal stance of the speaker (Pell, 2006a). In the next section attitudes that can be conveyed through prosody are highlighted, before examining how prosodic attitudes are encoded in the acoustic signal and processed in the human brain. Whilst this mode of attitude transmission is not independent of the lexical channel i.e. speech content; (Pell, 2006a), it nevertheless remains a powerful way to transmit attitudes. For example, “that is not the same thing” pronounced with stress and raised pitch on the last word indicates a contradiction; but if the second word is stressed instead, it is more likely to signify an endorsement of what has already been said (Monrad-Kohn, 1963). In certain circumstances, prosodic attitude cues are absolutely vital for a correct understanding of speaker intent. Positive comments spoken in negative tones give subtle clues to speaker attitude not apparent from the words alone, and are only successful if the listener correctly decodes the prosodic

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

cues and substitutes the intended/implied meaning for the literal one. In fact, if the affective prosodic message is at variance with the linguistic message, as in an ironic or sarcastic statement, adults will overwhelmingly believe the affective intent (Ackerman, 1983; Bolinger, 1980; Bowers et al., 1987; Mehrabian, 1972). Research with children aged 5–9 years has shown that they can only do likewise if they possess theory of mind (Filippova and Astington, 2008). Although most research suggests that the time-varying course of pitch is the most important cue to speaker attitude (Blanc and Dominey, 2003), amplitude, loudness and judicial use of pauses may also have an important role (Gelinas-Chebat and Chebat, 1992). For example, high speaker confidence is generally expressed with short and infrequent pauses, increased loudness and speaking rate, and a fall in intonation contour at the end of the utterance. In contrast, low speaker confidence (doubt) tends to be associated with pre-speech delays, rising intonation and elevated pitch (Monetta et al., 2008). At the functional level, prosody conveys attitudes through several interrelated functions. Along this pipeline, we can separate the expression of an attitude on the sender side, the transmission of the sound, and the impression on the receiver side, resulting in attitude inference. Researchers need to assess the complete process to grasp the complex determinants that combine to produce attitude inference, including the attitudinal state expressed, the acoustic characteristics of the vocal cues, the perceptual judgements of the vocal cues, and the process that integrates the cues into a judgement of the encoded attitude (Grandjean et al., 2006). Operating at the sender side someone might formulate an intention to persuade someone else that an activity is worthwhile by systematically manipulating their prosody, and thus indirectly manipulating the impressions formed by receivers. The power of prosodic attitude cues is illustrated by their potential to affect the direction or end result of communicative interactions. Prosodic cues that signal that a speaker is confident in what they are saying, if correctly interpreted, can shape certain aspects of subsequent discourse (Pell, 2006a). For example, a listener may go ahead and purchase a product, having been convinced it works. During a sales presentation, a salesperson would continuously evaluate the impact of what they are saying on the customer, yet at the same time, monitor and communicate their own level of interest in what the customer says. The success of a communicative encounter requires not only the ability to convey one’s own attitude but also the ability to accurately gauge that of the other person. This process does not always work well, particularly when engaging in telephone surveys where face-to-face contact is not possible and the reading of attitudes is solely dependent on the auditory channel of communication. In fact, research has shown that refusals are far greater for telephone compared to face-to-face surveys (Oksenberg et al., 2001), where facial cues and body posture cues are also available. Interestingly, the attitudinal prosody cues people typically report may partially depend on context. For example, in structured interviews Fichten et al. observed that in dating contexts, people tend to monitor carefully for interest cues rather than lack of interest cues, whereas in daily conversations, respondents note more boredom than interest cues (Fichten et al., 1992). To achieve an intended communicative purpose, it can thus be important for a speaker to use prosody to effectively manage the impression being created. Given the importance of understanding communicative intentions (Blair, 2005; Bora et al., 2006), correctly interpreting prosodic attitude cues is likely to be of great importance for psychosocial well-being, as has already been demonstrated for the interpretation of prosodic emotion cues (Blair et al., 2005; Hooker and Park, 2002; McCann and Peppe, 2003). Clinical populations prone to impaired prosodic attitude comprehension could include not only those known for impaired social cognition, but also those with impaired auditory perception or pragmatic language difficulties.

473

Regarding the implementation of prosodic attitude expression, it has been assumed by many that attitudes operate at a conscious level, but it is also apparent that they operate at the unconscious level (Frazier et al., 2006; Greenwald and Banaji, 1995). The effects of prosody may be accidental or intentional, and if intentional, either covertly or overtly so. A speaker’s tone of voice may create an impression of boredom or impatience without the speaker being aware of it. Knowing this, a sophisticated speaker may covertly manipulate their tone of voice to create an apparently accidental impression that he/she might not want to acknowledge as part of his/her meaning (Wilson and Wharton, 2006). This is similar to the concept of display rules mooted for the control of real and apparent facial expressions (Gosselin et al., 2002; Malatesta and Haviland, 1982; Parkinson, 2005). Thus Grice (Grice, 1957, 1969) distinguishes covert and accidental forms of information transmission from overt communication, or speaker’s meaning, where the speaker not only intends to convey a certain message but wants the audience to recognise this intention, and would acknowledge it if asked. A speaker can thus consciously or unconsciously portray the desired attitude to a listener, and in turn, the listener can portray their attitude about the speaker or the topic under discussion back to the speaker. The fact that prosody may be automatically processed by listeners at a level below conscious awareness could even be used to the deliberate advantage of a speaker.

4. Which attitudes can be conveyed by prosody? Now it has been seen how prosody conveys attitudes, we turn to address the kind of attitudes prosody can convey. According to Morlec, the attitudes commonly conveyed by prosody during communicative encounters are (i) declarations; (ii) simple questions; (iii) exclamations of surprise; (iv) doubt-incredulity (partial discord with what was previously expressed); (v) suspicious irony (doubt on the assertions of the speaker); (vi) evidence (profound belief of the speaker in his affirmation) (Morlec et al., 2001). However, surprise is sometimes considered to be a function of emotional rather than attitudinal prosody (Monrad-Kohn, 1947, 1963). Further examples include the use of varying prosodic representations as strategies for communicating the veracity of a statement, the extent to which a speaker wishes to affiliate with a particular idea or individual, or a speaker’s politeness towards a listener when making a request (Pell, 2006a). Yet a partial disparity exists between the prosodic attitudes important in everyday conversational use, and those targeted by focussed scientific research. Much has been written on social attitudes such as hostility/warmth or inferiority/superiority, whilst other attitudes such as boredom/interest have been somewhat neglected (Fichten et al., 1992). At the theoretical level, one of the things that current frameworks emphasise is the manner by which strategies such as changing prosody act to attenuate or boost the force of an implied communication act, thereby communicating the degree of belief, commitment or strength of a speaker’s intentions (Holmes, 1984). Thus attitudinally, vocal intonation can be seen as a ‘vehicle of persuasion’ (Chebat et al., 2007). Yet, scientific research again often overlooks the effects of prosodic cues on the persuasion process (Gelinas-Chebat et al., 1996). What is fairly well-accepted though is that prosody influences persuasion because it is an antecedent of source credibility (Gelinas-Chebat et al., 1996). This is evidenced by questionnaire data on the influence of voice on credibility judgements subsequent to telemarketing bank adverts. In particular, intensity of vocal expressivity and speech rate both affect identification of the credibility of the voice source, i.e. the similarities between the person who transmits the message and the one who receives it, or the extent to which someone accepts the veracity of the message (Chebat et al., 2007). Another mechanism by which

474

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

prosody affects source credibility is via speaker gender. Women often speak with less expressivity than men and with less variation in tones, which often results in listeners associating passiveness, vulnerability and submission with the female voice and dominance, self-assurance and independence to male voices, these traits being closely linked to the success of persuasion (Chebat et al., 2007).

5. The neuroanatomy of attitudinal prosody and its overlap with emotional prosody: clinical and lesion studies Most clinical studies have focused on emotional prosody. Initially, the modulation of affective (emotional and attitudinal) prosody was posited to be a dominant function of the right hemisphere (Heilman et al., 1975; Ross and Mesulam, 1979; Tucker et al., 1977). Although subsequent studies observed that comprehension and production of emotional prosody may be disrupted by lesions in either hemisphere (Adolphs et al., 2002; Cancelliere and Kertesz, 1990; Lima and Castro, 2011; Ross, 1997; Ross and Monnot, 2008), based on the pattern and functional-anatomic correlates of the observed deficits, it has been argued that affective prosody is, a dominant and lateralised right hemisphere function complimentary to the dominant and lateralised verbal-linguistic functions of the left hemisphere (Blonder et al., 1991; Bowers et al., 1993; Ross, 1997, 2010; Ross and Monnot, 2008). In patients with focal left hemisphere lesions, it has been shown using the ‘Aprosodia Battery’ of Ross et al. that incrementally reducing the verbal-articulatory demands of the test stimuli that carry the affective prosody (from a fully articulated sentence, through a monosyllabic sentence “ba.ba.ba.ba.ba.ba”, to an asyllabic vocalic “aaaaaahhhhhhhh”) dramatically improves comprehension and repetition of affective prosody to near normal whereas in patients with focal right hemisphere lesions no improvement is observed (Mitchell and Ross, 2008; Ross, 1997; Ross and Monnot, 2008; Ross et al., 1997). In addition, the cortical location of a right hemisphere lesion is highly predictive of the type of affective-prosodic deficit observed, i.e. frontal opercular lesions disrupt spontaneous affective prosody and affective-prosodic repetition but not affective prosody comprehension, whereas temporal opercular lesions disrupt affective prosody comprehension but not spontaneous affective prosody or repetition. In contrast, the cortical location of a left hemisphere lesion is not predictive of the type of affectiveprosodic deficit observed. In all these studies, only the emotional type of affective prosody was assessed though. Compared to emotional prosody, few clinical studies have formally assessed attitudinal prosody comprehension. One population for which attitudinal prosody comprehension has been probed is healthy older adults. Past research has shown unequivocally that normal ageing impairs the ability of older adults to comprehend emotional prosody (Allen and Brosgole, 1993; Kiss and Ennis, 2001; Mitchell, 2007; Mitchell and Kingston, 2011; Mitchell et al., 2011; Orbelo et al., 2003; Paulmann et al., 2008; Ruffman et al., 2008). Importantly, this deficit is not related to hearing loss or other age-related cognitive deficits (Mitchell, 2007; Orbelo et al., 2005). The pattern of deficit is most consistent with right hemisphere dysfunction because reducing the verbal-articulatory demands of the test stimuli does not improve performance (Orbelo et al., 2003, 2005). Possible right hemisphere mediation of attitudinal prosody is given further credence by complementary evidence from patients with focal right hemisphere brain damage demonstrating impaired ability to infer speaker attitude from prosody relative to healthy controls. Not only have patients with right hemisphere brain damage been shown to have an impaired ability to integrate prosodic cues to politeness attitude with politeness cues from lexico-semantic content (Pell, 2006a), but they also tend to place greater weight on lexico-semantic cues to confidence attitude than

prosodic cues (Pell, 2006b). This single hemispheric dissociation could be strengthened by parallel comparison to the performance of patients with left hemisphere brain damage in future research to demonstrate a double-dissociation, i.e. patients with damage to region ‘X’ being able to identify prosodic attitudes but not emotions, and patients with damage to region ‘Y’ being able to identify prosodic emotions but not attitudes. No such direct comparisons have yet been reported. In their second study, Orbelo et al. used the Ross et al. extended ‘Aprosodia Battery’ to test the additional ability of older adults to detect attitudinal prosody by asking them to determine if test sentences were articulated with a sincere or sarcastic tone of voice (Orbelo et al., 2005). They found that elderly compared to young participants were impaired on this task. Performance of the extended Aprosodia Battery has also been explored in patients with schizophrenia, who similarly show impaired performance on both the emotional (Ross et al., 2001), and attitudinal subtests (Leitman et al., 2006). One means of advancing this field would be to more systematically study the profile of impaired perception of emotional and attitudinal prosody across clinical populations. If at the behavioural level these two cognitions are impaired in tandem in several clinical populations, such results would suggest that emotional and attitudinal prosody share the same processing mechanisms and thus the same neurology. To then go on and determine if the clinically-derived possibility of shared neurology is correct, further research needs to be done by assessing attitudinal and emotional prosody using functional neuroimaging techniques such as functional magnetic resonance imaging (fMRI). However, at present, whilst there are several well-validated corpora of emotional prosody (e.g. the Linguistic Data Consortium ‘Emotional Prosody Speech and Transcripts’; EPST), there is nothing comparable for attitudinal prosody in range and size. Thus to probe the neuroanatomical independence of emotional and attitudinal prosody an equivalent corpus would have to be developed with neutral content being spoken in emotional tones and then again in attitudinal tones, using the same speakers. To add to what is known about interhemispheric lateralisation, the inclusion of right hemisphere patients with more varied intrahemispheric brain damage would enable the association of specific brain structures with impaired attitudinal prosody comprehension. Lesion studies traditionally favour the right hemisphere homologue of Wernicke’s area as being fundamentally ‘necessary’ for decoding prosodic emotion cues (Gorelick and Ross, 1987; Ross, 1981; Ross and Monnot, 2008), even though more recent neuroimaging evidence promotes complementary frontal brain regions as being ‘sufficient’ for advanced cognitions downstream (Schirmer and Kotz, 2006; Wildgruber et al., 2006). If attitudinal prosody comprehension does share its neurology with emotional prosody comprehension, then posterior superior lateral temporal cortex might well be of central importance. The concept that it is specifically the comprehension of emotional prosody that is lateralised to this region in the right hemisphere has colloquially become known as the ‘right hemisphere’ hypothesis (Demaree et al., 2005; Everhart et al., 2006; Killgore and Yurgelun-Todd, 2007; Kotz et al., 2006; Witteman et al., 2011). Van Lancker’s ‘task dependent’ hypothesis went on to add that the comprehension of other types of prosody e.g. linguistic prosody, is lateralised to the left-hemisphere (Behrens, 1985; Van Lancker, 1980). Whilst there have been dissentions in the literature about the reliability of this lateralisation (also see below, and (Kotz et al., 2006)), a recent meta-analysis of 38 lesion studies of emotional prosody comprehension gave overall support for the right hemisphere hypothesis by its demonstration that whilst both left and right hemispheric damage compromise task performance, right hemisphere damage degraded emotional prosody comprehension (Witteman et al., 2011). Thus if attitudinal prosody were closely related to emotional prosody, one might

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

hypothesise its comprehension as involving right posterior superior lateral temporal cortex, whereas if they were not conceptually related, one might expect it to recruit Wernicke’s area in the left hemisphere instead. A third theory, the ‘cue dependent lateralization’ hypothesis proposes that the lateralisation of prosody comprehension is based on the acoustic elements comprising prosody; the left hemisphere being best suited for processing durational cues, and the right hemisphere being best suited for spectral processing (Sidtis and Van Lancker Sidtis, 2003). One particular study illustrates this hypothesis well. When, both left- and right-hemisphere-damaged patients performed poorly at prosodic emotion comprehension, the authors then examined whether these two patient groups were using acoustic cues to prosodic emotion in the same way. The patients with right-hemisphere damage did not make use of fundamental frequency variability, but relied on duration cues to classify emotional prosody, but patients with left-hemisphere damage subjects did make use of fundamental frequency information (Van Lancker and Sidtis, 1992). As variation in pitch is one of the more important acoustic correlates of emotional prosody in non-tone languages such as English (Balaguer-Ballester et al., 2009; Ross et al., 1986, 1988), apparent right hemisphere mediation of emotional prosodic comprehension can be explained on the basis of right lateralisation for pitch processing. If emotional and attitudinal prosody depend on partially overlapping acoustic correlates as reviewed above (Blanc and Dominey, 2003; Bulut and Narayanan, 2008; Pell, 2006a; Scott et al., 2010), one way to probe the applicability of the cue lateralisation theory to both types of prosody would be to systematically examine in which brain regions active during comprehension of emotional and attitudinal prosody does performance of behavioural tests of e.g. pitch and duration perception correlate with the size of neural response. The cue lateralisation hypothesis could generate the test hypothesis that the correlations observed for duration perception would lie in left hemisphere auditory cortex for both emotional and attitudinal prosody, and in right hemisphere auditory cortex for pitch perception. Or one could examine how the acoustic measurements of the prosody stimuli used in classic comprehension studies predicted regional brain activity for the comprehension of both emotional and attitudinal prosody, as has been done for emotional prosody (Wiethoff et al., 2007). The cue dependent lateralisation hypothesis of prosody is related to other acoustic models that have attempted to explain why propositional and affective language functions are lateralised in the brain based on spectral-temporal features. These models including the ‘asymmetric-sampling-in-time’ (Boemio et al., 2005; Poeppel, 2003), ‘double-filtering-by-frequency’ (Ivry and Robertson, 1998) and ‘spectral–temporal’ (Jamison et al., 2006; Obleser et al., 2008; Schirmer and Kotz, 2006; Zatorre and Belin, 2001) hypotheses. However, as pointed out elsewhere, when discussing the neurobiological basis for lateralisation of affective prosody to the right hemisphere, the cue-dependent and spectraltemporal models of lateralisation are not always supported (Ross and Monnot, 2008). Spectrographic analysis of spoken English, a non-tone language, shows that linguistic-phonetic and affectiveprosodic constituents have overlapping temporal and spectral features (Kent and Read, 1992; Lieberman, 1977). For example, formants, which are the basis for vowel production and perception, are cued by the distribution of spectral energy bands that that change over time (Lieberman, 1977) and temporal manipulation of phonemes may be used to convey affective–attitudinal in formation, such as sarcasm and irony (Haverkate, 1990; Rockwell, 2000). Infants do not show hemispheric pre-attunement to specific types of acoustic signals (Dehaene-Lambertz, 2000; Dehaene-Lambertz et al., 2004) as a basis for guiding functional lateralisation. Although the acoustic cues underlying phonemes and affective prosody are

475

distinctly different in tone versus non-tone languages (Edmondson et al., 1987; Ross et al., 1986, 1992, 1988), both types of language have similar lateralisation of the propositional and affective components of communication to the left and right hemispheres respectively, suggesting that the lateralisation is based on the behavioural role the function plays in communication rather than on its acoustic underpinnings. Therefore, the cue dependent lateralisation hypothesis may not prove to be a valid theory for understanding why language functions show hemispheric lateralisation. With specific concern for attitudes rather than emotions, one direct line of support for the involvement of right posterior superior lateral temporal cortex in prosodic attitude comprehension comes from observations that patients with intractable epileptic foci centred in this approximate area show impaired attitudinal prosody comprehension, even when their ability to infer lexico-semantic attitude was unaffected (Tompkins and Mateer, 1985). Beyond, probing which brain regions are required for processing both types of prosody, assessing the similarities of the networks that mediates prosodic emotion comprehension and prosodic attitude comprehension has wider value. From a neurocognitive perspective, if it can be shown that attitudinal prosody is mediated by a different network to emotional prosody, this could imply that they are in fact separate psychological processes. Equally, in clinical studies of attitudinal prosody comprehension, there may be no evidence of impairment in a given population at the behavioural level, the abnormality only becoming apparent when a neuroanatomical study reveals that it is mediated differently to normal. As for emotional prosody (Kotz et al., 2003), neural network models of prosodic attitude differentiation suggest that subcortical regions such as the basal ganglia may also play a key role in prosodic attitude comprehension, in conjunction with their dopaminergic connections to the frontal lobe. In this vein, a temporal recurrent network of leaky integrator neurons originally developed to simulate the neurophysiology of the primate fronto-striatal system, even managed to surpass the human ability to classify attitudes from prosody (82.5% vs. 72.8%) (Blanc and Dominey, 2003). This possibility is amplified by research that shows that patients with Parkinson’s disease, i.e. with basal ganglia pathology, are impaired in judging attitudes from prosodic cues. In separate tests, people with this pathology were less able than healthy adults to differentiate intended levels of speaker confidence, and were less likely to judge utterances as being impolite (Monetta et al., 2008). In fact, an overall role for the basal ganglia in comprehending attitudes generally is made possible by neuroimaging evidence that the basal ganglia may also have a role in comprehending nonprosodic (lexico-semantic) attitudes or ‘expert power’ (Klucharev et al., 2008). The potential use of Parkinson’s disease as a model to probe the contribution of basal ganglia dysfunction to the modulation of prosodic comprehension can be problematic though (Ross, 2010; Ross and Monnot, 2008) because clinical researchers have not appreciated that more than half the dopamine projections that emanate from the nigral–ventral tegmental complex and course through the medial forebrain bundle actually project to non-striatal forebrain structures, including entorhinal, anterior cingulate and dorsolateral frontal cortex (Berger, 1992; Lindvall et al., 1974; Mishra et al., 1975; Scatton et al., 1983; Williams and GoldmanRakic, 1993). Thus deficits in deriving attitudes from prosody in Parkinson patients could in fact arise from loss of cortical dopamine function rather than loss of basal ganglia dopamine. However, since a number of publications have shown that discrete subcortical lesions are associated with deficits in emotional prosody (basal ganglia for production and thalamus for comprehension of emotional prosody (Ross, 2010; Ross and Monnot, 2008; Witteman et al., 2011)), then from the conceptual overlap between emotional and attitudinal prosody, one might extrapolate that attitudinal prosody

476

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

might be similarly affected and also have a partially subcortical basis.

6. The neuroanatomy of attitudinal prosody and its overlap with emotional prosody: neuroimaging studies Using neuroimaging data on emotional prosody comprehension to predict the likely functional neuroanatomy of prosodic attitude comprehension is perhaps more problematic than making predictions from the lesion data. This may partly be due to the greater divergence of results from neuroimaging studies, and partly because as neuroimaging data have shown, it is now apparent that comprehending prosodic gestalts is a complex, multi-stage process (Ethofer et al., 2008; Hesling et al., 2005; Schirmer and Kotz, 2006). Early neuroimaging studies of prosodic emotion comprehension did provide some support for the involvement of right posterior superior lateral temporal cortex as suggested by lesion studies (Buchanan et al., 2000; Mitchell et al., 2003), but right from the start, they also evidenced the involvement of unexpected frontal lobe regions too (Alba-Ferrara et al., 2012; George et al., 1996). Connectivity analyses went on to suggest that the neuroanatomical model that best explained the data was one in which right temporal cortex regions served as the input regions, with the transfer of information from temporal to frontal cortex occurring in parallel pathways in each hemisphere (Ethofer et al., 2006). This model has now also been supported by Diffusion Tensor Imaging (Ethofer et al., 2012). If there are differences in the functional neuroanatomy of emotional and attitudinal prosody, it can be hypothesised that they are likely to be frontal-lobe based, perhaps in the inferior frontal or orbitofrontal regions. Current neuroimaging models of prosodic emotion perception propose a three-stage process going from sensory processing in the auditory cortex, through integration of emotionally significant cues in lateral superior temporal cortex and the superior temporal sulcus, to the final stage, that of evaluative judgement (Schirmer and Kotz, 2006; Wildgruber et al., 2006). Given the assumption that the first two stages are likely to be shared processing stages common to prosodic gestalts irrespective of prosody type, it is likely to be the final processing stages in the inferior frontal and orbitofrontal regions where processing mechanisms diverge based on prosody type. Overall, a recent meta-analysis of neuroimaging data deduced a bilateral temporofrontal network as supporting emotional prosody comprehension, with relative right-lateralisation across the 21 studies reviewed in (early) auditory processing regions, suggesting that the right hemisphere specialisation for emotional prosody comprehension often reported in lesion studies might well be driven by hemispheric specialisation for non-prosody-specific fundamental acoustic information. This lends support to our hypothesis that attitudinal prosody likely overlaps with emotional prosody in the early auditory cortex processing stages that require similar decoding of basic auditory cues. With prosodic emotion and attitude stimuli carefully matched for identification difficulty, the relative recruitment of any specific brain region could be directly compared for the two types of prosody with neuroimaging region of interest analyses (Poldrack, 2007). One even closer clue to the likely functional neuroanatomy of attitudinal prosody perception comes from a recent neuroimaging study of emotional prosody perception that attempted to distinguish between simple or basic emotions such as happiness, sadness and anger, compared to complex (sometimes called social) emotions such as pride, guilt and boredom (Alba-Ferrara et al., 2011). Simple emotions are thought to have evolved for their “their adaptive value in dealing with fundamental life tasks” (Ekman, 1992), and involve limited cognitive processing (Zinck, 2008). In contrast, complex emotions, especially social emotions such as

pride and guilt, require the interpretation of social intentions (van Hooren et al., 2008), consideration of other people, comprehension of social norms and recognition of personal responsibility for the consequences of a situation (Bauminger et al., 2005). They thus require the monitoring of attitudes and opinions of others regarding our own behaviour (Capps et al., 1992). Using numbers as neutral semantic content to carry the prosody, the functional neuroanatomy mediating the identification from prosody of the two groups of emotions was compared. Emotional trials in general vs. neutral trials, activated a network of temporal and lateral frontal brain regions, whilst complex emotion trials in particular showed additional involvement of medial prefrontal cortex, frontal operculum and left insula. We might extrapolate from these findings the hypothesis that perception of prosodic attitudes might also be more likely (vs. emotional prosody comprehension) to necessitate the involvement of medial prefrontal cortex in the limbic system. The judgement of authenticity of prosodic emotions has also been linked to the medial prefrontal cortex, and was explained by the authors as reflecting the involvement of theory of mind (Drolet et al., 2011). Comprehending attitudes from prosody might well involve authenticity judgements, particularly in a marketing context, and theory of mind processes in the brain may help the listener determine Grice’s ‘speaker meaning’ from prosody (Beaucousin et al., 2006). If one accepts attitudes as a complex version of the basic emotions, the general principle that the neuroanatomy of prosodic gestalt processing might depend on the specific emotion (Ethofer et al., 2009), would support the concept of differential higher order processing mechanisms at the later stages where individual valence or attitudes are identified and ‘labelled’. The involvement of other limbic regions such as the amygdala also necessitate exploration in relation to attitudinal prosody. Prosodic cue salience or expressivity was manipulated in one particular fMRI study of emotional prosody comprehension with a parametric design that varied the degree of cue salience by altering the level of acoustic cues typically associated with the expression of certain emotions (Leitman et al., 2010). Although the amygdala does not appear in lesion studies to be neuropsychologically necessary for affective prosody comprehension, it may still be neuropsychologically ‘sufficient’ such that its activation is a correlate of affective prosody comprehension (Adolphs and Tranel, 1999). In addition to the posterior temporal cortex activity typically associated with the early acoustic processing stages, the level of amygdala activity correlated positively with salience. The authors hypothesised that this finding related to the increased arousal engendered by cue salience. Based on the more personal nature of the mental state of mind conveyed by attitudes (Pell, 2006a), it is possible that the more forceful the attitudinal message expressed through prosody, the more likely one is to observe amygdala activation in fMRI studies of attitudinal prosody comprehension. This possibility is also supported by the demonstration that the ability to detect social relevance correlated with amygdala activation during another fMRI study of emotional prosody comprehension (Schirmer et al., 2008).

7. Conclusion In this review, we have seen that prosodic aspects of speech such as pitch can serve as a powerful indicator of a speaker’s attitudinal stance, and thus their true communicative intent. Understanding prosodic attitudes is a key element of successful social cognition, and can shape the actions and discourse of both the sender and receiver. Relative to the use of prosody to convey emotions, much less is known about attitudinal prosody, a better understanding of its neuroanatomical basis being

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

particularly important. Based on the direct and indirect evidence reviewed in this article, a prediction of the functional neuroanatomy of attitudinal prosody comprehension is made for future testing in which the posterior superior lateral temporal cortex is crucial, in conjunction with the top-down involvement of (i) limbic system regions such as the medial prefrontal cortex and amygdala, and (ii) subcortical regions such as the basal ganglia. Of the three hypothesised stages of affective prosody processing, it is argued that there is less likely to be common activation of the inferior and orbitofrontal regions with prosodic emotion comprehension.

References Ackerman, B.P., 1983. Form and function in children’s understanding of ironic utterances. Journal of Experimental Child Psychology 35, 487–508. Adolphs, R., Damasio, H., Tranel, D., 2002. Neural systems for recognition of emotional prosody: a 3-D lesion study. Emotion 2, 23–51. Adolphs, R., Tranel, D., 1999. Intact recognition of emotional prosody following amygdala damage. Neuropsychologia 37, 1285–1292. Alba-Ferrara, L., Ellison, A., Mitchell, R.L.C., 2012. Decoding emotional prosody: resolving differences in functional neuroanatomy from fMRI and lesion studies using TMS. Brain Stimulation 5, 347–353. Alba-Ferrara, L., Hausmann, M., Mitchell, R.L., Weis, S., 2011. The neural correlates of emotional prosody comprehension: disentangling simple from complex emotion. PLoS One 6, e28701. Allen, R., Brosgole, L., 1993. Facial and auditory affect recognition in senile geriatrics, the normal elderly and young adults. International Journal of Neuroscience 68, 33–42. Altmann, T.K., 2008. Attitude: a concept analysis. Nursing Forum 43, 144–150. Aubergé, V., Gestalt, A., 2002. Morphology of prosody directed by functions, Speech Prosody , Aix en Province, France, pp. 151–154. Balaguer-Ballester, E., Clark, N.R., Coath, M., Krumbholz, K., Denham, S.L., 2009. Understanding pitch perception as a hierarchical process with top-down modulation. PLoS Computational Biology 5, e1000301. Barath, A., Cannell, C.F., 1976. Effect of interviewer’s voice intonation. Public Opinion Quarterly 40, 370–373. Bauminger, N., Edelsztein, H.S., Morash, J., 2005. Social information processing and emotional understanding in children with LD. Journal of Learning Disabilities 38, 45–61. Beaucousin, V., Lacheret, A., Turbelin, M.R., Morel, M., Mazoyer, B., Tzourio-Mazoyer, N., 2006. FMRI study of emotional speech comprehension. Cerebral Cortex 17, 339–352. Behrens, S.J., 1985. The perception of stress and lateralization of prosody. Brain and Language 26, 332–348. Berger, B., 1992. Dopaminergic innervation of the frontal cerebral cortex. Evolutionary trends and functional implications. Advances in Neurology 57, 525–544. Blair, R.J., 2005. Responding to the emotions of others: dissociating forms of empathy through the study of typical and psychiatric populations. Consciousness and Cognition 14, 698–718. Blair, R.J., Budhani, S., Colledge, E., Scott, S., 2005. Deafness to fear in boys with psychopathic tendencies. Journal of Child Psychology and Psychiatry 46, 327–336. Blanc, J.M., Dominey, P.F., 2003. Identification of prosodic attitudes by a temporal recurrent network. Cognitive Brain Research 17, 693–699. Blonder, L.X., Bowers, D., Heilman, K.M., 1991. The role of the right hemisphere in emotional communication. Brain 114, 1115–1127. Boemio, A., Fromm, S., Braun, A., Poeppel, D., 2005. Hierarchical and asymmetric temporal sensitivity in human auditory cortices. Nature Neuroscience 8, 389–395. Bolinger, D., 1980. Language: The Loaded Weapon. Longman Group, New York. Bora, E., Eryavuz, A., Kayahan, B., Sungu, G., Veznedaroglu, B., 2006. Social functioning, theory of mind and neurocognition in outpatients with schizophrenia; mental state decoding may be a better predictor of social functioning than mental state reasoning. Psychiatry Research 145, 95–103. Borod, J.C., 1993. Cerebral mechanisms underlying facial, prosodic, and lexical emotional expression: a review of neuropsychological studies and methodological issues. Neuropsychology 7, 445–463. Boss, B.J., 1996. Pragmatics: right brain communication. Axone 17, 81–85. Bowers, D., Bauer, R.M., Heilman, K.M., 1993. The nonverbal affect lexicon: theoretical perspectives from neuropsychological studies of affect perception. Neuropsychology 7, 433–444. Bowers, D., Coslett, H.B., Bauer, R.M., Speedie, L.J., Heilman, K.M., 1987. Comprehension of emotional prosody following unilateral hemispheric lesions: processing defect versus distraction defect. Neuropsychologia 25, 317–328. Buchanan, T.W., Lutz, K., Mirzazade, S., Specht, K., Shah, N.J., Zilles, K., Jancke, L., 2000. Recognition of emotional prosody and verbal components of spoken language: an fMRI study. Cognitive Brain Research 9, 227–238. Bulut, M., Narayanan, S., 2008. On the robustness of overall F0-only modifications to the perception of emotions in speech. Journal of the Acoustical Society of America 123, 4547–4558. Cancelliere, A.E., Kertesz, A., 1990. Lesion localization in acquired deficits of emotional expression and comprehension. Brain and Cognition 13, 133–147.

477

Capps, L., Yirmiya, N., Sigman, M., 1992. Understanding of simple and complex emotions in non-retarded children with autism. Journal of Child Psychology and Psychiatry, and Allied Disciplines 33, 1169–1182. Chebat, J.C., El Hedhli, K., Gelinas-Chebat, C., Boivin, R., 2007. Voice and persuasion in a banking telemarketing context. Perceptual and Motor Skills 104, 419–437. Cruttenden, A., 1986. Intonation. Cambridge University Press, Cambridge. Crystal, D., 1979. Prosodic development. In: Fletcher, P., Garman, M. (Eds.), Language Acquisition. Cambridge University Press, Cambridge, pp. 33–48. Dehaene-Lambertz, G., 2000. Cerebral specialization for speech and non-speech stimuli in infants. Journal of Cognitive Neuroscience 12, 449–460. Dehaene-Lambertz, G., Pena, M., Christophe, A., Landrieu, P., 2004. Phoneme perception in a neonate with a left sylvian infarct. Brain and Language 88, 26–38. Demaree, H.A., Everhart, D.E., Youngstrom, E.A., Harrison, D.W., 2005. Brain lateralization of emotional processing: historical roots and a future incorporating dominance. Behavioral and Cognitive Neuroscience Reviews 4, 3–20. Drolet, M., Schubotz, R.I., Fischer, J., 2011. Authenticity affects the recognition of emotions in speech: behavioral and fMRI evidence. Cognitive, Affective, and Behavioral Neuroscience 12, 140–150. Edmondson, J.A., Chan, J.L., Seibert, G.B., Ross, E.D., 1987. The effect of right-brain damage on acoustical measures of affective prosody in Taiwanese patients. Journal of Phonetics 15, 219–233. Ekman, P., 1992. Are there basic emotions? Psychological Review 99, 550–553. Ethofer, T., Anders, S., Erb, M., Herbert, C., Wiethoff, S., Kissler, J., Grodd, W., Wildgruber, D., 2006. Cerebral pathways in processing of affective prosody: a dynamic causal modeling study. NeuroImage 30, 580–587. Ethofer, T., Bretscher, J., Gschwind, M., Kreifelts, B., Wildgruber, D., Vuilleumier, P., 2012. Emotional voice areas: Anatomic location, functional properties, and structural connections revealed by combined fMRI/DTI. Cerebral Cortex 22, 191–200. Ethofer, T., Kreifelts, B., Wiethoff, S., Wolf, J., Grodd, W., Vuilleumier, P., Wildgruber, D., 2008. Differential influences of emotion, task, and novelty on brain regions underlying the processing of speech melody. Journal of Cognitive Neuroscience 21, 1255–1268. Ethofer, T., Van De Ville, D., Scherer, K., Vuilleumier, P., 2009. Decoding of emotional information in voice-sensitive cortices. Current Biology 19, 1–6. Everhart, D.E., Demaree, H.A., Shipley, A.J., 2006. Perception of emotional prosody: moving toward a model that incorporates sex-related differences. Behavioral and Cognitive Neuroscience Reviews 5, 92–102. Fichten, C.S., Tagalakis, V., Judd, D., Wright, J., Amsel, R., 1992. Verbal and nonverbalcommunication cues in daily conversations and dating. Journal of Social Psychology 132, 751–769. Filippova, E., Astington, J.W., 2008. Further development in social reasoning revealed in discourse irony understanding. Child Development 79, 126–138. Frazier, L., Carlson, K., Clifton Jr., C., 2006. Prosodic phrasing is central to language comprehension. Trends in Cognitive Sciences 10, 244–249. Gelinas-Chebat, C., Chebat, J.-C., 1992. Effects of two voice characteristica on the attitudes toward advertising messages. Journal of Social Psychology 132, 447–459. Gelinas-Chebat, C., Chebat, J.C., Vaninsky, A., 1996. Voice and advertising: effects of intonation and intensity of voice on source credibility, attitudes toward the advertised service and the intent to buy. Perceptual and Motor Skills 83, 243–262. George, M.S., Parekh, P.I., Rosinsky, N., Ketter, T.A., Kimbrell, T.A., Heilman, K.M., Herscovitch, P., Post, R.M., 1996. Understanding emotional prosody activates right hemisphere regions. Archives of Neurology 53, 665–670. Gorelick, P.B., Ross, E.D., 1987. The aprosodias: further functional-anatomical evidence for the organisation of affective language in the right hemisphere. Journal of Neurology, Neurosurgery and Psychiatry 50, 553–560. Gosselin, P., Warren, M., Diotte, M., 2002. Motivation to hide emotion and children’s understanding of the distinction between real and apparent emotions. Journal of Genetic Psychology 163, 479–495. Grandjean, D., Banziger, T., Scherer, K.R., 2006. Intonation as an interface between language and affect. Progress in Brain Research 156, 235–247. Greenwald, A.G., Banaji, M.R., 1995. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review 102, 4–27. Grice, G.R., 1957. On the relations between discrimination reaction-time, anxiety, and intelligence. Journal of Abnormal and Social Psychology 55, 146–147. Grice, H.P., 1969. Utterers meaning and intentions. Philosophical Review 78, 147–177. Grice, P., 1989. Studies in the Way of Words. Harvard University Press, Cambridge, Mass. Grichkovtsova, I., Morel, M., Lacheret, A., 2012. The role of voice quality and prosodic contour in affective speech perception. Speech Communication 54, 414–429. Haverkate, H., 1990. A speech act analysis of irony. Journal of Pragmatics 14, 77–109. Heilman, K.M., Scholes, R., Watson, R.T., 1975. Auditory affective agnosia. Disturbed comprehension of affective speech. Journal of Neurology, Neurosurgery and Psychiatry 38, 69–72. Hesling, I., Clement, S., Bordessoules, M., Allard, M., 2005. Cerebral mechanisms of prosodic integration: evidence from connected speech. NeuroImage 24, 937–947. Holmes, J., 1984. Modifying illocutionary force. Journal of Pragmatics 8, 345–365. Hooker, C., Park, S., 2002. Emotion processing and its relationship to social functioning in schizophrenia patients. Psychiatry Research 112, 41–50. Ivry, R., Robertson, L.C., 1998. The Two Sides of Perception. MIT Press, Cambridge. Jamison, H.L., Watkins, K.E., Bishop, D.V., Matthews, P.M., 2006. Hemispheric specialization for processing auditory nonspeech stimuli. Cerebral Cortex 16, 1266–1275.

478

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479

Kent, R.D., Read, C., 1992. The Acoustical Analysis of Speech. Singular Publishing, San Diego. Killgore, W.D.S., Yurgelun-Todd, D.A., 2007. The right-hemisphere and valence hypotheses: could they both be right (and sometimes left)? Social, Cognitive and Affective Neuroscience 2, 240–250. Kiss, I., Ennis, T., 2001. Age-related decline in perception of prosodic affect. Applied Neuropsychology 8, 251–254. Klucharev, V., Smidts, A., Fernandez, G., 2008. Brain mechanisms of persuasion: how ‘expert power’ modulates memory and attitudes. Social Cognitive and Affective Neuroscience 3, 353–366. Kotz, S.A., Meyer, M., Alter, K., Besson, M., von Cramon, D.Y., Friederici, A.D., 2003. On the lateralization of emotional prosody: an event-related functional MR investigation. Brain and Language 86, 366–376. Kotz, S.A., Meyer, M., Paulmann, S., 2006. Lateralization of emotional prosody in the brain: an overview and synopsis on the impact of study design. Progress in Brain Research 156, 285–294. Leitman, D.I., Wolf, D.H., Ragland, J.D., Laukka, P., Loughead, J., Valdez, J.N., Javitt, D.C., Turetsky, B.I., Gur, R.C., 2010. It’s not what you say, but how you say it: a reciprocal temporo-frontal network for affective prosody. Frontiers in Human Neuroscience 4, 19. Leitman, D.I., Ziwich, R., Pasternak, R., Javitt, D.C., 2006. Theory of mind (ToM) and counterfactuality deficits in schizophrenia: misperception or misinterpretation? Psychological Medicine 36, 1075–1083. Lieberman, P., 1977. Speech Physiology and Acoustic Phonetics. Macmillan, New York. Lima, C.F., Castro, S.L., 2011. Emotion recognition in music changes across the adult life span. Cognition and Emotion 25, 585–598. Lindvall, O., Bjorklund, A., Moore, R.Y., Stenevi, U., 1974. Mesencephalic dopamine neurons projecting to neocortex. Brain Research 81, 325–331. Malatesta, C.Z., Haviland, J.M., 1982. Learning display rules: the socialization of emotion expression in infancy. Child Development 53, 991–1003. McCann, J., Peppe, S., 2003. Prosody in autism spectrum disorders: a critical review. International Journal of Language and Communication Disorders 38, 325–350. Mehrabian, A., 1972. Nonverbal Communication. Transaction Publishers, New Brunswick. Mishra, R.K., Demirjian, C., Katzman, R., Makman, M.H., 1975. A dopamine-sensitive adenylate cyclase in anterior limbic cortex and mesolimbic region of primate brain. Brain Research 96, 395–399. Mitchell, R.L.C., 2007. Age-related decline in the ability to decode emotional prosody: primary or secondary phenomenon? Cognition and Emotion 21, 1435–1454. Mitchell, R.L.C., Elliott, R., Barry, M., Cruttenden, A., Woodruff, P.W., 2003. The neural response to emotional prosody, as revealed by functional magnetic resonance imaging. Neuropsychologia 41, 1410–1421. Mitchell, R.L.C., Kingston, R.A., 2011. Is age-related decline in vocal emotion identification an artefact of labelling cognitions? International Journal of Psychological Studies 3, 156–163. Mitchell, R.L.C., Kingston, R.A., Barbosa Boucas, S.L., 2011. The specificity of agerelated decline in prosodic emotion interpretation. Psychology and Aging 26, 406–414. Mitchell, R.L.C., Ross, E.D., 2008. fMRI evidence for the effect of verbal complexity on lateralisation of the neural response associated with decoding prosodic emotion. Neuropsychologia 46, 2880–2887. Monetta, L., Cheang, H.S., Pell, M.D., 2008. Understanding speaker attitudes from prosody by adults with Parkinson’s disease. Journal of Neuropsychology 2, 415–430. Monrad-Kohn, G.H., 1947. The prosodic quality of speech and its disorders (a brief survey from a neurologist’s point of view). Acta Psychiatrica et Neurologica Scandinavica 22, 255–269. Monrad-Kohn, G.H., 1963. The third element of speech: Prosody and its disorders. In: Halpern, L. (Ed.), Problems of Dynamic Neurology. The Department of Nervous Diseases of the Rothschild Hadassah, Jerusalem, pp. 101–118. Morlec, Y., Bailly, G., Aubergé, V., 2001. Generating prosodic attitudes in French: data, model and evaluation. Speech Communication 33, 357–371. Mozziconacci, S.J.L., 2001. Modeling emotion and attitude in speech by means of perceptually based parameter values. User Modeling and User-Adapted Interaction 11, 297–326. Noveck, I.A., Reboul, A., 2008. Experimental pragmatics: a Gricean turn in the study of language. Trends in Cognitive Sciences 12, 425–431. O’Connor, J.D., 1973. Phonetics. Penguin Books, Middlesex. Obleser, J., Eisner, F., Kotz, S.A., 2008. Bilateral speech comprehension reflects differential sensitivity to spectral and temporal features. Journal of Neuroscience 28, 8116–8123. Oksenberg, L., Coleman, L., Cannell, C.F., 2001. Interviewers’ voices and refusal rates in telephone surveys. Public Opinion Quarterly 50, 97–111. Orbelo, D.M., Grim, M.A., Talbott, R.E., Ross, E.D., 2005. Impaired comprehension of affective prosody in elderly subjects is not predicted by age-related hearing loss or age-related cognitive decline. Journal of Geriatric Psychiatry and Neurology 18, 25–32. Orbelo, D.M., Testa, J.A., Ross, E.D., 2003. Age-related impairments in comprehending affective prosody with comparison to brain-damaged subjects. Journal of Geriatric Psychiatry and Neurology 16, 44–52. Parkinson, B., 2005. Do facial movements express emotions or communicate motives? Personality and Social Psychology Review 9, 278–311. Paulmann, S., Pell, M.D., Kotz, S.A., 2008. How aging affects the recognition of emotional speech. Brain and Language 104, 262–269.

Pell, M.D., 2006a. Judging emotion and attitudes from prosody following brain damage. Progress in Brain Research 156, 303–317. Pell, M.D., 2006b. Reduced sensitivity to prosodic attitudes in adults with focal right hemisphere brain damage. Brain and Language 101, 64–79. Poeppel, D., 2003. The analysis of speech in different temporal integration windows: cerebral lateralization as ‘asymmetric sampling International time’. Speech Communication 41, 245–255. Poldrack, R.A., 2007. Region of interest analysis for fMRI. Social, Cognitive and Affective Neuroscience 2, 67–70. Rockwell, P., 2000. Lower, slower, louder: vocal cues of sarcasm. Journal of Psycholinguistic Research 29, 483–495. Ross, E.D., 1981. The aprosodias. Functional-anatomic organization of the affective components of language in the right hemisphere. Archives of Neurology 38, 561–569. Ross, E.D., 1997. Cortical representation of the emotions. In: Timble, M., Cummings, J. (Eds.), Behavioural Neurology. Butterworth-Heinemann, Oxford. Ross, E.D., 2000. Affective prosody and the aprosodias. In: Mesulam, M.-M. (Ed.), Principles of Behavioral and Cognitive Neurology. Oxford University Press, New York, pp. 316–331. Ross, E.D., 2010. Cerebral localization of functions and the neurology of language: fact versus fiction or is it something else? Neuroscientist 16, 222–243. Ross, E.D., Edmondson, J.A., Seibert, G.B., 1986. The effect of affect on various acoustic measures of prosody in tone and non-tone languages: a comparison based on computer analysis of voice. Journal of Phonetics 14, 283–302. Ross, E.D., Edmondson, J.A., Seibert, G.B., Chan, J.L., 1992. Affective exploitation of tone in Taiwanese: an acoustical study of tone latitude. Journal of Phonetics 20, 441–456. Ross, E.D., Edmondson, J.A., Seibert, G.B., Homan, R.W., 1988. Acoustic analysis of affective prosody during right-sided Wada test: a within-subjects verification of the right hemisphere’s role in language. Brain and Language 33, 128–145. Ross, E.D., Mesulam, M.M., 1979. Dominant language functions of the right hemisphere? Prosody and emotional gesturing. Archives of Neurology 36, 144–148. Ross, E.D., Monnot, M., 2008. Neurology of affective prosody and its functionalanatomic organization in right hemisphere. Brain and Language 104, 51–74. Ross, E.D., Orbelo, D.M., Cartwright, J., Hansel, S., Burgard, M., Testa, J.A., Buck, R., 2001. Affective-prosodic deficits in schizophrenia: comparison to patients with brain damage and relation to schizophrenic symptoms [corrected]. Journal of Neurology, Neurosurgery and Psychiatry 70, 597–604. Ross, E.D., Thompson, R.D., Yenkosky, J., 1997. Lateralization of affective prosody in brain and the callosal integration of hemispheric language functions. Brain and Language 56, 27–54. Ruffman, T., Henry, J.D., Livingstone, V., Phillips, L.H., 2008. A meta-analytic review of emotion recognition and aging: implications for neuropsychological models of aging. Neuroscience and Biobehavioral Reviews 32, 863–881. Scatton, B., Javoy-Agid, F., Rouquier, L., Dubois, B., Agid, Y., 1983. Reduction of cortical dopamine, noradrenaline, serotonin and their metabolites in Parkinson’s disease. Brain Research 275, 321–328. Scherer, K.R., 2003. Vocal communication of emotions: a review of research paradigms. Speech Communication 40, 227–256. Scherer, K.R., Banse, R., Wallbott, H.G., 2001. Emotion inferences from vocal expression correlate across languages and cultures. Journal of Cross-Cultural Psychology 32, 76–92. Schirmer, A., Escoffier, N., Zysset, S., Koester, D., Striano, T., Friederici, A.D., 2008. When vocal processing gets emotional: on the role of social orientation in relevance detection by the human amygdala. NeuroImage 40, 1402–1410. Schirmer, A., Kotz, S.A., 2006. Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing. Trends in Cognitive Sciences 10, 24–30. Schmitt, J.J., Hartje, W., Willmes, K., 1997. Hemispheric asymmetry in the recognition of emotional attitude conveyed by facial expression, prosody and propositional speech. Cortex 33, 65–81. Schul, P.L., Lamb, C.W.J., 1982. Decoding nonverbal and vocal communications: a laboratory study. Journal of the Academy of Marketing Science 10, 154–164. Scott, S.K., Sauter, D.A., Eisner, F., Calder, A.J., 2010. Perceptual cues in nonverbal vocal expressions of emotion. Quarterly Journal of Experimental Psychology 63, 2251–2272. Sidtis, J.J., Van Lancker Sidtis, D., 2003. A neurobehavioral approach to dysprosody. Seminars in Speech and Language 24, 93–105. Tompkins, C.A., Mateer, C.A., 1985. Right hemisphere appreciation of prosodic and linguistic indications of implicit attitude. Brain and Language 24, 185–203. Tucker, D.M., Watson, R.T., Heilman, K.M., 1977. Discrimination and evocation of affectively intoned speech in patients with right parietal disease. Neurology 27, 947–950. Turner, S., 2002. Communication still the key. Marketing News 36, 21–22. van Hooren, S., Versmissen, D., Janssen, I., Myin-Germeys, I., a Campo, J., Mengelers, R., van Os, J., Krabbendam, L., 2008. Social cognition and neurocognition as independent domains in psychosis. Schizophrenia Research 103, 257–265. Van Lancker, D., 1980. Cerebral lateralization of pitch cues in the linguistic signal. International Journal of Human Communication 13, 201–277. Van Lancker, D., Sidtis, J.J., 1992. The identification of affective-prosodic stimuli by left- and right-hemisphere-damaged subjects: all errors are not created equal. Journal of Speech and Hearing Research 35, 963–970. Wiethoff, S., Wildgruber, D., Kreifelts, B., Becker, H., Herbert, C., Grodd, W., Ethofer, T., 2007. Cerebral processing of emotional prosody-influence of acoustic parameters and arousal. NeuroImage 39, 885–893.

R.L.C. Mitchell, E.D. Ross / Neuroscience and Biobehavioral Reviews 37 (2013) 471–479 Wildgruber, D., Ackermann, H., Kreifelts, B., Ethofer, T., 2006. Cerebral processing of linguistic and emotional prosody: fMRI studies. Progress in Brain Research 156, 249–268. Williams, S.M., Goldman-Rakic, P.S., 1993. Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cerebral Cortex 3, 199–222. Wilson, D., Wharton, T., 2006. Relevance and prosody. Journal of Pragmatics 38, 1559–1579.

479

Witteman, J., van Ijzendoorn, M.H., van de Velde, D., van Heuven, V.J., Schiller, N.O., 2011. The nature of hemispheric specialization for linguistic and emotional prosodic perception: a meta-analysis of the lesion literature. Neuropsychologia 49, 3722–3738. Zatorre, R.J., Belin, P., 2001. Spectral and temporal processing in human auditory cortex. Cerebral Cortex 11, 946–953. Zinck, A., 2008. Self-referential emotions. Consciousness and Cognition 17, 496–505.