Asymmetry in facial expression of emotions by chimpanzees

Neuropsychologia 40 (2002) 1523–1533

Asymmetry in facial expression of emotions by chimpanzees Samuel Fernández-Carriba a,∗ , Ángela Loeches b , Ana Morcillo b , William D. Hopkins c,d a Language Research Center, Georgia State University, Atlanta, GA, USA Área de Psicobiolog´ıa, Facultad de Psicolog´ıa, Universidad Autónoma de Madrid, Madrid, Spain Living Links Center, Yerkes Regional Primate Research Center, Emory University, Atlanta, GA, USA d Department of Psychology, Berry College, Mount Berry, GA, USA

b c

Received 2 August 2000; received in revised form 1 February 2002; accepted 5 February 2002

Abstract Asymmetries in human facial expressions have long been documented and traditionally interpreted as evidence of brain laterality in emotional behavior. Recent findings in nonhuman primates suggest that this hemispheric specialization for emotional behavior may have precursors in primate evolution. In this study, we present the first data collected on our closest living relative, the chimpanzee. Objective measures (hemimouth length and area) and subjective measures (human judgements of chimeric stimuli) indicate that chimpanzees’ facial expressions are asymmetric, with a greater involvement of the left side of the face in the production of emotional responses. No effect of expression type (positive versus negative) on facial asymmetry was found. Thus, chimpanzees, like humans and some other nonhuman primates, show a right hemisphere specialization for facial expression of emotions. © 2002 Elsevier Science Ltd. All rights reserved. Keywords: Brain asymmetry; Emotions; Facial expressions; Chimeric stimuli; Chimpanzees

1. Introduction The neuropsychological basis of emotional behavior has been investigated for more than three decades in human subjects, with specific attention focused on the issue of brain lateralization in the control and mediation of emotions (for a review, see [11]). Emotion has been understood a biologically based state involving several components: perception, experience, physiological arousal, goal-directed activities and expression [37]. The human literature has typically explored the different processing modes (perception, expression, experience and physiological arousal) as well as different channels at the communication level (facial, prosodic, lexical, gestural and postural) (for a componential theoretical approach to emotional behavior, see [2]). As a result of these studies, two distinct conceptual models of emotion and laterality have emerged. One model proposes that the right hemisphere is dominant or specialized for the processing of all emotions [8,5]. In contrast, the valence theory proposes that there is differential hemispheric involvement as a function of emotional valence, or pleasantness–unpleasantness, and approach–withdrawal ∗ Corresponding author. Present address: Language Research Center, Georgia State University, 3401 Panthersville Road, Decatur, GA 30034, USA. Tel.: +1-404-2445845; fax: +1-404-2445752. E-mail address: [email protected] (S. Fern´andez-Carriba).

[10,11]. In this model, the right hemisphere is more active during negative/withdrawal emotion in human adults, whereas the left hemisphere is more dominant for positive/ approach emotions. With the aim to explore the phylogeny of brain asymmetries in emotional behavior, as well as the phylogeny of other functional asymmetries, attention has shifted very recently to studies in nonhuman primates and other species [6]. As with other reports on neuroanatomical and functional asymmetries in nonhuman primates (for comprehensive reviews, see [6,23]), evidence emerging from research on emotional behavior supports a human-like pattern of lateralization of emotion in nonhuman primates. Specifically, physiological studies and behavioral studies have found a greater involvement of the right hemisphere in emotional activity, at least in the case of negative emotions (for review, see [29]). In the human literature, the facial channel, at a communicative level, is one of the behavioral components of emotion that has been most extensively investigated in relation to brain asymmetries both in terms of production (for a review and metanalysis, see [5]) and perception of emotions [1,30,42,43]. This is not surprising, since the relationship between facial expressions and emotions has been reported for centuries, particularly in the writings of Charles Darwin [15]. Regarding nonhuman primates, although a few studies have measured asymmetries in perception of faces and perception of emotional expressions [17,18,32,33,48,49], only

0028-3932/02/$ – see front matter © 2002 Elsevier Science Ltd. All rights reserved. PII: S 0 0 2 8 - 3 9 3 2 ( 0 2 ) 0 0 0 2 8 - 3

1524

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

two studies have explored facial asymmetries in the production of emotions in monkeys [19,24]. To our knowledge, no study has been conducted on brain asymmetries in the production of facial expressions in great apes. Hauser [19] tested facial asymmetries in the production of four different facial expressions, including the fear grimace, copulation grimace, open mouth threat and ear flap, in 4–19 rhesus monkeys (the number of different expressions per individual is not specified in this study). Hauser [19] reported that the left side of the face began to move first during the production of the fear grimace and the open mouth threat and it maintained the expression longer than the right in the case of the copulation grimace. Moreover, the left side of the face was reported to be more expressive than the right side in the fear grimace (as reflected in the number of skin folds and height of the corner of the mouth). In addition, Hauser [19] created chimerics (composites made of each half of the face paired with its mirror-reversed duplicate) of three images representing the fear grimace expression of three different individuals and asked 43 human subjects to rate which chimeric expression looked more like the original expression. The left–left chimerics were reported to be more expressive than the right–right chimerics by 41 of the 43 human subjects. Whether that was the case for all three images is something that was not reported in this study. Hook-Costigan and Rogers [24] found in marmosets slightly different results compared to those reported by Hauser [19]. Three facial expressions were experimentally elicited in 8–11 marmosets, depending on the expression (ten different faces in each individual). Two expressions had an accompanying vocalization and were referred to as the tisk (characterized as fearful) and the twitter expression (defined as a social contact call). The third expression was simply referred to as the silent fear expression. For each call, the experimenters recorded areas left and right of midline of the mouth to quantify asymmetry in the intensity of the expression. They also recorded the distance from the midline to the side of the mouth as an indicator of asymmetry. For the area measure, a left side asymmetry was found for the fear and tsik expression while a right side asymmetry was found for the twitter expression. For the distance to midline measure, a left side bias was found for the fear and tsik expression but no effect was found for the twitter expression. No single study has explored facial asymmetries in the production of facial expressions in great apes despite the fact that apes, and chimpanzees in particular, would be ideal subjects for these studies, not only due to their close genetic relatedness to humans, but also because they are highly social individuals with a rich repertoire of facial expressions that have been described in detail elsewhere [9,16,35,44–47]. The aim of this study was to address the issue of brain lateralization in the production of emotions by chimpanzees by examining facial asymmetries in spontaneously produced emotional expressions. We hypothesized that the left side of chimpanzees’ faces and hence the right hemisphere will be more involved in the production of

emotional facial expressions, at least for the negative ones, based on previous findings in humans and monkeys.

2. Experiment 1: objective measures 2.1. Methods 2.1.1. Subjects and setting Observations were made on a sample of 36 chimpanzees (Pan troglodytes) from the Yerkes Regional Primate Research Center (YRPRC) (Atlanta, USA) and the Madrid Zoo-Aquarium (Madrid, Spain). Facial expressions were recorded on 10 adults (older than 16 years of age), 8 females and 2 males; 15 subadults (between 7 and 16), 9 females and 6 males; and 11 juveniles (younger than 7), 5 females and 6 males. Twenty seven subjects were housed at the YRPRC Field Station in two different groups with 11 belonging to one group and 16 to the other. Both compounds have an outdoor area approximately 550 m2 and five indoor rooms each about 12.6 m2 . The outdoor area is surrounded by grate walls 6 m tall with an observation tower in one of the corners from where the chimpanzees were videotaped. At the YRPRC, all observations were recorded in the outdoor portion of their home cage. Nine chimpanzees at the Madrid zoo also served as subjects. Their enclosure consists of an outdoor area 200 m2 and four indoor rooms each 10 m2 . All walls are made of concrete and 3 m-tall glass screens, through which chimpanzees’ behavior could be recorded, surround the open area and two of the four indoor rooms. 2.1.2. Procedure The chimpanzees at the Madrid zoo were studied over two 9 month periods (i.e. October 1996 to June 1997, and October 1997 to June 1998). Observations were made three times a week, 2 h each time on average, once in the morning and twice in the afternoon. Observations of the two groups at the Yerkes Field Station took place during July–September 1998, five times a week, three times in the morning and twice in the afternoon, 2 h a day (1 h each group). This resulted in about 600 h of observation in total. At both locations, chimpanzees were observed ad libitum (there was no limitation on what, who or when to observe) and all social interactions that spontaneously took place were recorded with a video camera (Sony SVHS) by one of the authors. That same lone observer, with a 4 year experience with chimpanzees, then reviewed the videotapes and categorized all present facial expressions according to morphological and functional criteria based on Parr et al. [35] description of the chimpanzee repertoire of facial emotions. Parr et al. [35] based their descriptions on other classic ethograms for the study of chimpanzees behavior (for example, [16,44–47]). Five categories of facial expressions (pant-hooting, play face, silent pout, silent bared-teeth display and staring bared-teeth scream face) and a neutral nonemotional category were recorded in nine

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

1525

Fig. 1. Examples of the chimpanzee’s facial expressions. From left to right: pant-hooting, play face, silent pout, silent bared-teeth display, staring bared-teeth scream face and neutral face.

or more subjects in our sample. Fig. 1 shows a visual example of each of these expressions, accompanied by a brief description in Table 1 of each expression in terms of both morphology and function. A second observer, another author, independently categorized 25% of the facial expressions. The interobserver reliability in categorizing the images was 93%, i.e. the categorization made by one and another observer matched in 42 out of 45 images.

Four different observers, the two same authors and two trained observers that were na¨ıve to the study’s hypothesis, took part in the task of selecting the best images out of the video sequences that contained facial expressions (i.e. those in which the chimpanzee was facing the camera directly). Video images were analyzed using frame-by-frame procedures (24 frames/s) and then digitized in a bitmap format using a video capture card (WinView 601). The fact that

Table 1 The chimpanzee’s facial expressions recorded in the present study Expression

Other names

Morphological description

Context

Pant-hooting

Pant-hoot [16,35,44–47]

Lips are pushed forward resulting in a round opening in the mouth. Vocalizations “hoo-hoo” occur.

Play face

Play face [16,47]; relaxed open mouth display [35,46]

Silent pout

Silent pout [35]; pout [47]

Silent bared-teeth display

Grin [16,44]; horizontal bared-teeth expression [45]; silent bared-teeth display [35,46]

Face is relaxed with the mouth open to expose the bottom teeth. The upper lip may be raised slightly exposing the upper teeth. Vocalizations may include a fast rhytmic staccato breathing. Slightly open mouth in which the lips are pushed forward and rounded creating a small aperture. No vocalizations are apparent. Mouth partially open or closed, with the mouth corners and lips retracted laterally, fully exposing the top and bottom teeth.

Used often in long-distance communication, when greeting group members after separation, or to signal the presence of food Individuals also hoot when performing a bluff display. Expression typically given in the context of play or during a friendly approach.

Staring bared-teeth scream face

Staring bared-teeth scream face [35]

Mouth wide or partially open, lips fully withdrawn exposing the teeth. It includes loud harsh, rasping screams.

Given most often by young infants when they are distressed. It is also exhibited by adults when they are begging. Performed most often by nervous, fearful and submissive individuals. It may be given when a more dominant individual is approaching, after being attacked, during reconciliations or when soliciting support from another individual. Most often seen in individuals who have been the focus of an attack, although the tone of this expression is one of protest rather than submission.

1526

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

Fig. 2. Example of the manipulation of an image. The midline of the face is calculated and used as a reference to put the image in an upright position. The distance between the outer corners of the eyes to the midline is also measured.

chimpanzees had been recorded during their natural interactions limited the number of useful frames to usually one in each suitable sequence. If more than one frame was found in the same sequence, the cleanest and most head-on was selected. A criterion used in the selection was to obtain, at least, one image for each subject in each category of emotional expression, and two, where possible. The total number of useable images of facial expressions was 183 including 39 images in the hooting category, 29 images in the play category, 14 images in the pout category, 31 images in the silent bared-teeth, 19 images in scream face and 51 neutral faces. The number of individuals subjects represented in each of the six categories were 22, 18, 9, 20, 11 and 30, respectively. 2.1.3. Quantification of asymmetry in facial expression An objective index of facial asymmetry, based on previously published procedures [24], was employed in this study. This index consisted of the length and area measures of the left and right hemimouth for each facial expression. To obtain these data, a line was drawn on the face of the focal subject between the inner corners of the two eyes using the program Adobe Photoshop 4.0.1. (Adobe Systems Incorporated, San Jose, CA) and its midpoint was calculated. A perpendicular line that split the face into two halves was then drawn at the midpoint (see Fig. 2). The image was then vertically rotated until the midline made a perfect 90◦ angle with the horizontal line. For the hemimouth length, a straight line was drawn from each outer corner of the mouth to the midline (mm). To measure the area, using a freehand tool, a line surrounding the inner side of the mouth perimeter was drawn and the surface calculated (mm2 ). Both area and length were measured using Scion Image (Scion Corporation, Frederick, Maryland) (see Fig. 3). Following the procedure used by Hook-Costigan and Rogers [24], we also tried to prevent the use of portraits that were not absolutely frontal. For this, we calculated the distance between the outer corner of each eye to the midline for all the images using the program Scion Image (see Fig. 2). This served two purposes: First, based on this measure, we could assess the degree of variation on the frontal view of

the subjects depicted in the image. Second, this measure allowed us to compare the relative degree of asymmetry of the mouth compared to the symmetry of the image. 2.1.4. Data analysis For each image, the measure of the left hemimouth was subtracted from the measure of the right hemimouth and divided by the sum of right and left measures (right − left)/(right + left) to derive a facial asymmetry index (FAI). The FAI was based on other lateral bias indices (handedness indices) that some investigators working with nonhuman primates have adopted (for example [26,27]), assuming that brain asymmetry is not a discrete phenomenon but a continuous one. Such FAI was calculated for both the area and length measures, and it allowed us to compare the asymmetries of images with different pixel densities and resolutions and interpret them in the same way (negative values as left asymmetry, positive values as right asymmetry and 0 as symmetry). 1 FAI values were calculated also for the measures of the distance from the outer corners of the eyes to the midline. With respect to the measures of length and area, the first step was to calculate the average FAI in the case of those subjects for which we had more than one image in a category. Therefore, the sample was reduced from 183 images to 120 usable measures from 22 different individuals in the hooting category, 18 individuals in play, 9 in pout, 20 in silent bared-teeth and 30 in the neutral category. In order to prevent the influence of peripheral morphological factors (like the width of each half of a subject’s face) on emotional expressions asymmetries, we subtracted the length FAI calculated for each subject in the neutral category from their corresponding FAI in the rest of the emotion categories. Length FAIs for neutral faces were also used to adjust area FAIs because the neutral faces had their mouths closed, 1 Values of −1 or +1 would be the extreme cases of facial asymmetry in which all the mouth would be respectively in the left and right side of the face, something impossible to obtain, because only frontal images were used.

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

1527

Fig. 3. Examples of length and area measures in the different categories. From left to right: pant-hooting, play face, silent pout, silent bared-teeth display, staring bared-teeth scream face and neutral face.

despite the dropping of the lower jaw when relaxed is fairly common among chimpanzees [16]. Thus, an area measure could have only been derived for the neutral face with great difficulty and a high chance of error. Six subjects were not represented in the neutral category and, accordingly, their FAIs in the emotion categories remained the same, like if their neutral FAI was 0 (this was, indeed, the case in nine neutral expressions analyzed). 2.2. Results An independent observer who was naive to the hypotheses of our study measured hemimouth lengths and areas in 38 different images (10% of the sample in each case). These data were then correlated with the primary observers measures in order to assess inter-rater reliability in the measurement of asymmetries. For both the length (r = 0.65, d.f. = 17, P < 0.01) and area (r = 0.75, d.f. = 17, P < 0.01), significant correlations were found. The FAIs calculated for the distance from the outer corner of each eye to the middle line in each image were analyzed using a one-sample t-test. This was done to determine whether there was significantly more bias toward a greater exposure of one side or the other of the faces that comprised the data set. None of the analyses revealed significant differences, indicating that on average the images were depicting faces that were head-on. Although the left side was slightly overrepresented in the categories of pout, silent

bared-teeth and neutral and the right side in hooting, play and scream face no average reached statistical significance when compared with 0 (P > 0.20 in all cases, see Tables 2 and 3 for test values used in the t-test). However, even this minimal deviation was taken into consideration when analyzing hemimouth asymmetries. Listed in Tables 2 and 3 are the mean FAIs for each expression type and associated t-values. Population-level biases in facial expression were tested using a one-sample t-test and compared not to 0 (perfect symmetry) but to the FAI measure that was derived from the category average for the distances from the outer corner of the eyes to the middle line. The results indicated that length measures in the hooting, silent bared-teeth and scream face categories deviated significantly to the left. With respect to area measures, hooting, play and silent bared-teeth also showed a significant leftward asymmetry. The correlation between the two hemimouth measures (length and area) taken in each image was positive and significant (r = 0.72, N = 132, P < 0.00). In the case of those individuals for whom we had two different images from the same expression category (72 dyads), we correlated the facial asymmetry indexes obtained for the two images. The correlations for both length and area measures, were positive but nonsignificant (r = 0.15, d.f. = 71, n.s. for length and r = 0.05, d.f. = 49, n.s. for area), which suggests a low consistency of facial asymmetry across the two different facial expressions of the same emotion. However, measures 1 and 2 were not significantly different from

1528

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

Table 2 Category averages for the FAIs based on the mouth length N Hooting Play Pout SBTa Scream face Neutral

22 18 9 20 11 30

Mean FAI (mm)

Mean FAI (%)

−0.041991 −0.017953 −0.039614 −0.028045 −0.027733 −0.0061722

52 51 52 51 51 50

S.E.

t (two-tailed)

Test values

0.01575 0.02365 0.03303 0.008189 0.007842 0.005281

−2.892∗∗

0.003543 0.0001915 −0.0062194 −0.0049855 0.003965 −0.0029232

−0.767 −1.011 −2.816∗ −4.042∗∗ −0.615

The percentage of left bias in each facial expression is also given as a means to help to understand the data %FAI = left/(left + right) × 100. Values under 50 represent right bias, 50 represents symmetry and values over 50 represent left asymmetry. a Silent bared-teeth. ∗ P < 0.05. ∗∗ P ≤ 0.01. Table 3 Category averages for the FAIs based on the mouth area N Hooting Play Pout SBTa Scream face

22 18 9 20 11

Mean FAI (mm) −0.066754 −0.077481 −0.042790 −0.047683 −0.057860

Mean FAI (%) 53 54 52 52 53

S.E.

t

Test values (two-tailed)

0.02487 0.03167 0.06340 0.01509 0.02940

−2.826∗∗

0.003543 0.0001915 −0.0062194 −0.0049855 0.003965

−2.452∗ −0.577 −2.830∗ −2.103

Percentage of left bias in each facial expression is given. %FAI = left/(left + right) × 100. Values under 50 represent right bias, 50 represents symmetry and values over 50 represent left asymmetry. a Silent bared-teeth. ∗ P < 0.05. ∗∗ P ≤ 0.01.

each other either, according to a paired t-test (t (71) = 0.93, n.s. for length and t (50) = 0.77, n.s. for area). Finally, the possible influence of demographic characteristics of the chimpanzees (age, sex and handedness) on results from Experiment 1 was analyzed. A Pearson product moment correlation between the age of the chimpanzees, taken as a continuous variable, and both length and area measures was significant only in the case of area FAIs in two categories, play (r = 0.60, P < 0.01) and scream (r = −0.75, P < 0.01). Age was then considered as a covariable in a following analysis of covariance in which sex was the only fixed factor. No significant effect of sex was found in any of the categories for neither length nor area measures. Regarding handedness, 35 of the 36 subjects of the sample received a punctuation (−5, −3, −1, +1, +3 or +5), from extremely left-handed to extremely right-handed, according to their performance in two tasks of food manipulation described elsewhere [28]. Spearman correlations between such handedness punctuations and the length and area measures were not significant in any of the categories. 2.3. Discussion According to our results, the left hemimouth area was larger than the right in the facial expressions of hooting, play and silent bared-teeth; and it extended laterally more than the right in the categories of hooting, silent-bared-teeth

and scream face. These findings are not the result of a greater exposure of the left side of the chimpanzees’ faces in the recorded images, as demonstrated by measures of the distance between the outer corner of the eyes to the middle line of the face. Moreover, these asymmetries are not merely an extension of morphological asymmetries in the chimpanzees’ faces, as demonstrated by an analysis and control for each individual neutral portrait. On the contrary, these facial asymmetries seem to reflect a functional asymmetry in the production of emotional behavior. If the lower portion of the face in chimpanzees, like in humans [3,39], is mainly controlled by the contralateral hemisphere, that means that the right hemisphere is more involved than the left in the production of emotional expressions by chimpanzees. The fact that the left bias was not significant in the pout category in either the length nor the area measure is probably due to the limited representation of the pout expressions in the sample (nine subjects). The limited sample in the scream category (11 subjects) probably accounts for the lack of significance in the area measure. Play, on the other hand, seems fairly well represented (18 subjects), so we must conclude that left facial asymmetry in the play face category did not take place in all components (i.e. hemimouth length) of such emotional expression. However, mouth length might be an appropriate physical feature to measure facial asymmetry only in those displays in which the mouth is drawn horizontally and not so much in those expressions in

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

which the mouth is opened vertically rather than horizontally, like play. Whether this facial asymmetry is consistent within individuals is something that our study cannot address. The observational approach taken to record the chimpanzees’ facial expressions in this study, although it had the advantage of a great ecological validity and the accuracy of the registered emotions, it severely limited the data collection and no repeated measures, rather than a second image for some subjects, could be taken. For similar reasons, the lack of correlation between the measures of the two different images of the same individual within the same expression category cannot be accurately interpreted since they could have been easily taken at different intensity peaks of the expression sequence. Facial expressions rarely persist more than 5 s but they are part of dynamic interactions in which the face moves and transitions from one expression to another and the magnitude of the facial asymmetry varies along the whole sequence in both human [38] and nonhuman primates [24]. On the other hand, the findings described here do not seem to have been influenced by demographic variables like age, sex and handedness. The fact that the oldest subjects exhibited a bigger area on the right side only in the case of the play category, and that the same ones had a bigger left side in scream, are isolated results and, for that reason, difficult to interpret. Although some researchers [25] have reported in chimpanzees an increase as a function of age in the strength of another functional brain asymmetry, handedness, most of the recent studies about facial asymmetries in emotional behavior in human primates have not found systematic patterns according to age [31], sex [4] or handedness [5]. The objective asymmetry index obtained so far provided a measure of the differential involvement of the muscles that control each hemimouth during the production of facial expressions. However, that information is limited to a very particular feature and the extent to which it can be translated into “emotional involvement” of each side of the face is arguable. Because human primates have developed particularly good skills in the course of evolution to interpret information coming from facial stimuli [12] they have been traditionally used as judges in tasks that required an emotional assessment of human faces [4,5] and also rhesus monkeys’ faces [19]. (Having chimpanzees judge the emotional information in human facial expressions has also been done; see [32]). Accordingly, the next step in this study was to obtain an assessment of emotional asymmetry in chimpanzees’ facial expressions by having human subjects assess the intensity of expressions in each half of the face (left versus right). 3. Experiment 2: subjective measures 3.1. Methods 3.1.1. Participants The human sample consisted of 47 college students, 24 females and 23 males, who participated in the task as

1529

part of class requirements. No one had any expertise on chimpanzees’ behavior and all of them were also naive regarding the hypothesis and procedures of this study. 3.1.2. Stimuli In order to prevent subjects’ fatigue, the number of facial expressions was reduced to 110 color images, randomly selecting only one image per chimpanzee and category in the cases where there were two. The images were then cut 1–21 pixels approximately (0.5–10 mm.) above and below the head of the chimpanzee and 0–4 pixels approximately (0–2 mm) at the left and right of their ears. 2 The resulting first shot was enlarged or lessened to homogenize the size of all images to 70 mm ×82 mm. The vertical midline obtained for the objective measure was now used to split the face into two halves. Each half was then duplicated and paired with its mirror-reversed version, which resulted in the creation of two chimeric stimuli, one made of two left halves of the original image (LL) and one right–right chimeric stimulus (RR) (see Fig. 4). The size of the chimeric stimuli was approximately the same as the original images. Thus, the final set of stimuli consisted of 110 pairs of chimeric stimuli created from each original image. The orientation of the dyads was horizontal in order to better adjust them to the shape of a computer screen, and the position (left or right) of the LL and RR chimerics was counterbalanced across the images (i.e. the LL chimeric was on the left 50% of the times). 3.1.3. Procedure The stimuli were displayed and the responses were recorded by a program written in Visual Basic 6.0. Upon arrival at the laboratory, subjects were asked to sign a consent form and subsequently seated in front of the computer. Subjects were individually tested on a PC with a screen (19 in.) and verbally instructed to select the most emotionally intense chimpanzee’s facial expression within each dyad. The program was initiated by the experimenter. At the onset of testing, a window on the computer screen was displayed with three icons positioned on the right of the window. One icon was labeled “click here to display new image” a second one was labeled “click here to clear screen” and the last one was labeled “exit”. The subjects were instructed to click on icon one to initiate testing. Upon clicking icon one, a pair of chimeric stimuli were displayed on the screen. Below each image were two additional dialog boxes labeled “left picture” or “right picture”. The subjects were further instructed to click when ready on the dialog box corresponding to the image that they perceived as being the more intense expression. Therefore, time during which the pair of chimeric stimuli were exposed was unlimited. Upon selecting a dialog box, the subjects were instructed to click on icon two, followed by icon one. The subjects were instructed to continue in this sequence until the program terminated and all the stimuli had been presented. Each dyad 2

The equivalence mm/pixels is 1/2.12598.

1530

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

Fig. 4. Examples of chimerics in the different categories. From left to right: pant-hooting, play face, silent pout, silent bared-teeth display, staring bared-teeth scream face and neutral face. Left–left chimerics are in the top and the right–right ones in the bottom.

was presented only once, unless the subject failed to make an accurate response (i.e. checking either both or neither of the dialog boxes under the images), in which case the same dyad was presented immediately afterwards. The order of presentation followed four different possibilities that were counterbalanced across subjects. 3.2. Results The proportion of trials on which each subject selected the LL chimeric image relative to the RR was calculated for each image and averaged across each expression type. The mean percentage of LL preference for each category within the sample was then calculated. Overall, the majority of the human subjects considered the left–left composites as being the most intense facial expressions. That preference reached statistical significance (i.e. was significantly higher than 50%) in the emotional categories of play, silent bared-teeth and also in the neutral category (see Table 4). We also explored the relationship between the asymmetry found in the physical characteristics of the facial Table 4 Mean percentage of left–left choices per category

Hooting Play Pout SBTa Scream face Neutral ∗ a

N

Mean % of left– left preference

S.E.

t

Test values (two-tailed)

47 47 47 47 47 47

52 58 54 59 53 54

1.9504 1.6143 2.6722 1.5407 2.1417 1.3417

1.113 4.982∗ 1.497 5.842∗ 1.490 3.219∗

50 50 50 50 50 50

P < 0.01. Silent bared-teeth.

expressions (in particular, each hemimouth length and area) and the asymmetry in the emotional intensity assessed by the human judges. With this in mind, a “subjective” facial asymmetry index (SFAI) was calculated for each of the images viewed by the human observers. The new SFAI was based on the number of subjects who had selected the left–left chimeric versus the number that had selected the right–right chimeric, relative to the total number of human subjects, i.e. SFAI = (# of left − # of right)/(# of left and right). Pearson product moment correlations between the SFAIs and the “objective” FAIs obtained for the length (r = 0.10, d.f. = 108, n.s.) and area (r = 0.36, d.f. = 78, P < 0.01) measures of the very same images were positive in both cases but only significant in the case of the area measures. 3.3. Discussion The human judgements of the emotional intensity of the left and right side of the chimpanzees’ facial expressions were consistent with the objective indexes of facial asymmetry in the case of the play and silent bared-teeth categories. Having humans judging the emotional intensity of human chimeric stimuli has been a traditional method to explore facial asymmetries in emotional behavior (see [4] for a review on the human literature). In fact, some authors have suggested that judges may be more sensitive than facial coding procedures in capturing the nuances of human communicative facial expression [7]. The human judges in this study, however, were assessing facial expressions of a different species and one in which they had no familiarity. The low but significant correlation between their judgement and the area measures indicates that, at least, one of the cues they were paying attention to was the size of the

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

chimpanzee mouth. Whether these subjects were taking into account other single features of the chimpanzees’ faces or applying their innate abilities to process human emotions in the assessment of very similar stimuli is a question that remains unanswered. Apparently, humans with no previous chimpanzee experience may be able to correctly identify many chimpanzee facial expressions, as suggested by Ekman’s [13] review of Foley’s [14] classic study. Foley’s experiment had failed to demonstrate the ability of college students to accurately identify six chimpanzee facial expressions, but once Ekman obtained a second categorization of these emotions, found that two had been labeled incorrectly and that the three of the four remaining expressions had been identified by the students significantly above chance levels (Foley did not perform statistics). This would imply that the naive human observers in this study may have understood to some extent chimpanzee emotions and approached their faces in similar ways as they would human faces. Notwithstanding, future studies should not only attempt to replicate Ekman’s findings but also have human judges with a high level of expertise in chimpanzee’s behavior judging chimpanzees’ chimeric facial expressions. Some discussion of the nonsignificant preference (although towards the left half of the chimpanzees’ faces) found in the case of three emotional categories (hooting, pout and scream face) and the significant left preference in the neutral category seems warranted. The limited size of the sample in the pout and scream categories (9 and 11 subjects respectively) might account once again for the lack of significance in the subjective measures despite the presence of a left bias. Hooting, on the other hand, was well represented (22 subjects) and the physical evidence of facial asymmetries very consistent. The hooting category may be unique among the human-rated chimpanzee expressions because it does not resemble any human emotional facial expression either morphologically or functionally, as concluded from studies of homology between human an nonhuman primates [9]. Thus, it could be the case that the human subjects would have failed in their attempt to make a judgement about the intensity of that particular emotion (but not in the other cases) basing it on their skills to process human expressions instead of consciously looking at single features, like the size of the mouth. The same face-processing mechanism could be invoked to explain the results regarding the neutral category. In fact, they replicate what has been traditionally and consistently reported in the case of humans judging humans, i.e. the two halves of the resting face reflect different qualities [40,50]. It is again unlikely that humans simply took into account the length or size of the mouth of the chimpanzees’ expressions to make their judgement. What made them consider the left side of the chimpanzees’ faces as the most emotionally intense is something that future research should address.

1531

4. General discussion To our knowledge, this is the first study to explore and report facial asymmetries in the production of emotions in chimpanzees. Taking advantage of the procedures used by studies with similar objectives, we have demonstrated that the left side of chimpanzees faces is more involved than the right in the production of emotional behavior. Following Hook-Costigan and Rogers [24], we objectively measured physical differences between the left and right side of the chimpanzees’ faces. In addition, we used human judgement capitalizing on the sophisticated skills humans posses to interpret the configurational information in facial features, skills that are believed to be innate in the case of humans judging humans (see, for example, [41]). Both the objective and the subjective indexes of facial asymmetry show that the left side of the chimpanzee’s face, and hence the right hemisphere, is more involved than the left hemisphere in the production of facial expressions of emotion. Although only the silent bared-teeth display was consistently asymmetric on all measures, both judges and area measures found similar patterns of asymmetry in another category, play. In fact, this resembles the findings coming from the human facial asymmetry literature, in which both trained judges and muscle quantification have found that facial expressions are left sided [4]. Notwithstanding, Experiment 1 in our study was not necessarily measuring facial asymmetry similarly as Experiment 2, where human judgements were possibly influenced by the upper 1/3 of the face (under bilateral hemispheric control). Unfortunately, how that upper portion of the chimpanzee face affected results from Experiment 2 cannot be assessed in the present study. Future research should attempt to asses separately the upper and lower halves of chimpanzees’ chimeric faces, like it has been done in human studies [5]. The facial expressions in this study which were left sided according to judges or measurement techniques had both positive and negative valence and half of them were accompanied, to a greater or lesser extent, by vocalizations (all but silent bared-teeth and the neutral category). However, they cannot be simply considered as vocalizations, since their value as visual communicative discrete signals has been documented and recently demonstrated by Parr et al. [34]. These authors showed how chimpanzees were able to distinctly categorize photographs of their own facial expressions (including, among others, hooting, play, silent bared-teeth and scream face). The accompanying vocalizations of such visual signals are, on the other hand, part of the whole communicative behavior, a behavior that has been shown in our study to be mainly controlled by the right hemisphere. Our results are then contradictory with the left hemisphere advantage reported in monkeys processing or discriminating species-specific auditory stimuli [20–22,36] since it would make no evolutionary sense that the perception and production of vocalizations were in opposite hemispheres. In

1532

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533

fact, Hook-Costigan and Rogers’ [24] finding of a left hemisphere superiority in the production of social contact calls by marmosets is consistent with that left hemisphere specialization in species-specific calls. Furthermore, this result is interpreted by the authors as a left hemisphere superiority in the production of a call in which the information about the social identity of the caller is predominant versus a right hemisphere superiority in the fear call, where the communication of the emotional context might be most important. Thus, the distinction between the facial and auditory communicative channels might be diluted in another distinction, referential information versus emotional signals, which has also been suggested by Hauser [19]. However, any attempt to find a general explanation for the existence of functional brain asymmetries in human and nonhuman primates must necessarily consider phylogenetic distance among species. The valence of the emotions doesn’t seem to have had an influence on our results, since the only expression given in a broadly positive social context, play, was also “left-sided” (even though play didn’t reach statistical significance in the length FAI). These results are consistent with the findings coming from the studies on emotional facial expressions in human primates that show that the left hemiface is more involved than the right hemiface in the expression of both positive and negative facial emotions [5], although some data point to the possibility of differential hemispheric involvement as a function of emotional valence. Also in humans, left-sided facial asymmetries have been reported for both posed and spontaneous expressions, despite some suggestion about differences and degree of facial asymmetry between the two conditions [5]. Our study would be thus in the line of those that have shown a greater involvement of the left hemiface in the spontaneous expression of facial emotion. Studies of lateralization of emotions in nonhuman primates have mostly focused on negative emotions and a right hemisphere involvement in emotional behavior has overall been reported [29]. The only study in nonhuman primates that has found an effect of the valence in the lateralization of emotional behavior has been the one by Hook-Costigan and Rogers [24], although this finding may be interpreted in other possible ways (see above). On the other hand, it should not be argue that the right hemisphere has exclusive control over emotional behavior, as both hemispheres must participate and coordinate in the production of the facial expressions. However, the data clearly indicate that the contribution to the signal is not equally distributed. In summary, this study is the first one to have provided empirical support for the hypothesis of a right hemisphere lateralization in the production of emotions by chimpanzees. These results are consistent with those reported by the only study that explored brain lateralization in perception of human emotions in chimpanzees [32] and with the literature on nonhuman primate emotional behavior. Furthermore, they are consistent with the human literature, which suggests that this functional asymmetry, in particular, is homologous in all these species.

Acknowledgements Support for this research was provided by the MEC grant AP96- 20194893 to SFC and the NIH grants NS-29574 and NS-36605 to WDH. We would like to thank the Madrid Zoo-Aquarium and the YRPRC for their collaboration making possible to carry on this study inside their settlements. We also would like to thank Ms. Elva Fernández-Carriba and Mr. Rubén Gómez Soriano for their inestimable collaboration collecting and manipulating the data. Dr. Lisa A. Parr, Dr. Susana Sánchez and Dr. David Washburn also provided helpful comments on the manuscript. References [1] Borod JC. Interhemispheric and intrahemispheric control of emotion: a focus on unilateral brain damage. Journal of Consulting and Clinical Psychology 1992;60:339–48. [2] Borod JC. Emotion and the brain—anatomy and theory. Neuropsychology 1993;7:427–32. [3] Borod JC, Koff E. Asymmetries in affective facial expression: anatomy and behavior. In: Fox N, Davidson R, editors. The psychobiology of affective development. Hillsdale, NJ: Erlbaum, 1984:293–323. [4] Borod JC, Koff E, Yecker S, Santschi C, Schmidt JM. Facial asymmetry during emotional expression: gender, valence, and measurement technique. Neuropsychologia 1998;36:1209–15. [5] Borod JC, Haywood CS, Koff E. Neuropsychological aspects of facial asymmetry during emotional expression: a review of the normal adult literature. Neuropsychology Review 1997;7:41–60. [6] Bradshaw JL, Rogers L. The evolution of lateral asymmetries, language, tool-use and intellect. San Diego: Academic Press, 1993. [7] Buck R. Using FACS versus communication scores to measure spontaneous facial expression of emotion in brain-damaged subjects. Cortex 1990;26:275–80. [8] Campbell R. The lateralization of emotion: a critical review. International Journal of Psychology 1982;17:211–29. [9] Chevalier-Skolnikoff S. Facial expressions of emotions in nonhuman primates. In: Ekman P, editor. Darwin and facial expressions. New York: Academic Press, 1973:11–89. [10] Davidson RJ. Emotion and affective style: hemispheric substrates. Psychological Science 1992;3:39–43. [11] Davidson RJ. Cerebral Asymmetry, emotion and affective style. In: Davidson RJ, Hugdahl K, editors. Brain asymmetry. Cambridge, MA: MIT Press, 1995:361–87. [12] Dopson W, Beckwith B, Tucker DM, Bullard-Bates P. Asymmetry of facial expression in spontaneous emotion. Cortex 1984;20:243–52. [13] Ekman P. Cross-cultural studies of facial expression. Darwin and facial expressions. New York: Academic Press, 1973:169–222. [14] Foley Jr JP. Judgement of facial expression of emotion in the chimpanzee. Journal of Social Psychology 1935;6:31–54. [15] Fridlund AJ. Human facial expression. New York: Academic Press, 1994. [16] Goodall J van Lawick. A preliminary report on expressive movements and communication in the Gombe Stream chimpanzees. In: Jay PC, editor. Primates: studies in adaptation and variability. New York: Holt, Rinehart & Winston, 1968:313–519. [17] Hamilton CR, Vermeire BA. Discrimination of monkey faces by split-brain monkeys. Behavioural Brain Research 1983;9:263–75. [18] Hamilton CR, Vermeire BA. Complimentary hemispheric specialization in monkeys. Science 1988;242:1694–6. [19] Hauser MC. Right hemisphere dominance in the production of facial expression in monkeys. Science 1993;261:475–7.

S. Fern´andez-Carriba et al. / Neuropsychologia 40 (2002) 1523–1533 [20] Hauser MD, Andersson K. Left hemisphere dominance for processing vocalizations in adult, but not infant, rhesus monkeys: field experiments. Proceeding of the National Academy of Sciences 1994;91:3946–8. [21] Hauser MD, Agnetta B, Perez C. Orienting asymmetries in rhesus monkeys: the effect of time-domain changes on acoustic perception. Animal Behaviour 1998;56:41–7. [22] Heffner HE, Heffner RS. Temporal lobe lesions and perception of species-specific vocalizations by macaques. Science 1984;226:75–6. [23] Hiscock M, Kinsbourne M. Phylogeny and ontogeny of cerebral lateralization. In: Davidson RJ, Hugdahl K, editors. Brain asymmetry. Cambridge, MA: MIT Press, 1995:535–78. [24] Hook-Costigan MA, Rogers LJ. Lateralised use of the mouth in production of vocalizations by marmosets. Neuropsychologia 1998;36:1265–73. [25] Hopkins WD. Heritability of hand preference in chimpanzees: evidence from a partial interspecies cross-fostering study. Journal of Comparative Psychology 1999;113:1–7. [26] Hopkins WD, Bard KA. Hemispheric specialization in infant chimpanzees (Pan troglodytes): evidence for a relation with gender and arousal. Developmental Psychobiology 1993;26:219–35. [27] Hopkins WD, Bard KA. Evidence of asymmetries in spontaneous head turning in infant chimpanzees (Pan troglodytes). Behavioral Neuroscience 1995;109:808–12. [28] Hopkins WD, Fernández-Carriba S. The effect of situational factors on hand preferences for feeding in 177 captive chimpanzees (Pan troglodytes). Neuropsychologia 2000;38:403–9. [29] Hopkins WD, Fernández-Carriba S. Laterality in communicative behaviors in nonhuman primates: a critical analysis. In: Rogers LJ, Andrews R, editors. Comparative vertebrate lateralization. New York: Cambridge University Press, 2002. [30] Ley RG, Bryden MP. Hemispheric differences in processing emotions and faces. Brain & Language 1979;7:127–38. [31] Moreno CR, Borod JC, Welkowitz J, Alpert M. Lateralization for the expression and perception of facial emotion as a function of age. Neuropsychologia 1990;28:199–209. [32] Morris RD, Hopkins WD. Perception of human chimeric faces by chimpanzees (Pan troglodytes): evidence for a right hemisphere asymmetry. Brain & Cognition 1993;21:111–22. [33] Overman WH, Doty RW. Hemispheric specialization displayed by man but not macaques in the analysis of faces. Neuropsychologia 1982;20:113–28. [34] Parr LA, Hopkins WD, de Waal FBM. The perception of facial expressions by chimpanzees, Pan troglodytes. Evolution of Communication 1998;2:1–23.

1533

[35] Parr LA, Preuschoft S, de Waal FBM. Research on facial emotion in chimpanzees: 75 years since Kohts. In: de Wahl FBM, editor. Infant chimpanzee, human child. Oxford University Press, New York, 2002, p. 44–6. [36] Petersen MR, Beecher MD, Zoloth SR, Moody DB, Stebbins WC. Neural lateralization of species-specific vocalizations in Japanese macaques (Macaca fuscata). Science 1978;202:324–7. [37] Plutchnik R. Emotions: a general psychoevolutionary theory. In: Scherer KR, Ekman P, editors. Approaches to emotion. Hillsdale, NJ: Erlbaum, 1984:197–219. [38] Richardson CK, Bowers D, Bauer RM, Heilman KM, Leonard CM. Digitizing the moving face during dynamic displays of emotion. Neuropsychologia 2000;38:1028–39. [39] Rinn WB. The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expression. Psychological Bulletin 1984;95:52–77. [40] Sackeim HA, Weiman LA, Forman BD. Asymmetry of the face at rest: size, area and emotional expression. Cortex 1984;20:165–78. [41] Serrano JM, Iglesias J, Loeches A. Visual discrimination and recognition of facial expressions of anger, fear and surprise in 4-to6-month-old infants. Developmental Psychobiology 1992;25:411–25. [42] Silberman EK, Weingartner H. Hemispheric lateralization of functions related to emotion. Brain & Cognition 1986;5:322–53. [43] Tucker DM. Lateral brain function, emotion and conceptualization. Psychological Bulletin 1981;89:19–46. [44] van Hooff J. Facial expressions in higher primates. Symposia of the Zoological Society of London 1962;8:97–125. [45] van Hooff J. The facial displays of the Catarrhine monkeys and apes. In: Morris D, editor. Primate ethology. Chicago, IL: Aldine, 1967:7–68. [46] van Hooff J. A comparative approach to the phylogeny of laughter and smiling. In: Hinde R, editor. Non-verbal communication. Cambridge: Cambridge University Press, 1972:209–41. [47] van Hooff J. A structural analysis of the social behaviour of a semi-captive group of chimpanzees. In: von Cranach M, Vine I, editors. Expressive movement and non-verbal communication. London: Academic Press, 1973:75–162. [48] Vermeire BA, Hamilton CR. Laterality in monkeys discriminating inverted faces. Neuroscience Abstracts 1988;14:1139. [49] Vermeire BA, Hamilton CR. Inversion effect for faces in split-brain monkeys. Neuropsychology 1998;36:1003–14. [50] Yecker S, Borod JC, Brozgold A, Martin C, Alpert M, Welkowitz J. Lateralization of facial emotional expression in schizophrenic and depressed patients. Journal of Neuropsychiatry and Clinical Neurosciences 1999;11:370–9.