- Email: [email protected]
Task relevant effects on the assessment of cerebral specialization for facial emotion
BRAIN
AND
LANGUAGE
10,
51-59 (1980)
Task Relevant Effects on the Assessment of Cerebral Specialization for Facial Emotion EDWARD
C. HANSCH University
AND FRANCIS
of California,
J. PIROZZOLO
Los Angeles
Hemispheric specialization for processing different types of rapidly exposed stimuli was examined in a forced choice reaction time task. Four conditions of recognition were included: tacial emotion, neutral faces, emotional words, and neutral words. Only the facial emotion condition produced a significant visual field advantage (in favor of the left visual field), but this condition did not differ significantly from the neutral face condition’s left visual field superiority. The verbal conditions produced significantly decreased latencies with RVF presentation, while the LVF presentation was associated with decreased latencies on the facial conditions. These results suggested that facial recognition and affective processing cannot be separated as independent factors generating right hemisphere superiority for facial emotion perception, and that task parameters (verbal vs. nonverbal) are important influences upon effects in studies of cerebral specialization.
Support for hemispheric asymmetries in emotional processing comes from. a variety of clinical observations and studies. A study by Gainotti (1972) supported the earlier observations of Goldstein (1939) and DennyBrown, Mayer, and Horenstein (1952) by demonstrating that patients with left hemisphere damage displayed such behaviors as anxiety reactions, outburst of tears, and vocal utterances (catastrophic reaction), while minimization and joking (indifference reaction) were commonly observed behaviors in right hemisphere-impaired patients. While the “indifference reaction” was correlated with neglect phenomenon of visual space in right brain-damaged patients, Gainotti believed that the behavioral effect was partly independent of neglect, and thus reflected a differential role in emotional behavior for the right hemisphere. Wechsler (1973) examined the recall of unilaterally brain-damaged paThis research was supported, in part, by a Biomedical Research Grant to the second author from the United States Public Health Service (Grant RR 07009-12). Address correspondence and reprint requests to Dr. F. J. Firozzolo, GRECC, VA Medical Center, Minneapolis, MN 66517. 51 0093-934x/80/030051-09$02.00/0 Copyright 0 1980 by Academic Rcss. Inc. All rights of reproduction in any form reserved.
52
HANSCH
AND
PIROZZOLO
tients for affective or neutral passages. Patients with right cerebral lesions recalled more of both the emotionally charged and neutral text, yet committed greater numbers of symbolic distortions and substitutions. These symbolic distortions were especially pronounced for the affective text, leading to the suggestion that the right hemisphere might play a more significant role in producing an emotional response. Aphasics and nonaphasics within the left cerebral-damaged group did not differ in the number of symbolic distortions, suggesting that the reduced number of such errors in this group could not be accounted for simply by a loss in capacity to generate speech needed for symbolic distortions. Heilman, &holes, and Watson (1975) found a deficit in the comprehension of the affective tone of tape-recorded sentences in subjects with lesions of the right temperoparietal cortex, relative to patients with left temperoparietal lesions. No such difference between the unilateral lesion groups existed when subjects judged the semantic content of the sentences. In a subsequent experiment, Tucker, Watson, and Heilman (1977) probed into the underlying mechanism for this auditory affective agnosia. By including a condition in which subjects had only to indicate whether the affective tone of successive identical content sentences was the same or different, the authors tested whether the affective agnosia was an associative or discrimination deficit. Patients with right temperoparietal lesions performed significantly poorer in this condition than did those with left brain lesions, suggesting that the deficit was due largely to a fundamental problem in discrimination. Had the original affective agnosia been caused merely by a failure of association of perceived affect to response (pointing to a face displaying the emotion), then performance on this simple same-different matching task should not have shown a disturbance. In another condition, Tucker, Watson, and Heilman examined production of emotional intonation in patients with right temperoparietal dysfunction and in a control group of nonaphasic patients without cerebral damage. Subjects were required to verbalize a previously occurring auditorially presented sentence of neutral affective tone with specified emotional intonation, signalled by an emotional word following the neutral sentences. Those patients with lesions in the right hemisphere performed significantly poorer than did the control group, which supports the notion of a specialized role for the right in the production of emotional responses. Several lines of investigation employing subjects with intact cerebral functioning have provided convergent evidence consistent with clinical studies of brain-damaged populations with regard to right hemisphere specialization for affective processing. Carmen and Nachison (1973), for instance, have demonstrated a slight but significant left ear superiority in the identification of nonverbal human emotional voices. Studies of both brain-damaged and normal subjects suggest that facial recognition is mediated by the right hemisphere (Warrington & James,
CEREBRAL
SPECIALIZATION
FOR FACIAL
EMOTION
53
1967; Hilliard, 1973). Recently, Suberi and McKeever (1977) demonstrated that the addition of emotional expression specifically increased the LVF’s advantage for recognizing faces, suggesting a special role for the right hemisphere in processing emotional faces. Contradictory evidence comes from Berent (1977) who used single unilateral electroconvulsive treatments (ECT) to assess hemispheric specialization for two separate tasks. In Task I, subjects judged which of a set of three faces was most similar to an original; while none of the faces was of the.original person, the correct choice was a face displaying the same emotton as depicted in the original stimulus. Task II also required choosing from three test faces the one most similar to the original, yet differed in that the correct choice consisted of the same person seen in the original facial stimulus. Left ECT resulted in a significant deterioration in performance on Task I (with right ECT having no effect) and right ECT was disruptive to facial recognition in Task II (with left ECT having no effect). Since Task I clearly involved processing of affective expression, the finding that left ECT resulted in a decrement in performance suggests that assessments of cerebral dominance for facial emotion perception must take into account task-dependent parameters. Berent suggested that performance on Task I was more dependent on verbal processes, while Task II was more spatial in nature, accounting for the reversal in lateralization effects between the tasks. The question of cerebral specialization for facial emotion, then, remains a problem since conflicting results have been obtained by changing task demands. By focusing on the categorization of facial expression, rather than on matching-to-sample of emotional and nonemotional faces, the present study attempted to more adequately test the notion of independence of affective processing from facial recognition in producing a right hemisphere superiority effect in a tachistoscopic task. Inclusion of both word and facial stimuli provided an additional measure of the importance of task demands on cerebral lateralization effects and interspersing of verbal and facial stimuli assured that any cerebral asymmetry discovered in the present experiment could not be attributed to an attentional bias (Kinsbourne, 1970). METHOD
Subjects Thirty-six undergraduates (twenty-one male and fifteen female) in an introductory psychology course at the University of California, Los Angeles served as subjects. All subjects were right handed and had normal or corrected to normal vision.
Stimulus
Materials
The stimuli used in both the facial emotion and neutral faces conditions were slides of human facial emotion from the standardized Paul Ekman: U. C. Berkeley set. In the facial emotion condition, slide photographs of happy, angry, and surprised emotional expressions of two female models were reproduced to yield a total of 8 slide stimuli for each emotion. For
54
HANSCH AND PIROZZOLO
the neutral faces condition, a neutral slide photograph of each of three different female models was reproduced eight times, resulting in a total of 24 slides, as in the facial emotion condition. The three females used in this condition were chosen so as to exclude featural differences that were easily verbally encoded such as age, hair length, or hair color. All facial stimuli had a projected image size of 8-cm width by 12-cm height. In the emotional words condition, 8 slide photographs of each of the words HAPPY, ANGRY, and SURPRISED were prepared, while in the neutral words condition 8 slides were produced of each of the.words HOLLY, ALICE, and STEPHANIE. All word slide stimuli had white letters with dark background and consisted entirely of uppercase letters arranged horizontally. The five-letter words subtended a smaller horizontal visual angle than did the nine-letter words when projected, in order to maintain a constant letter size. The projected images of five-letter words measured 4.5~cm width by 1.5-cm height and nine-letter words measured 8 by 1.5 cm. A familiarization session for facial stimuli employed one slide for each of the three facial emotion expressions, and one slide for each of the three neutral faces. In this session, a Kodak Custom Carousel Model 860h slide projector displayed the stimuli. A Ralph Gerbrands Company Model G1176 three field projection tachistoscope with six channel timer exposed the slide stimuli in the main experiment, with one field reserved for display of a red 3-mm fixation dot. Subjects indicated their response on an aluminum response box with labeled “yes” and “no” 9-mm buttons. Response latencies were recorded on a Lafayette Instruments Company Model 54419-A O.OOl-set digital recording clock. A Davis Scientific Instruments control panel recorded the “yes” and “no” responses and reset the clock for each trial, A wooden frame head rest with cushion allowed subjects to maintain steady head position while viewing the stimuli.
Procedure A forced choice reaction time paradigm requiring “same”-“different” judgments was employed for the four main experimental conditions of recognition of facial emotion, neutral faces, emotional words, and neutral words. Subjects were seated 152 cm away from the projection screen with head resting on the cushioned frame. In the two emotional conditions, an oral cue word of “happy,” “angry,” or “surprised” was given to the subjects, while in the neutral conditions the possible cues were “Holly,” Alice,” or “Stephanie.” Two seconds after giving the cue word the experimenter signaled for subjects to direct and maintain their gaze on the centered fixation dot on the screen by the spoken word, “fixate.” Immediately following this, a slide stimulus was exposed to either the LVF or RVF for 50 msec, an exposure interval determined appropriate for the speed-accuracy relationship in previous pilot testing. All stimuli began at 2” horizontal visual angle and extended to no more than 5” of visual angle (for five-letter words only to 3.7”). The vertical visual angle for facial stimuli was 4.5”, while word stimuli subtended a vertical angle of 0.6”. When the exposed stimulus was a word, subjects responded by pressing the “yes” button if this word matched the oral cue word, or by pressing the “no” button if it was different. When the exposed stimulus was apictureoffacial emotions, subjects indicated whether the emotional expression was the same or different from the emotional cue word in a similar fashion. Likewise, when the stimulus was a neutral face subjects responded “same” or “different” according to whether the face matched the neutral cue word (a female proper name). Each subject was given the response box and allowed to become familiar with the “yes”-“no” assignment appropriate to his/her response hand condition for a few minutes before the experimental session. Half of the subjects used the right index finger for “yes” responses, left index finger for “no,” while half were run with the reverse arrangement. Subjects were instructed to respond as quickly as possible but not at the expense of a large number of errors; a practice session of from 5 to 10 trials was employed to establish this criterion satisfactorily and to acquaint the subjects to the task. All subjects participated in two 48-trial blocks, with one block involving emotional cue
CEREBRAL
SPECIALIZATION
FOR FACIAL
EMOTION
55
words and the other neutral cue words. Thus, facial emotion and emotional word condition trials always occurred together and likewise for neutral face and neutral word condition trials, with each block occurring first half of the time. A familiarization session preceded each block, lasting approximately 5 min, with the emotional expressions and neutral faces displayed until subjects felt confident about the identity of the stimuli. Within each of the two 48-trial blocks two 24-&l blocks were created. Utilizing a sampling without replacement randomization technique, each unique combination of four-crossed factors occupied one of the 24-t& slots for each of these smaller blocks. The factors were condition (facial emotion or emotional words) within the emotional 48-trial block, neutral faces or neutral words within the neutral large block, cue word (HAPPY, ANGRY, SURPRISED or HOLLY, ALICE, STEPHANIE depending again on the larger block), visual field of presentation (left or right), and response type (yes or no). A particular combination, such as facial emotion condition-cue happy-left visual field-yes response, then, was represented by 2 trials, once in each 24-t&l block within the larger block of emotional or neutral cue. Each 24-trial block within the larger 48-trial block occurred first half of the time, producing four unique experimental orderings in conjunction with the aforementioned counterbalancing of emotional or neutral cue blocks. Nine subjects received each of the four random-block orderings, with response hand assignment represented approximately equal in each block (either four or five left-hand “yes” subjects appeared in each block ordering). On “no” response trials, when cue word and slide stimulus did not match, the two alternatives for the stimulus were randomly assigned, one to each 24-trial block. A variable intertrial interval, averaging close to 10 set was employed, during which time latencies and correctness of response were recorded.
RESULTS Error Data
Trials on which subjects showed latencies of above 2.5 set were recorded as errors along with incorrect or absent “yes’‘-“no” responses. Errors comprised only 3.3% of the trials and were not associated with a given visual field or experimental condition. These error trials were eliminated from the reaction time analysis described below. Reactjon Time Analysis
A four-way analysis of variance with factors of emotion-nonemotion, verbal-facial, hemisphere, and response type was performed on the response latencies. The first two factors were derived from breaking the four experimental conditions into a 2-by-2 representation, separating out the presence or absence of emotion as a dimension, and the verbal or facial nature of the stimuli as the second factor. By collapsing across the cue word dimension and using trial latencies for each unique factor combination in the two 24-trial blocks, this 2 x 2 x 2 x 2 design tested response latency entries based on six trials unless an error trial was present. Table 1 presents the main effects in this four-way analysis of variance. All main effects except for hemisphere were highly significant (p < .Ol), with decreased latencies for neutral stimulus conditions, verbal stimuli, and “yes” responses responsible for the effects. The 35msec advantage of “yes” response trials over “no” trails was expected, and
56 MAIN
HANSCH
AND
PIROZZOLO
TABLE 1 EFFECTS FOR FOUR-WAY ANALYSIS OF VARIANCE WITH EMOTION-NONEMOTION, VERBAL-FACIAL, HEMISPHERE, AND RESPONSE TYPE INCLUDED AS FACTORS
Source Emot-Nonemot Verbal-Facial Hemisphere Response
df 1 1 1 1
ss 554652.5 765187.5 7ooo.l 214832.2
MS 554652.5 765187.5 7ooo. 1 214832.2
F 14.10* 25.73* 1.29 18.09*
* p < .Ol.
this factor was included only to examine higher-order interactions. Of importance was the significant verbal-facial x hemisphere interaction (F = 6.62, p < .05), due to the verbal conditions displaying a RVF-left hemisphere advantage, while LVF presentation led to decreased latencies for facial stimuli conditions. This hemispheric difference cast doubt on the notion that attentional biases underlie laterality effects and demonstrated the importance of task demands in altering patterns of performance in laterality studies. The emotion-nonemotion x hemisphere interaction failed to produce significance, with no significant latency decrease in the LVF for emotional conditions. An interesting finding was a significant response type x hemisphere interaction (F = 4.84, p < .05), with “yes” responses being associated with decreased latencies for LVF presentation of stimuli, while “no” responses were made faster in the RVF. In order to further investigate whether any differential hemispheric effects occurred with the emotional stimuli, the four experimental conditions of facial emotion, neutral faces, emotional words, and neutral words were treated as a single factor with four levels and crossed with hemisphere and response type in a three-way analysis of variance. The condition x hemisphere interaction was significant at the .05 level (F = 3.20), indicating that these four conditions had differential hemispheric latency patterns. As suggested in the significant verbal-facial x hemisphere interaction in the previous four-way ANOVA, however, the significance of this effect was largely due to laterality differences existing only for the verbal-facial dimension of the four conditions. Figure 1 presents the means for the LVF and RVF in each of the four experimental conditions. The facial emotion, neutral faces, emotional words, and neutral words conditions displayed the following RVF-LVF response latency differentials respectively: +35, +19, -10, and -17 msec. When testing the four conditions for simple main effects of hemisphere, only the facial emotion stimulus condition produced significance (F = 10.84, p < .Ol), with a 35msec advantage for LVF presentation. This condition’s combination of facial and emotional components would have been expected to result in the greatest RVF-LVF latency differential,
CEREBRAL
SPECIALIZATION
PAClAL EMOTION
NE;TRAL FACE8
FOR FACIAL
EMOiIONAL WORDS
EMOTION
57
NEUTRAL WORDS
FIG. 1. Means for left visual field (LVF) and right visual field (RVF) in each of the four experimental conditions.
which it did indeed; however, a direct comparison with the neutral faces condition latency differential failed to reveal a significant difference (F = 32). The addition of affective processing demands did not lead to a significant shift toward right hemisphere superiority. DISCUSSION The obtained significance in the task (verbal or facial) x hemisphere interaction, where LVF presentation of facial stimuli and RVF presentation of verbal stimuli yielded decreased latencies, has a dual implication. Since subjects had no means to develop an expectancy to either verbal or facial stimuli and then render a given visual field more salient through selective hemispheric activation, the visual half field effects, it seems, reflect an underlying cerebral asymmetry. A similar conclusion was reached by Pirozzolo and Rayner (1977) who used bilateral face-word combinations on the same trial. Second, the reversal in direction of cerebral specialization for the processing of words and faces supported Berent’s idea that task demands, such as the verbal-nonverbal nature of processing, play a dominant role in the production of laterality effects. In the present experiment, degree of verbal mediation was at least partially controlled through employment of both facial and word stimuli, with this task factor proving to be the critical one in determining cerebral lateralization effects. Unlike the experiment of Suberi and McKeever, the inclusion of emotional expression in facial stimuli did not lead to a significant LVF latency
58
HANSCH
AND
PIROZZOLO
decrease in recognition time. It is important to note that the present experiment changed the task requirements from facial recognition (matching-to-sample) to perception of emotional expression (categorization), in addition to changing from neutral to affective facial stimuli. This difference from Suberi and McKeever’s finding, however, casts further doubt on the independence of emotional processing from facial recognition in producing a right hemisphere laterality effect, as now both stimuli and processing demands were emotional. Suberi and McKeever, it should be noted, avoided the conclusion that specialization for emotional processing underlies their demonstrated right hemisphere superiority, preferring instead to state simply that the right hemisphere plays a special role in memory for affective faces. Employment of verbal cues in the present experiment may have diminished the LVF advantage somewhat, although no differential attenuation of LVF superiority for these facial conditons would be expected. Thus, this study supported Berent’s finding that shifting from facial recognition to the processing of emotional face stimuli does not lead to enhanced right hemisphere superiority. It appears that the evidence for laterality of facial emotion perception is in contradiction to the large literature on right hemisphere specialization for emotion. An important consideration in the present experiment was the degree to which recognition of the briefly presented stimuli reflected featural analysis and higher-order processes, or more immediate, holistic comparisons of the uncoded visual input in the icon. Bogen (1969) and others have suggested that the right hemisphere specializes in concrete, holistic decisions, while the left hemisphere displays a characteristic serial, analytic mode of processing. The finding that “yes” responses were associated with decreased latencies in the LVF supports the notion that when matched cue and slide stimulus permit a “holistic identity match” of cue memory trace and slide representation in uncoded form, the right hemisphere may selectively utilize this process and show an advantage. Subjects also reported confusion of “angry” and “happy” upon LVF presentation in the emotional word condition, where similar “y” endings would be represented on the retina nearest the fovea. Such confusion would again suggest that the comparison process in the right hemisphere consisted of the matching of raw, uncoded visual input to memory, rather than by a higher-order auditory or semantic coding process. If featural analysis is crucial to emotional abstraction, while facial recognition is accomplished through a more holistic decision process, this might account for the lack of a significant difference between the facial emotion and neutral faces conditions. Further investigation is needed to determine the exact nature of the emotional abstraction process, and whether it requires a type of processing that would attenuate right hemisphere dominance. Variation of the stimulus dimension in a facial emotion task might
CEREBRAL
SPECIALIZATION
FOR FACIAL
EMOTION
59
allow for the more characteristic processing mode of the right hemisphere, and more clearly demonstrate right hemisphere lateralization for emotion. Clinical studies of facial emotion perception with brain-damaged patients may ultimately prove very successful in separating processing style from affective capabilities in assessing the role of the right hemisphere in emotion. REFERENCES Benton, A. L., and Van Allen, M. W. 1%8. Facial recognition in patients with cerebral disease. Cortex, 4, 344-358. Berent, S. 1977. Functional asymmetry of the human brain in the recognition of faces. Neuropsychologia, 15, 829-831. Bogen, J. E. 1969. The other side of the brain: An appositional mind. Bulletin of the Los Angeles Neurological Societies, 34(3), 135-162. Carmen, A., and Nachison, I. 1973. Ear asymmetry and the perception of emotional nonverbal stimuli. Acta Psychologica, 37, 351-357. Denny-Brown, D., Meyer, J. S., and Horenstein, S. 1952. The significance of perceptual rivalry resulting from parietal lesions. Brain, 75, 433-471. Gainotti, G. 1972. Emotional behavior and hemispheric side of the lesion. Cor#e.x, 8,41-55. Goldstein, K. 1939. The organism: A holistic approach to biology, derivedfrompathological data in man. New York: American Books. Heilman, K. M., Scholes, R., and Watson, R. T. 1975. Auditory affective agnosia. Journal
of Neurology,
Neurosurgery,
and Psychiatry,
38, 69-72.
Hilliard, R. D. 1973. Hemisphere laterality effects on a facial recognition task in normal subjects. Cortex, 9, 246-258. Kinsboume, M. 1970. The cerebral basis of lateral asymmetries in attention. Acta Psychologica, 33, 193-201. Pirozzolo, F. J., and Rayner, K. 1977. Hemispheric specialization in reading and word recognition. Brain and Language, 4, 248-271. Suberi, M., and McKeever, W. F. 1977. Differential right hemisphere memory storage of emotional and nonemotional faces. Neuropsychologia, 15, 757-768. Tucker, D. M., Watson, R. T., and Heilman, K. M. 1977. Discrimination and evocation of affectively intoned speech in patients with right parietal disease. Neurology, 27, 947950. Warrington, E. K., and James, M. 1967. An experimental investigation of facial recognition in patients with unilateral cerebral lesions. Cortex, 3, 317-326. Wechsler, A. F. 1973:The effect of organic brain disease on recall of emotionally charged versus neutral narrative texts. Neurology, 23, 130-135.
AND
LANGUAGE
10,
51-59 (1980)
Task Relevant Effects on the Assessment of Cerebral Specialization for Facial Emotion EDWARD
C. HANSCH University
AND FRANCIS
of California,
J. PIROZZOLO
Los Angeles
Hemispheric specialization for processing different types of rapidly exposed stimuli was examined in a forced choice reaction time task. Four conditions of recognition were included: tacial emotion, neutral faces, emotional words, and neutral words. Only the facial emotion condition produced a significant visual field advantage (in favor of the left visual field), but this condition did not differ significantly from the neutral face condition’s left visual field superiority. The verbal conditions produced significantly decreased latencies with RVF presentation, while the LVF presentation was associated with decreased latencies on the facial conditions. These results suggested that facial recognition and affective processing cannot be separated as independent factors generating right hemisphere superiority for facial emotion perception, and that task parameters (verbal vs. nonverbal) are important influences upon effects in studies of cerebral specialization.
Support for hemispheric asymmetries in emotional processing comes from. a variety of clinical observations and studies. A study by Gainotti (1972) supported the earlier observations of Goldstein (1939) and DennyBrown, Mayer, and Horenstein (1952) by demonstrating that patients with left hemisphere damage displayed such behaviors as anxiety reactions, outburst of tears, and vocal utterances (catastrophic reaction), while minimization and joking (indifference reaction) were commonly observed behaviors in right hemisphere-impaired patients. While the “indifference reaction” was correlated with neglect phenomenon of visual space in right brain-damaged patients, Gainotti believed that the behavioral effect was partly independent of neglect, and thus reflected a differential role in emotional behavior for the right hemisphere. Wechsler (1973) examined the recall of unilaterally brain-damaged paThis research was supported, in part, by a Biomedical Research Grant to the second author from the United States Public Health Service (Grant RR 07009-12). Address correspondence and reprint requests to Dr. F. J. Firozzolo, GRECC, VA Medical Center, Minneapolis, MN 66517. 51 0093-934x/80/030051-09$02.00/0 Copyright 0 1980 by Academic Rcss. Inc. All rights of reproduction in any form reserved.
52
HANSCH
AND
PIROZZOLO
tients for affective or neutral passages. Patients with right cerebral lesions recalled more of both the emotionally charged and neutral text, yet committed greater numbers of symbolic distortions and substitutions. These symbolic distortions were especially pronounced for the affective text, leading to the suggestion that the right hemisphere might play a more significant role in producing an emotional response. Aphasics and nonaphasics within the left cerebral-damaged group did not differ in the number of symbolic distortions, suggesting that the reduced number of such errors in this group could not be accounted for simply by a loss in capacity to generate speech needed for symbolic distortions. Heilman, &holes, and Watson (1975) found a deficit in the comprehension of the affective tone of tape-recorded sentences in subjects with lesions of the right temperoparietal cortex, relative to patients with left temperoparietal lesions. No such difference between the unilateral lesion groups existed when subjects judged the semantic content of the sentences. In a subsequent experiment, Tucker, Watson, and Heilman (1977) probed into the underlying mechanism for this auditory affective agnosia. By including a condition in which subjects had only to indicate whether the affective tone of successive identical content sentences was the same or different, the authors tested whether the affective agnosia was an associative or discrimination deficit. Patients with right temperoparietal lesions performed significantly poorer in this condition than did those with left brain lesions, suggesting that the deficit was due largely to a fundamental problem in discrimination. Had the original affective agnosia been caused merely by a failure of association of perceived affect to response (pointing to a face displaying the emotion), then performance on this simple same-different matching task should not have shown a disturbance. In another condition, Tucker, Watson, and Heilman examined production of emotional intonation in patients with right temperoparietal dysfunction and in a control group of nonaphasic patients without cerebral damage. Subjects were required to verbalize a previously occurring auditorially presented sentence of neutral affective tone with specified emotional intonation, signalled by an emotional word following the neutral sentences. Those patients with lesions in the right hemisphere performed significantly poorer than did the control group, which supports the notion of a specialized role for the right in the production of emotional responses. Several lines of investigation employing subjects with intact cerebral functioning have provided convergent evidence consistent with clinical studies of brain-damaged populations with regard to right hemisphere specialization for affective processing. Carmen and Nachison (1973), for instance, have demonstrated a slight but significant left ear superiority in the identification of nonverbal human emotional voices. Studies of both brain-damaged and normal subjects suggest that facial recognition is mediated by the right hemisphere (Warrington & James,
CEREBRAL
SPECIALIZATION
FOR FACIAL
EMOTION
53
1967; Hilliard, 1973). Recently, Suberi and McKeever (1977) demonstrated that the addition of emotional expression specifically increased the LVF’s advantage for recognizing faces, suggesting a special role for the right hemisphere in processing emotional faces. Contradictory evidence comes from Berent (1977) who used single unilateral electroconvulsive treatments (ECT) to assess hemispheric specialization for two separate tasks. In Task I, subjects judged which of a set of three faces was most similar to an original; while none of the faces was of the.original person, the correct choice was a face displaying the same emotton as depicted in the original stimulus. Task II also required choosing from three test faces the one most similar to the original, yet differed in that the correct choice consisted of the same person seen in the original facial stimulus. Left ECT resulted in a significant deterioration in performance on Task I (with right ECT having no effect) and right ECT was disruptive to facial recognition in Task II (with left ECT having no effect). Since Task I clearly involved processing of affective expression, the finding that left ECT resulted in a decrement in performance suggests that assessments of cerebral dominance for facial emotion perception must take into account task-dependent parameters. Berent suggested that performance on Task I was more dependent on verbal processes, while Task II was more spatial in nature, accounting for the reversal in lateralization effects between the tasks. The question of cerebral specialization for facial emotion, then, remains a problem since conflicting results have been obtained by changing task demands. By focusing on the categorization of facial expression, rather than on matching-to-sample of emotional and nonemotional faces, the present study attempted to more adequately test the notion of independence of affective processing from facial recognition in producing a right hemisphere superiority effect in a tachistoscopic task. Inclusion of both word and facial stimuli provided an additional measure of the importance of task demands on cerebral lateralization effects and interspersing of verbal and facial stimuli assured that any cerebral asymmetry discovered in the present experiment could not be attributed to an attentional bias (Kinsbourne, 1970). METHOD
Subjects Thirty-six undergraduates (twenty-one male and fifteen female) in an introductory psychology course at the University of California, Los Angeles served as subjects. All subjects were right handed and had normal or corrected to normal vision.
Stimulus
Materials
The stimuli used in both the facial emotion and neutral faces conditions were slides of human facial emotion from the standardized Paul Ekman: U. C. Berkeley set. In the facial emotion condition, slide photographs of happy, angry, and surprised emotional expressions of two female models were reproduced to yield a total of 8 slide stimuli for each emotion. For
54
HANSCH AND PIROZZOLO
the neutral faces condition, a neutral slide photograph of each of three different female models was reproduced eight times, resulting in a total of 24 slides, as in the facial emotion condition. The three females used in this condition were chosen so as to exclude featural differences that were easily verbally encoded such as age, hair length, or hair color. All facial stimuli had a projected image size of 8-cm width by 12-cm height. In the emotional words condition, 8 slide photographs of each of the words HAPPY, ANGRY, and SURPRISED were prepared, while in the neutral words condition 8 slides were produced of each of the.words HOLLY, ALICE, and STEPHANIE. All word slide stimuli had white letters with dark background and consisted entirely of uppercase letters arranged horizontally. The five-letter words subtended a smaller horizontal visual angle than did the nine-letter words when projected, in order to maintain a constant letter size. The projected images of five-letter words measured 4.5~cm width by 1.5-cm height and nine-letter words measured 8 by 1.5 cm. A familiarization session for facial stimuli employed one slide for each of the three facial emotion expressions, and one slide for each of the three neutral faces. In this session, a Kodak Custom Carousel Model 860h slide projector displayed the stimuli. A Ralph Gerbrands Company Model G1176 three field projection tachistoscope with six channel timer exposed the slide stimuli in the main experiment, with one field reserved for display of a red 3-mm fixation dot. Subjects indicated their response on an aluminum response box with labeled “yes” and “no” 9-mm buttons. Response latencies were recorded on a Lafayette Instruments Company Model 54419-A O.OOl-set digital recording clock. A Davis Scientific Instruments control panel recorded the “yes” and “no” responses and reset the clock for each trial, A wooden frame head rest with cushion allowed subjects to maintain steady head position while viewing the stimuli.
Procedure A forced choice reaction time paradigm requiring “same”-“different” judgments was employed for the four main experimental conditions of recognition of facial emotion, neutral faces, emotional words, and neutral words. Subjects were seated 152 cm away from the projection screen with head resting on the cushioned frame. In the two emotional conditions, an oral cue word of “happy,” “angry,” or “surprised” was given to the subjects, while in the neutral conditions the possible cues were “Holly,” Alice,” or “Stephanie.” Two seconds after giving the cue word the experimenter signaled for subjects to direct and maintain their gaze on the centered fixation dot on the screen by the spoken word, “fixate.” Immediately following this, a slide stimulus was exposed to either the LVF or RVF for 50 msec, an exposure interval determined appropriate for the speed-accuracy relationship in previous pilot testing. All stimuli began at 2” horizontal visual angle and extended to no more than 5” of visual angle (for five-letter words only to 3.7”). The vertical visual angle for facial stimuli was 4.5”, while word stimuli subtended a vertical angle of 0.6”. When the exposed stimulus was a word, subjects responded by pressing the “yes” button if this word matched the oral cue word, or by pressing the “no” button if it was different. When the exposed stimulus was apictureoffacial emotions, subjects indicated whether the emotional expression was the same or different from the emotional cue word in a similar fashion. Likewise, when the stimulus was a neutral face subjects responded “same” or “different” according to whether the face matched the neutral cue word (a female proper name). Each subject was given the response box and allowed to become familiar with the “yes”-“no” assignment appropriate to his/her response hand condition for a few minutes before the experimental session. Half of the subjects used the right index finger for “yes” responses, left index finger for “no,” while half were run with the reverse arrangement. Subjects were instructed to respond as quickly as possible but not at the expense of a large number of errors; a practice session of from 5 to 10 trials was employed to establish this criterion satisfactorily and to acquaint the subjects to the task. All subjects participated in two 48-trial blocks, with one block involving emotional cue
CEREBRAL
SPECIALIZATION
FOR FACIAL
EMOTION
55
words and the other neutral cue words. Thus, facial emotion and emotional word condition trials always occurred together and likewise for neutral face and neutral word condition trials, with each block occurring first half of the time. A familiarization session preceded each block, lasting approximately 5 min, with the emotional expressions and neutral faces displayed until subjects felt confident about the identity of the stimuli. Within each of the two 48-trial blocks two 24-&l blocks were created. Utilizing a sampling without replacement randomization technique, each unique combination of four-crossed factors occupied one of the 24-t& slots for each of these smaller blocks. The factors were condition (facial emotion or emotional words) within the emotional 48-trial block, neutral faces or neutral words within the neutral large block, cue word (HAPPY, ANGRY, SURPRISED or HOLLY, ALICE, STEPHANIE depending again on the larger block), visual field of presentation (left or right), and response type (yes or no). A particular combination, such as facial emotion condition-cue happy-left visual field-yes response, then, was represented by 2 trials, once in each 24-t&l block within the larger block of emotional or neutral cue. Each 24-trial block within the larger 48-trial block occurred first half of the time, producing four unique experimental orderings in conjunction with the aforementioned counterbalancing of emotional or neutral cue blocks. Nine subjects received each of the four random-block orderings, with response hand assignment represented approximately equal in each block (either four or five left-hand “yes” subjects appeared in each block ordering). On “no” response trials, when cue word and slide stimulus did not match, the two alternatives for the stimulus were randomly assigned, one to each 24-trial block. A variable intertrial interval, averaging close to 10 set was employed, during which time latencies and correctness of response were recorded.
RESULTS Error Data
Trials on which subjects showed latencies of above 2.5 set were recorded as errors along with incorrect or absent “yes’‘-“no” responses. Errors comprised only 3.3% of the trials and were not associated with a given visual field or experimental condition. These error trials were eliminated from the reaction time analysis described below. Reactjon Time Analysis
A four-way analysis of variance with factors of emotion-nonemotion, verbal-facial, hemisphere, and response type was performed on the response latencies. The first two factors were derived from breaking the four experimental conditions into a 2-by-2 representation, separating out the presence or absence of emotion as a dimension, and the verbal or facial nature of the stimuli as the second factor. By collapsing across the cue word dimension and using trial latencies for each unique factor combination in the two 24-trial blocks, this 2 x 2 x 2 x 2 design tested response latency entries based on six trials unless an error trial was present. Table 1 presents the main effects in this four-way analysis of variance. All main effects except for hemisphere were highly significant (p < .Ol), with decreased latencies for neutral stimulus conditions, verbal stimuli, and “yes” responses responsible for the effects. The 35msec advantage of “yes” response trials over “no” trails was expected, and
56 MAIN
HANSCH
AND
PIROZZOLO
TABLE 1 EFFECTS FOR FOUR-WAY ANALYSIS OF VARIANCE WITH EMOTION-NONEMOTION, VERBAL-FACIAL, HEMISPHERE, AND RESPONSE TYPE INCLUDED AS FACTORS
Source Emot-Nonemot Verbal-Facial Hemisphere Response
df 1 1 1 1
ss 554652.5 765187.5 7ooo.l 214832.2
MS 554652.5 765187.5 7ooo. 1 214832.2
F 14.10* 25.73* 1.29 18.09*
* p < .Ol.
this factor was included only to examine higher-order interactions. Of importance was the significant verbal-facial x hemisphere interaction (F = 6.62, p < .05), due to the verbal conditions displaying a RVF-left hemisphere advantage, while LVF presentation led to decreased latencies for facial stimuli conditions. This hemispheric difference cast doubt on the notion that attentional biases underlie laterality effects and demonstrated the importance of task demands in altering patterns of performance in laterality studies. The emotion-nonemotion x hemisphere interaction failed to produce significance, with no significant latency decrease in the LVF for emotional conditions. An interesting finding was a significant response type x hemisphere interaction (F = 4.84, p < .05), with “yes” responses being associated with decreased latencies for LVF presentation of stimuli, while “no” responses were made faster in the RVF. In order to further investigate whether any differential hemispheric effects occurred with the emotional stimuli, the four experimental conditions of facial emotion, neutral faces, emotional words, and neutral words were treated as a single factor with four levels and crossed with hemisphere and response type in a three-way analysis of variance. The condition x hemisphere interaction was significant at the .05 level (F = 3.20), indicating that these four conditions had differential hemispheric latency patterns. As suggested in the significant verbal-facial x hemisphere interaction in the previous four-way ANOVA, however, the significance of this effect was largely due to laterality differences existing only for the verbal-facial dimension of the four conditions. Figure 1 presents the means for the LVF and RVF in each of the four experimental conditions. The facial emotion, neutral faces, emotional words, and neutral words conditions displayed the following RVF-LVF response latency differentials respectively: +35, +19, -10, and -17 msec. When testing the four conditions for simple main effects of hemisphere, only the facial emotion stimulus condition produced significance (F = 10.84, p < .Ol), with a 35msec advantage for LVF presentation. This condition’s combination of facial and emotional components would have been expected to result in the greatest RVF-LVF latency differential,
CEREBRAL
SPECIALIZATION
PAClAL EMOTION
NE;TRAL FACE8
FOR FACIAL
EMOiIONAL WORDS
EMOTION
57
NEUTRAL WORDS
FIG. 1. Means for left visual field (LVF) and right visual field (RVF) in each of the four experimental conditions.
which it did indeed; however, a direct comparison with the neutral faces condition latency differential failed to reveal a significant difference (F = 32). The addition of affective processing demands did not lead to a significant shift toward right hemisphere superiority. DISCUSSION The obtained significance in the task (verbal or facial) x hemisphere interaction, where LVF presentation of facial stimuli and RVF presentation of verbal stimuli yielded decreased latencies, has a dual implication. Since subjects had no means to develop an expectancy to either verbal or facial stimuli and then render a given visual field more salient through selective hemispheric activation, the visual half field effects, it seems, reflect an underlying cerebral asymmetry. A similar conclusion was reached by Pirozzolo and Rayner (1977) who used bilateral face-word combinations on the same trial. Second, the reversal in direction of cerebral specialization for the processing of words and faces supported Berent’s idea that task demands, such as the verbal-nonverbal nature of processing, play a dominant role in the production of laterality effects. In the present experiment, degree of verbal mediation was at least partially controlled through employment of both facial and word stimuli, with this task factor proving to be the critical one in determining cerebral lateralization effects. Unlike the experiment of Suberi and McKeever, the inclusion of emotional expression in facial stimuli did not lead to a significant LVF latency
58
HANSCH
AND
PIROZZOLO
decrease in recognition time. It is important to note that the present experiment changed the task requirements from facial recognition (matching-to-sample) to perception of emotional expression (categorization), in addition to changing from neutral to affective facial stimuli. This difference from Suberi and McKeever’s finding, however, casts further doubt on the independence of emotional processing from facial recognition in producing a right hemisphere laterality effect, as now both stimuli and processing demands were emotional. Suberi and McKeever, it should be noted, avoided the conclusion that specialization for emotional processing underlies their demonstrated right hemisphere superiority, preferring instead to state simply that the right hemisphere plays a special role in memory for affective faces. Employment of verbal cues in the present experiment may have diminished the LVF advantage somewhat, although no differential attenuation of LVF superiority for these facial conditons would be expected. Thus, this study supported Berent’s finding that shifting from facial recognition to the processing of emotional face stimuli does not lead to enhanced right hemisphere superiority. It appears that the evidence for laterality of facial emotion perception is in contradiction to the large literature on right hemisphere specialization for emotion. An important consideration in the present experiment was the degree to which recognition of the briefly presented stimuli reflected featural analysis and higher-order processes, or more immediate, holistic comparisons of the uncoded visual input in the icon. Bogen (1969) and others have suggested that the right hemisphere specializes in concrete, holistic decisions, while the left hemisphere displays a characteristic serial, analytic mode of processing. The finding that “yes” responses were associated with decreased latencies in the LVF supports the notion that when matched cue and slide stimulus permit a “holistic identity match” of cue memory trace and slide representation in uncoded form, the right hemisphere may selectively utilize this process and show an advantage. Subjects also reported confusion of “angry” and “happy” upon LVF presentation in the emotional word condition, where similar “y” endings would be represented on the retina nearest the fovea. Such confusion would again suggest that the comparison process in the right hemisphere consisted of the matching of raw, uncoded visual input to memory, rather than by a higher-order auditory or semantic coding process. If featural analysis is crucial to emotional abstraction, while facial recognition is accomplished through a more holistic decision process, this might account for the lack of a significant difference between the facial emotion and neutral faces conditions. Further investigation is needed to determine the exact nature of the emotional abstraction process, and whether it requires a type of processing that would attenuate right hemisphere dominance. Variation of the stimulus dimension in a facial emotion task might
CEREBRAL
SPECIALIZATION
FOR FACIAL
EMOTION
59
allow for the more characteristic processing mode of the right hemisphere, and more clearly demonstrate right hemisphere lateralization for emotion. Clinical studies of facial emotion perception with brain-damaged patients may ultimately prove very successful in separating processing style from affective capabilities in assessing the role of the right hemisphere in emotion. REFERENCES Benton, A. L., and Van Allen, M. W. 1%8. Facial recognition in patients with cerebral disease. Cortex, 4, 344-358. Berent, S. 1977. Functional asymmetry of the human brain in the recognition of faces. Neuropsychologia, 15, 829-831. Bogen, J. E. 1969. The other side of the brain: An appositional mind. Bulletin of the Los Angeles Neurological Societies, 34(3), 135-162. Carmen, A., and Nachison, I. 1973. Ear asymmetry and the perception of emotional nonverbal stimuli. Acta Psychologica, 37, 351-357. Denny-Brown, D., Meyer, J. S., and Horenstein, S. 1952. The significance of perceptual rivalry resulting from parietal lesions. Brain, 75, 433-471. Gainotti, G. 1972. Emotional behavior and hemispheric side of the lesion. Cor#e.x, 8,41-55. Goldstein, K. 1939. The organism: A holistic approach to biology, derivedfrompathological data in man. New York: American Books. Heilman, K. M., Scholes, R., and Watson, R. T. 1975. Auditory affective agnosia. Journal
of Neurology,
Neurosurgery,
and Psychiatry,
38, 69-72.
Hilliard, R. D. 1973. Hemisphere laterality effects on a facial recognition task in normal subjects. Cortex, 9, 246-258. Kinsboume, M. 1970. The cerebral basis of lateral asymmetries in attention. Acta Psychologica, 33, 193-201. Pirozzolo, F. J., and Rayner, K. 1977. Hemispheric specialization in reading and word recognition. Brain and Language, 4, 248-271. Suberi, M., and McKeever, W. F. 1977. Differential right hemisphere memory storage of emotional and nonemotional faces. Neuropsychologia, 15, 757-768. Tucker, D. M., Watson, R. T., and Heilman, K. M. 1977. Discrimination and evocation of affectively intoned speech in patients with right parietal disease. Neurology, 27, 947950. Warrington, E. K., and James, M. 1967. An experimental investigation of facial recognition in patients with unilateral cerebral lesions. Cortex, 3, 317-326. Wechsler, A. F. 1973:The effect of organic brain disease on recall of emotionally charged versus neutral narrative texts. Neurology, 23, 130-135.