- Email: [email protected]
An important role for high-spatial frequencies in recognition of facial expressions
International Congress Series 1278 (2005) 53 – 56
www.ics-elsevier.com
An important role for high-spatial frequencies in recognition of facial expressions Reimi Tsurusawaa,b,*, Yoshinobu Gotoa, Akihisa Mitsudomeb, Shozo Tobimatsua a
Department of Clinical Neurophysiology, Neurological Institute, Graduate School of Medical Sciences, Kyusyu University, 3-1-1 Maidashi, Higashi-ku, Fukuoka 812-8582, Japan b Department of Pediatrics, School of Medicine, Fukuoka University, Fukuoka, Japan
Abstract. We investigated the information processing of facial expression by ERPs to Chernoff’s face. Nine healthy, right-handed adult volunteers served as subjects. The Chernoff’s face is a simple drawing which is made-up of rich high-spatial frequency components. Neutral and angry faces were determined psychophysically by changing the angles of eyebrow and mouth of Chernoff’s face. Three drawings for non-target stimuli (neutral face, angry face, and wheelchair) and a target stimulus (cactus) were used. Stimuli were presented for either 200 or 300 ms in a random order. ERPs were recorded from 10 electrodes according to the international 10–20 system, and were referred to an electrode at the tip of the nose. At least 150 responses were averaged off-line after artifacts rejection. The latency and amplitude of P100 at O1 and O2 were unaffected by the nature of the stimuli. In contrast, the latency of N170 at T5 and T6 for neutral and angry faces was significantly shorter, and its amplitude was larger than those elicited by the object ( pb0.05). A slow negative shift was observed over the 230–450 ms time period to the angry face compared with the neutral face. This negative shift was significantly enhanced at a stimulus duration of 300 ms. Our findings suggest that the recognition of facial expression is set between 230 and 450 ms after the appearance of the face and is influenced by the duration of stimulus. Therefore, the high-spatial frequency components of a face appear to be crucial for the recognition of facial expression. D 2005 Elsevier B.V. All rights reserved. Keywords: Facial expression; Event-related potentials; N170; Chernoff’s face; Late negative component
1. Introduction Facial expressions are means of communication that are more rapid than language, with which people can quickly infer state of mind of their companions [1]. Accurate recognition * Corresponding author. Tel.: +81 92 642 5543; fax: +81 92 642 5545. E-mail address: [email protected] (R. Tsurusawa). 0531-5131/ D 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.ics.2004.11.190
54
R. Tsurusawa et al. / International Congress Series 1278 (2005) 53–56
Fig. 1. The visual stimuli used in this study. The appearance rate is embedded in the bottom of the right corner.
and interpretation of facial emotion is critical to normal social and cognitive development, adaptive social function, and psychological health and well-being across the life-span [2]. However, little is known about the time course of these processes. ERPs are suitable for examining the timing of processes involved in various aspects of face processing [3]. An early face-specific component (N170) elicited at the posterotemporal region has been considered to reflect the early stage of face recognition [4,5]. In contrast, ERP components sensitive to facial expression were usually observed N300-ms post-stimulus in early studies [6–8]. Previous studies focused on the perception of emotion in the photographed faces [3]. Faces of photographs contain a lot of various factors, such as gender, face color and race, etc. We have tried to simplify the various factors of faces so that we could present the simple change of the facial expression to the subjects. To achieve this, we used bChernoff’s facesQ as the stimuli (Fig. 1) which were invented by Herman Chernoff [9] for representing multivariate data. In this procedure, each facial feature denotes a particular variable. More importantly, the Chernoff’s face is made-up of rich high-spatial frequency (HSF) components or contour information. Therefore, the purpose of the present study was to examine the temporal characteristics of facial expression using the HSF faces as visual stimuli. 2. Materials and methods Nine adults, aged 19 to 29 years, were studied. The subjects were right-handed female. All subjects were tested after informed consent had been obtained. The stimuli were selected from 91 Chernoff’s faces, wheelchair and cactus (Fig. 1). The subject was seated and the visual angle of the stimulus was 7.59.58. In each of eight blocks, subjects were presented to 4 different stimuli in a random order: neutral and angry faces and a picture of wheelchair were presented 30 times while a picture of cactus was shown 10 times as a measure of attention. The subjects were required to push the button of targets (cactus). In each trial, the stimuli were presented for either 200 or 300 ms. The interstimulus intervals were randomly changed from 1100 to 1300 ms. Locations of recording electrodes were based on a variation of the 10–20 international electrode placement system including Fz, Cz, Pz, Oz, T5, T6, P1, P2, O1 and O2. An electrode at nose tip served as a reference. The ERPs were collected with a bandpass of 0.05–200 Hz and the sweep time of 770 ms, starting 224 ms before stimulus presentation. The component of interest was N170 at T5 and T6 [4,5] and its peak was measured with a window of 130–180 ms. The late negative component over the 230–450 ms time period probably related to either positive or negative valence was also measured. The data were subjected to repeated measures ANOVAs.
R. Tsurusawa et al. / International Congress Series 1278 (2005) 53–56
55
3. Results The waveforms of the grand average ERPs are shown in Fig. 2. A target stimulus, cactus, elicited a large P300, which was maximal over the posterior scalp (Pz) at around 400 ms, which indicated that the subjects were correctly performing the task. The large negative ERP with a peak latency of 140 ms (N170) was larger for faces and smaller for the wheelchair. The amplitudes of N170 elicited by neutral and angry faces were significantly more negative and larger than that of wheelchair. Repeated measures ANOVA showed that this amplitude difference was statistically significant at T5 [200 ms presentation, F(2,16)=10.51645, pb0.01; 300 ms presentation, F(2,16)=5.58569, pb0.01] and T6 [300 ms presentation, F(2,16)=4.333516, pb0.05]. The peak latencies of N170 elicited by faces were significantly shorter than that of wheelchair (157 ms) both T5 [200 ms presentation, F(2,16)=36.26323, pb0.01; 300 ms presentation F(2,16)=29.4618, pb0.02] and T6 [200 ms presentation F(2,16)=37.76581, pb0.01; 300 ms presentation F(2,16)=44.22403, pb0.01]. At T5 and T6, the N170 elicited by neutral and angry faces was equal in amplitude and latency. However, ERPs to angry faces were more negative than ERPs elicited by neutral faces. This difference between neutral and angry faces was observed in the 230–450 ms latency range after the stimulus onset. Interestingly, this negative shift for the stimulus duration of 300 ms was greater than that of 200 ms ( pb0.05, Wilcoxon signed rank-test). 4. Discussion In the present study, the face-specific N170 and the late negative component (230–450 ms) related to the facial expression were clearly elicited by Chernoff’s faces. This suggests that even a simple drawing or HSF component of the face is useful for evaluating the face recognition and facial expressions. A face-specific brain EEG potential at approximately 160 ms after stimulus presentation has recently been described by various research groups. Bentin et al. [4] reported that human faces evoked a negative potential at around 170 ms (N170) which was the largest over the posterotemporal scalp compared with animal faces and other non-face
Fig. 2. Grand averages of ERPs (n=9) elicited by 4 different stimuli. Note that P300 is only observed for target stimulus. The amplitude and latency of N170 for face stimuli were larger and shorter than those of wheelchair at T5 and T6 electrodes. In addition, a slow negative shift was observed to the angry face compared with the neutral face between 230 and 450 ms.
56
R. Tsurusawa et al. / International Congress Series 1278 (2005) 53–56
objects. The shorter N170 latency in response to human faces might also be taken as evidence that this component is sensitive to the precise configuration of the human face [4,5]. The delayed and attenuated ERPs to the wheelchairs relative to the faces in this study are consistent with the results of previous studies. Difference in N170 between the angry and neutral faces was not apparent, which was also consistent with previous studies in which facial expressions did not significantly alter N170 [3,5–7]. Our results showed that emotional faces elicited a greater negativity than neutral faces over the posterotemporal sites with a latency of 230–450 ms after the stimulus onset, depending on the presentation time of the faces. Since subjects had to respond to target stimuli (Cactus) with button press, it is unlikely that subjects were engaged in an extensive process of face recognition. Therefore, the neural activity associated with facial emotion was activated automatically, perhaps reflecting mandatory processing of facial emotion information. Interestingly, N170 amplitudes and latencies were entirely unaffected by the facial expression of either presentation time 200 or 300 ms. Several studies reported early emotional expression effect [10], but early effect was not observed in our study. Whether emotional expression effect is early or late probably depends on the experimental procedures. However, obvious difference in the amplitude of late negative component between neutral and angry faces in our study suggests that it reflects recognition of principal facial expression. In conclusion, our findings suggest that the recognition of facial expression is set between 230 and 450 ms after the appearance of the face and is influenced by the duration of stimulus. Therefore, the HSF components of a face appear to be crucial for the recognition of facial expression. Acknowledgements The authors thank Dr. Duco I. Hamasaki for the valuable comments on this manuscript. This study was supported in part by Grant-in-Aid for the 21st Century COE Program. References [1] M. Batty, M.J. Taylor, Early processing of the six basic facial emotional expressions, Cogn. Brain Res. 17 (2003) 613 – 620. [2] S. Lewis, R.J. Thoma, Visual processing of facial affect, NeuroReport 14 (2003) 1841 – 1845. [3] M. Miyoshi, J. Katayama, T. Morotomi, Face-specific N170 component is modulated by facial expressional change, NeuroReport 15 (2004) 911 – 914. [4] S. Bentin, et al., Electrophysiological studies of face perception in humans, J. Cogn. Neurosci. 8 (1996) 551 – 565. [5] M. Eimer, Does the face-specific N170 component reflect the activity of a specialized eye processor? NeuroReport 9 (1998) 2945 – 2948. [6] L. Carretie, J. Iglesias, An ERP study on the specificity of facial expression processing, Int. J. Psychophysiol. 19 (1995) 183 – 192. [7] T.F. Munte, et al., Brain potentials reveal the timing of face identity and expression judgments, Neurosci. Res. 30 (1998) 25 – 34. [8] S. Orozco, C.L. Ehlers, Gender differences in electrophysiological responses to facial stimuli, Biol. Psychiatry 44 (1998) 281 – 289. [9] H. Chernoff, The use of faces to represent points in k-Dimensional space graphically, J. Am. Stat. Assoc. (1973) 361 – 368. [10] M. Eimer, An ERP study on the time course of emotional face processing, NeuroReport 13 (2002) 427 – 431.
www.ics-elsevier.com
An important role for high-spatial frequencies in recognition of facial expressions Reimi Tsurusawaa,b,*, Yoshinobu Gotoa, Akihisa Mitsudomeb, Shozo Tobimatsua a
Department of Clinical Neurophysiology, Neurological Institute, Graduate School of Medical Sciences, Kyusyu University, 3-1-1 Maidashi, Higashi-ku, Fukuoka 812-8582, Japan b Department of Pediatrics, School of Medicine, Fukuoka University, Fukuoka, Japan
Abstract. We investigated the information processing of facial expression by ERPs to Chernoff’s face. Nine healthy, right-handed adult volunteers served as subjects. The Chernoff’s face is a simple drawing which is made-up of rich high-spatial frequency components. Neutral and angry faces were determined psychophysically by changing the angles of eyebrow and mouth of Chernoff’s face. Three drawings for non-target stimuli (neutral face, angry face, and wheelchair) and a target stimulus (cactus) were used. Stimuli were presented for either 200 or 300 ms in a random order. ERPs were recorded from 10 electrodes according to the international 10–20 system, and were referred to an electrode at the tip of the nose. At least 150 responses were averaged off-line after artifacts rejection. The latency and amplitude of P100 at O1 and O2 were unaffected by the nature of the stimuli. In contrast, the latency of N170 at T5 and T6 for neutral and angry faces was significantly shorter, and its amplitude was larger than those elicited by the object ( pb0.05). A slow negative shift was observed over the 230–450 ms time period to the angry face compared with the neutral face. This negative shift was significantly enhanced at a stimulus duration of 300 ms. Our findings suggest that the recognition of facial expression is set between 230 and 450 ms after the appearance of the face and is influenced by the duration of stimulus. Therefore, the high-spatial frequency components of a face appear to be crucial for the recognition of facial expression. D 2005 Elsevier B.V. All rights reserved. Keywords: Facial expression; Event-related potentials; N170; Chernoff’s face; Late negative component
1. Introduction Facial expressions are means of communication that are more rapid than language, with which people can quickly infer state of mind of their companions [1]. Accurate recognition * Corresponding author. Tel.: +81 92 642 5543; fax: +81 92 642 5545. E-mail address: [email protected] (R. Tsurusawa). 0531-5131/ D 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.ics.2004.11.190
54
R. Tsurusawa et al. / International Congress Series 1278 (2005) 53–56
Fig. 1. The visual stimuli used in this study. The appearance rate is embedded in the bottom of the right corner.
and interpretation of facial emotion is critical to normal social and cognitive development, adaptive social function, and psychological health and well-being across the life-span [2]. However, little is known about the time course of these processes. ERPs are suitable for examining the timing of processes involved in various aspects of face processing [3]. An early face-specific component (N170) elicited at the posterotemporal region has been considered to reflect the early stage of face recognition [4,5]. In contrast, ERP components sensitive to facial expression were usually observed N300-ms post-stimulus in early studies [6–8]. Previous studies focused on the perception of emotion in the photographed faces [3]. Faces of photographs contain a lot of various factors, such as gender, face color and race, etc. We have tried to simplify the various factors of faces so that we could present the simple change of the facial expression to the subjects. To achieve this, we used bChernoff’s facesQ as the stimuli (Fig. 1) which were invented by Herman Chernoff [9] for representing multivariate data. In this procedure, each facial feature denotes a particular variable. More importantly, the Chernoff’s face is made-up of rich high-spatial frequency (HSF) components or contour information. Therefore, the purpose of the present study was to examine the temporal characteristics of facial expression using the HSF faces as visual stimuli. 2. Materials and methods Nine adults, aged 19 to 29 years, were studied. The subjects were right-handed female. All subjects were tested after informed consent had been obtained. The stimuli were selected from 91 Chernoff’s faces, wheelchair and cactus (Fig. 1). The subject was seated and the visual angle of the stimulus was 7.59.58. In each of eight blocks, subjects were presented to 4 different stimuli in a random order: neutral and angry faces and a picture of wheelchair were presented 30 times while a picture of cactus was shown 10 times as a measure of attention. The subjects were required to push the button of targets (cactus). In each trial, the stimuli were presented for either 200 or 300 ms. The interstimulus intervals were randomly changed from 1100 to 1300 ms. Locations of recording electrodes were based on a variation of the 10–20 international electrode placement system including Fz, Cz, Pz, Oz, T5, T6, P1, P2, O1 and O2. An electrode at nose tip served as a reference. The ERPs were collected with a bandpass of 0.05–200 Hz and the sweep time of 770 ms, starting 224 ms before stimulus presentation. The component of interest was N170 at T5 and T6 [4,5] and its peak was measured with a window of 130–180 ms. The late negative component over the 230–450 ms time period probably related to either positive or negative valence was also measured. The data were subjected to repeated measures ANOVAs.
R. Tsurusawa et al. / International Congress Series 1278 (2005) 53–56
55
3. Results The waveforms of the grand average ERPs are shown in Fig. 2. A target stimulus, cactus, elicited a large P300, which was maximal over the posterior scalp (Pz) at around 400 ms, which indicated that the subjects were correctly performing the task. The large negative ERP with a peak latency of 140 ms (N170) was larger for faces and smaller for the wheelchair. The amplitudes of N170 elicited by neutral and angry faces were significantly more negative and larger than that of wheelchair. Repeated measures ANOVA showed that this amplitude difference was statistically significant at T5 [200 ms presentation, F(2,16)=10.51645, pb0.01; 300 ms presentation, F(2,16)=5.58569, pb0.01] and T6 [300 ms presentation, F(2,16)=4.333516, pb0.05]. The peak latencies of N170 elicited by faces were significantly shorter than that of wheelchair (157 ms) both T5 [200 ms presentation, F(2,16)=36.26323, pb0.01; 300 ms presentation F(2,16)=29.4618, pb0.02] and T6 [200 ms presentation F(2,16)=37.76581, pb0.01; 300 ms presentation F(2,16)=44.22403, pb0.01]. At T5 and T6, the N170 elicited by neutral and angry faces was equal in amplitude and latency. However, ERPs to angry faces were more negative than ERPs elicited by neutral faces. This difference between neutral and angry faces was observed in the 230–450 ms latency range after the stimulus onset. Interestingly, this negative shift for the stimulus duration of 300 ms was greater than that of 200 ms ( pb0.05, Wilcoxon signed rank-test). 4. Discussion In the present study, the face-specific N170 and the late negative component (230–450 ms) related to the facial expression were clearly elicited by Chernoff’s faces. This suggests that even a simple drawing or HSF component of the face is useful for evaluating the face recognition and facial expressions. A face-specific brain EEG potential at approximately 160 ms after stimulus presentation has recently been described by various research groups. Bentin et al. [4] reported that human faces evoked a negative potential at around 170 ms (N170) which was the largest over the posterotemporal scalp compared with animal faces and other non-face
Fig. 2. Grand averages of ERPs (n=9) elicited by 4 different stimuli. Note that P300 is only observed for target stimulus. The amplitude and latency of N170 for face stimuli were larger and shorter than those of wheelchair at T5 and T6 electrodes. In addition, a slow negative shift was observed to the angry face compared with the neutral face between 230 and 450 ms.
56
R. Tsurusawa et al. / International Congress Series 1278 (2005) 53–56
objects. The shorter N170 latency in response to human faces might also be taken as evidence that this component is sensitive to the precise configuration of the human face [4,5]. The delayed and attenuated ERPs to the wheelchairs relative to the faces in this study are consistent with the results of previous studies. Difference in N170 between the angry and neutral faces was not apparent, which was also consistent with previous studies in which facial expressions did not significantly alter N170 [3,5–7]. Our results showed that emotional faces elicited a greater negativity than neutral faces over the posterotemporal sites with a latency of 230–450 ms after the stimulus onset, depending on the presentation time of the faces. Since subjects had to respond to target stimuli (Cactus) with button press, it is unlikely that subjects were engaged in an extensive process of face recognition. Therefore, the neural activity associated with facial emotion was activated automatically, perhaps reflecting mandatory processing of facial emotion information. Interestingly, N170 amplitudes and latencies were entirely unaffected by the facial expression of either presentation time 200 or 300 ms. Several studies reported early emotional expression effect [10], but early effect was not observed in our study. Whether emotional expression effect is early or late probably depends on the experimental procedures. However, obvious difference in the amplitude of late negative component between neutral and angry faces in our study suggests that it reflects recognition of principal facial expression. In conclusion, our findings suggest that the recognition of facial expression is set between 230 and 450 ms after the appearance of the face and is influenced by the duration of stimulus. Therefore, the HSF components of a face appear to be crucial for the recognition of facial expression. Acknowledgements The authors thank Dr. Duco I. Hamasaki for the valuable comments on this manuscript. This study was supported in part by Grant-in-Aid for the 21st Century COE Program. References [1] M. Batty, M.J. Taylor, Early processing of the six basic facial emotional expressions, Cogn. Brain Res. 17 (2003) 613 – 620. [2] S. Lewis, R.J. Thoma, Visual processing of facial affect, NeuroReport 14 (2003) 1841 – 1845. [3] M. Miyoshi, J. Katayama, T. Morotomi, Face-specific N170 component is modulated by facial expressional change, NeuroReport 15 (2004) 911 – 914. [4] S. Bentin, et al., Electrophysiological studies of face perception in humans, J. Cogn. Neurosci. 8 (1996) 551 – 565. [5] M. Eimer, Does the face-specific N170 component reflect the activity of a specialized eye processor? NeuroReport 9 (1998) 2945 – 2948. [6] L. Carretie, J. Iglesias, An ERP study on the specificity of facial expression processing, Int. J. Psychophysiol. 19 (1995) 183 – 192. [7] T.F. Munte, et al., Brain potentials reveal the timing of face identity and expression judgments, Neurosci. Res. 30 (1998) 25 – 34. [8] S. Orozco, C.L. Ehlers, Gender differences in electrophysiological responses to facial stimuli, Biol. Psychiatry 44 (1998) 281 – 289. [9] H. Chernoff, The use of faces to represent points in k-Dimensional space graphically, J. Am. Stat. Assoc. (1973) 361 – 368. [10] M. Eimer, An ERP study on the time course of emotional face processing, NeuroReport 13 (2002) 427 – 431.