- Email: [email protected]
Brain potentials reflect access to visual and emotional memories for faces
Biological Psychology 75 (2007) 146–153 www.elsevier.com/locate/biopsycho
Brain potentials reflect access to visual and emotional memories for faces Maria A. Bobes a,*, Ileana Quin˜onez a, Jhoanna Perez a, Inmaculada Leon b, Mitchell Valde´s-Sosa a a
Cognitive Neuroscience Department, Cuban Center for Neuroscience, Ave. 25 y 158, Cubanacan, Apartado 6880, Havana, Cuba b Psychology Faculty, University of La Laguna, Campus de Guajara 38071, Tenerife, Spain Received 15 July 2006; accepted 19 January 2007 Available online 1 February 2007
Abstract Familiar faces convey different types of information, unlocking memories related to social–emotional significance. Here, the availability over time of different types of memory was evaluated using the time-course of P3 event related potentials. Two oddball paradigms were employed, both using unfamiliar faces as standards. The infrequent targets were, respectively, artificially-learned faces (devoid of social–emotional content) and faces of acquaintances. Although in both tasks targets were detected accurately, the corresponding time-course and scalp distribution of the P3 responses differed. Artificially-learned and acquaintance faces both elicited a P3b, maximal over centro-parietal sites, and a latency of 500 ms. Faces of acquaintances elicited an additional component, an early P3 maximal over frontal sites: with a latency of 350 ms. This suggests that visual familiarity can only trigger the overt recognition processes leading to the slower P3b, whereas emotional–social information can also elicit fast and automatic assessments (indexed by the frontal-P3) crucial for successful social interactions. # 2007 Elsevier B.V. All rights reserved. Keywords: P3b; P3a; Faces; Memory; Emotion; Familiarity; Social
1. Introduction Faces afford different types of useful social information. From faces, we can infer age, race, or sex, as well as the emotional states and intentions of others. Also important is the ability to recognize person-identity from familiar faces, which is at the heart of important social transactions. Familiar faces unlock memories related to hierarchical status, degree of friendship (or otherwise), and personality traits among other aspects, plus a host of biographical details all of which go beyond the memory traces leading to purely visual familiarity. This emotional and social information associated with specific individuals will be dubbed here as ‘‘emotion-from-identity’’ to distinguish it from the more thoroughly studied emotional information that is derived from facial expressions. Existing models of face processing (i.e. Bruce and Young, 1986; Haxby et al., 2000) accept the idea that different kinds of face-derived information are processed in distinct and
* Corresponding author. Tel.: +53 7 2083990; fax: +53 7 2086321. E-mail address: [email protected] (M.A. Bobes). 0301-0511/$ – see front matter # 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.biopsycho.2007.01.006
specialized brain sub-systems. Originally, models postulated separate routes for identity and emotional expression recognition, with little emphasis on emotion-from-identity, although more recent models are beginning to remedy this (Gobbini and Haxby, 2007; Rolls, 2007). Neuroimaging studies have begun to identify the neural structures comprising these sub-systems (Haxby et al., 1994; Kanwisher et al., 1997), with a number of studies examining the neural basis of face familiarity. Several studies have compared brain responses to previously known faces and new unfamiliar faces, but of course the results depend of the type and degree of knowledge acquired for the familiar faces used in the comparison. Studies with functional magnetic resonance imaging (fMRI) have evinced brain regions activated more by faces that are familiar at a purely visual level (i.e. only structural memories are available) than by completely unfamiliar faces. These areas include the medial and posterior right face fusiform area (FFA), the left posterior FFA and the posterior cingulate gyrus (Kosaka et al., 2003; Leube et al., 2003; Rossion et al., 2003). On the other hand, for familiar faces that, beyond visual familiarity, and also are associated with biographical, emotional and social cues (such as famous faces and faces of acquaintances)
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
additional brain regions are recruited. Such additional brain regions include areas in the temporal lobe, the occipitotemporal gyrus including the FFAs, the anterior and posterior cingulate cortex, medial and inferior frontal gyrus, and the orbitofrontal cortex (for famous > unfamiliar, see Gorno-Tempini and Price, 2001; Henson et al., 2003; Leveroni et al., 2000; for acquaintances > unfamiliar, see Nakamura et al., 2000; Shah et al., 2001; Sugiura et al., 2001). Ecological considerations suggest that brain areas handling emotionally relevant memories should activate rapidly (Phelps, 2006). However, it is difficult to infer the timing of these activations from neuroimaging studies due to their poor temporal resolution. Information about these time-courses could potentially be obtained with the higher temporal resolution of event related potentials (ERPs). However, even so several studies have been performed, conclusive information is not available yet. Early ERP components, highly sensitive to faces, such as the intracranial negativity at 200 ms (Allison et al., 1999), and the scalprecorded N170 (Bentin et al., 1996), are not sensitive to face familiarity (Eimer, 2000a,b; Henson et al., 2003). Potentially relevant ERPs effects related to familiarity have been reported at about 250–300 ms (Bentin and Deouell, 2000; Schweinberger et al., 2002). To our knowledge no previous studies have used ERPs to compare the time-course of activations for different types of memories related to faces, nor have they specifically examined the time-course of memory activation for emotion-from-identity. One approach to this issue is to use the well-known P3 ERP, which has previously been used to study face familiarity (Bobes et al., 2004; Gunnar and Nelson, 1994; Renault et al., 1989). The P3 is usually examined within an oddball paradigm, which consists of presenting rare target stimuli amidst a train of more frequent standard stimuli, the former evoking the P3. Although the P3 is not specifically related to faces processing (in contrast to the N170), it has been frequently used to index stimulus evaluation time (Kutas et al., 1977; Picton, 1992; Polich, 2003; but see Verleger, 1997). Since P3 can only arise after the categorization of the target stimuli as different from the standard, its onset latency offers an upper bound on the time needed for this processes. Hence, if infrequent familiar faces are interspersed amidst frequent familiar faces, the P3 can be used to index the availability of the information distinguishing these two types of faces, in other words, relevant memories for people. This strategy has even evinced a P3 in patients with prosopagnosia, who were unable to explicitly recognize familiar faces, thus demonstrating covert recognition (Renault et al., 1989; Bobes et al., 2004). In the present study, the availability over time of different types of memory related to person-identity was examined using two variants of the oddball paradigm. Both variants contained frequent unfamiliar faces as standard stimuli. In the first variant the infrequent target stimuli were newly-learned faces devoid of emotional and social content. These newly-learned were previously unfamiliar faces that had been studied to achieve a high degree of recognition accuracy. In the second variant, the target stimuli were faces of acquaintances with emotional significance (i.e., their close relatives and friends), which were
147
tailored to each subject. In both variants the target and standard stimuli differed purely in visual familiarity. In the case of acquaintances, the targets also afforded emotion-from-identity. Therefore, visual familiarity and emotion-from identity, confounded in most natural situations, could be dissociated. To validate the difference in emotional content for these two types of target faces, skin conductance responses (SCR) were measured. Previous studies have shown that SCR responses elicited by familiar (acquaintance and famous) faces are larger than those triggered by unfamiliar faces (Bauer, 1984; Tranel and Damasio, 1985). The P3 elicited by the targets were used as an index of the availability of the relevant memories associated to the target faces: visual familiarity for newly-learned faces and both visual familiarity and emotion-from-identity for acquaintance faces. The onset latencies of the P3 could provide an upper bound to the availability of the memory traces. An important question was thus whether emotion-from-identity becomes available only after visual familiarity is overtly recognized, as predicted by some theories of emotion (Lazarus, 1984), or whether emotion-from-identity can be processed in parallel with (or even before) explicit recognition of visual familiarity, as suggested by recent models for facial expressions (Gobbini and Haxby, 2007). Moreover, consideration of the scalp topography of the P3 elicited by the two kinds of familiar faces was interesting in itself. Previous work has shown that P3 is really a family of distinct late positive components, each sensitive to different experimental factors, and with divergent distributions over the scalp, thus reflecting distinct mental operations (Squires et al., 1975; Soltani and Knight, 2000). For example, the parietocentral P3b is generated when active attention is given to the stimuli (Ruchkin et al., 1990; Picton, 1992), whereas the frontally distributed P3a (Squires et al., 1975) and the novelty P3 (Courchesne et al., 1975) are considered to be more automatic and less dependent on attention (Escera et al., 1998; Na¨a¨ta¨nen, 1992; Schroger et al., 2000). This novelty P3 is related to the stimulus prediction and is not modality specific (Knight, 1984; Ranganath and Paller, 1999, and see Ranganath and Rainer, 2003 for a review). Therefore, the scalp distribution of the P3 in the two paradigms used in the present study could indicate if different neural systems are involved in the processing of visual familiarity and of emotion-from-identity. 2. Methods 2.1. Participants Ten healthy adults, six males and four females, with ages ranging from 23 to 33 years (mean = 27) participated in the study as non-paid volunteers. All the participants had university degrees, were right-handed (as ascertained by personal report) and had normal or corrected-to-normal vision.
2.2. Face stimuli The stimuli consisted of digitized photographs of faces, which were all software-edited using Adobe Photoshop. All photographs were converted to grayscale, and their background removed. An attempt was made to homogenize the stimuli with respect to average luminance contrast using this software. Each
148
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
face was displayed within a circular border, which masked the external features, clothes, and background. All faces consisted of frontal views and could be either familiar or unfamiliar to the subjects (see Bobes et al., 2004, for a similar procedure and testing for low level physical difference between known and unknown faces). Two different sets of familiar faces were obtained, acquaintance faces, which were selected among the family and close friends of each subject (i.e. this was different for each subject), and newly-learned faces, which were previously unknown faces, learned by practicing in the lab (as explained below). Each of the familiar faces sets, randomly mixed with the unfamiliar faces, comprised a stimulation block for an oddball paradigm: Block I, acquaintance-unknown faces, in which the familiar target faces were acquaintances and Block II, newlylearned-unknown faces, in which the learned faces was the target stimuli. Each block contained 15 different familiar faces and 75 different unfamiliar faces, and every face was presented twice, meaning that each stimuli block comprised a total of 30 familiar (17%) and 150 unfamiliar (83%) faces for each stimuli block. Different unfamiliar faces were used as standards in each block and the order of face presentation in the two blocks was counterbalanced across subjects. Face stimuli were presented for 1000 ms, with an inter-stimulus interval (ISI) of 1000 ms. Subjects were required to discriminate between familiar and unfamiliar faces and to respond via the computer keyboard during the ISI (a go/ go design). Task performance was evaluated by measuring hits percent corrected by chance level: PHits PFalseAlarms 1 PFalseAlarms in each subject.
2.3. Training procedure for newly-learned faces Learning took place on six sessions (two session per day). Each session consist of a study and a test phase. In the study phase, the subject viewed a training set of 15 initially unknown faces (the same in all sessions), presented on the CRT monitor in a random order, with onset and offset times under subject’s control. The test phase consisted of a familiarity decision task. The 15 ‘‘old’’ faces for the training set were all presented again, randomly mixed with 15 unfamiliar ‘‘new’’ faces from the unknown face pool. The unfamiliar faces were never repeated. The subjects’ responses were measured via two mouse-keys (familiar or unfamiliar). Immediate acoustic feedback on the accuracy of the response was provided. The faces in the training set were thus seen 12 times during training. (This procedure was modified from Olivares et al., 1994.)
2.4. ERP recording and analysis Subjects sat in front of a sVGA monitor (about 1 m from the observer) and were instructed to minimize body and eye movements during the experiments. Stimuli on the CRT subtended a vertical visual angle of 3.78 and a horizontal visual angle of 2.58. Data acquisition was carried out with 120 monopolar derivations, using electrodes mounted in an elastic cap homogeneously distributed over the scalp, as well as two channels of EOG signals on a MEDICID-128 system (Neuronic, SA, Havana). A notch filter with peak at 60 Hz was used and the signals were amplified by a factor of 10,000 and filtered between 0.5 and 30 Hz (3 dB down). All electrodes were referred to linked earlobes and then re-referenced to an average reference, and the inter-electrode impedance was always below 10 kV. The onset of the stimuli served to synchronize data collection. The EEG was digitally recorded at a sampling rate of 200 Hz, with 700 ms epochs, and a 300 ms pre-stimulus baseline. Each EEG epoch was stored on magnetic disk, and was visually inspected offline. Those epochs with generalized artifacts or detectable eye-movement in the EOG were eliminated (19% on average). In the 120-channel recording, electrodes with excessive noise were eliminated and substituted by an interpolation of the 11 closest neighbors. A mean of 19 channels (1.58% on average) was interpolated across subjects. Averaged evoked responses were digitally low-pass filtered (18 Hz cut-off). ERP amplitudes were corrected by subtracting the average pre-stimulus amplitude value. Only standard trials preceding deviant stimuli were averaged for this condition in
order to equalize the signal-to-noise ratio of the resulting ERPs with those related to the standard ones. For display purposes, scalp voltage maps were calculated for the average amplitude in a time window centered around the maximum of the P3 peak (see Fig. 4). These voltages were expressed as a percentage of the largest electrode measurement. Thus maps from the two tasks could be compared after eliminating possible differences in amplitude scale. These topographic 3D maps are plotted over the surface of the MNI average brain (ICBM 305). An 11-neighbors interpolation method was used for obtaining intermediate points in the map. Electrophysiological data were analyzed with permutation tests. Permutation tests have the following advantages for ERP research: the tests are distribution free, no assumptions of an underlying correlation structure between time points or electrodes are required, and they provide exact p-values for any number of subjects, time points and recording sites (see Blair and Karniski, 1993; Gala´n et al., 1997). For tests of differences between the ERPs associated with two conditions in a group of subjects, the analysis consists of the following steps: 1. The null hypothesis (H0) of equal mean values for the two groups of ERPs is decomposed into the marginal hypotheses H0dt: mdt1 = mdt2, where mdtj is the mean value of the ERP obtained in the d site and t time for the j experimental condition. 2. All the marginal hypotheses are tested by:
a. Computing the t statistic for the original ERP data contrasting the two conditions in n subjects. b. Obtaining a large number of permutation resample from the data (without replacement) and constructing the permutation distribution of the t statistic (the complete permutation space, 2n). c. Finding the p-value by locating the original statistic on this t statistic permutation distribution. A set of nested decisions about hypotheses H0 is performed by using the maximum of several combinations of marginal hypothesis permutation tests thus controlling the experiment wise error for the simultaneous univariate comparisons. Combining all sites and time points give a global test; combining all time points for each electrode gives a test for sites, combining for each time point across sites gives a test for temporal differences. This last option was used to estimate the P3 onset latencies in each subject by contrasting single-trial epochs associated with standards and those epochs associated with targets. A similar procedure was used with the average ERPs for all subjects in order to estimate the amplitude differences between two experimental conditions. The statistical difference between two scalp distributions was also tested applying permutation techniques. In this case, the vector of amplitudes used to obtain the scalp maps (see above) was firstly normalized as suggested by McCarthy and Wood (1985) to eliminate differences in amplitudes and allow for testing of topographic distributions. Then the global test based on permutations (described above) was performed.
2.5. SCR In a separate session, SCRs were measured for two stimuli blocks, including acquaintance or newly-learned faces respectively. Each block comprised a total of 52 faces, 39 unfamiliar faces randomly mixed with 13 familiar faces (a subset of those presented in the ERP paradigm). The stimuli were presented on a sVGA monitor as explained before. Each stimulus was presented for a duration of 2 s, and the next stimulus was delayed until the SCR recording related to the previous one had returned to baseline. This resulted in inter-stimulus intervals longer than 20 s. Subjects were instructed to passively view the faces. Electrodermal activity was recorded by using Ag/AgCl electrodes taped to the palmar surface of the proximal phalanx of left index and middle fingers after
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153 cleaning the attachment sites and applying conductive gel, and the signal was fed via a skin conductance-processing unit (GSR-2100, Nihon Kohden, Japan) to a channel of the MEDICID III/E system. The filtered analog output of the SCR was displayed online and recorded digitally (sample rate, 100 Hz), in synchronism with the onset of the face, using custom-made software following the procedure described in Bobes et al. (2004).
3. Results 3.1. SCR The amplitudes of the SCR responses for the two experimental blocks (acquaintance versus unknown faces, and newlylearned versus unknown faces) are presented in Fig. 1. In the acquaintance versus unknown faces block, the mean SCR response to acquaintance faces (mean = 0.6 mS, S.D. = 0.39) was larger than for unknown faces (0.42 mS, S.D. = 0.29). However, in the newly-learned versus unknown faces block, the SCR response to newly-learned (mean = 0.449 mS, S.D. = 0.3) and unknown faces (mean = 0.461 mS, S.D. = 0.4) were very similar. A repeated measures ANOVA was performed, with two within-subject factors: FAMILIARITY (known versus unknown) and EXPERIMENTAL BLOCK (newly-learned versus acquaintance). Although the two mains effects were not significant, the interaction between FAMILIARITY and EXPERIMENTAL BLOCK was marginally significant (F(1,9) = 4.4, p = 0.06). Planned comparisons showed a highly significant difference between the SCR to acquaintance faces and unknown faces (F(1,9) = 7.3, p = 0.025), whereas no difference was found between the SCR to newly-learned and unknown faces (F(1,9) = 0.06, p = 0.8). These results confirm the lack of emotional valence of newly-learned faces. 3.2. Behavioral data during the ERP recording session Familiar faces were accurately detected in the two experimental ERP blocks. The mean percentage of hits for
Fig. 1. Skin conductance response (SCR) obtained in the two stimulation blocks. On the left, the responses obtained to acquaintance faces randomly interspersed with unknown faces. On the right, the responses obtained to newlylearned faces randomly interspersed with unknown faces. Means across the subject sample and S.E.M.s (error bars) are presented.
149
newly-learned faces was 94% (S.D. = 0.07) and 96% (S.D. = 0.05) for acquaintances. Accuracy was not different for the two tasks (t = 88, d.f. = 18, p = 0.39). 3.3. ERPs The grand average ERPs elicited by newly-learned and unfamiliar faces are overlaid in Fig. 2A. The amplitudes of the earlier ERP components elicited by these two stimuli were not different. Both responses exhibited the N170 component with similar latency and amplitude. For newly-learned faces, the mean latency of the N170 at T6 was 180 ms (S.D. = 36.6) and for unfamiliar faces it was 176 ms (S.D. = 25.5). After around 400 ms, the recordings corresponding to the two stimuli differed; the ERPs associated with the newly-learned faces presented an enhanced positivity widely distributed over the head, with larger amplitudes at central and posterior sites. The peak latency of this positivity is about 505 ms at Pz. We considered this component to be a P3b. The effect of familiarity was tested statistically by performing permutation analysis for dependent samples over the averaged ERPs obtained in each subject. The comparison between ERPs associated with newlylearned and unfamiliar faces are shown in Fig. 2B, displaying the probability values obtained by the permutation test for each time point (amplitudes for targets > amplitudes for standards). The test was significant in the time window between 450 and 600 ms ( p < 0.05), which corresponds with the timing of the P3b elicited by newly-learned faces. The ERPs’ waveforms evoked by acquaintance faces are shown in Fig. 3A. No difference was observed in components earlier than 300 ms between the ERPs elicited by the two stimuli types. The mean N170 latency at T6 was 183 ms (S.D. = 37) for acquaintance faces and 176 ms (S.D. = 25.1) for unfamiliar faces. A repeated measures ANOVA including TYPE of familiar faces (newly-learned versus acquaintances) and FAMILIARITY (familiar versus unfamiliar) did not find significant differences in N170 latency due to TYPE (F(1,8.1) = 0.11, p = 0.747) or to FAMILIARITY (F(1,292) = 0.306, p = 0.594). No significant interaction between these two factors was found (F(1,32) = 1.92, p = 0.199). A centro-parietal positive component was elicited by acquaintance faces (Fig. 3A). Similar to that found for newly-learned faces, this component reached a maximum around 500 ms at Pz, and was also considered a P3b. However, in the case of acquaintance faces an additional positivity was produced at frontal sites, which shall be referred to as the frontal-P3. The peak latency of this frontal positivity was 355 ms at Fz. Results of the permutation tests for the P3 effect (amplitudes for targets > amplitudes for standards) are shown in Fig. 3B ( p < 0.05). The tests were significant in two time windows: one between 355 and 425 ms and another from 450 and 550 ms. This time windows correspond with the timing of the two P3b and the frontal-P3. The scalp distributions of the P3 components elicited by both types of familiar faces are shown in more detail in Fig. 4. The P3b component elicited by newly-learned faces (in a time window between 500 and 580 ms) was of maximum amplitude
150
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
Fig. 2. (A) ERPs obtained during the discrimination of face familiarity in the newly-learned faces block. ERPs associated with unfamiliar faces (thin line) are overlaid on ERPs elicited by newly-learned faces (thick lines) in selected derivations. (B) Results of permutation over the 120 channels of the individuals ERPs are shown for each time point. Probability is presented in a logarithmic scale. Markers indicate time points where probability was below 0.05.
at central–parietal sites, although widely spread across the scalp (Fig. 4A). A similar topography was exhibited by the P3b elicited by acquaintance faces in the 460–570 ms time window. The topography of these two P3b was not statistically different as evinced by the permutation test (global p = 0.29). In contrast, the earlier frontal-P3 – elicited by acquaintance faces – had a
distribution restricted to frontal sites, and was of low amplitude at central and posterior sites (Fig. 4C). The statistical analysis confirmed that this topography was different from the P3b component elicited by acquaintances’ faces (global p = 0.0001), and also different from the P3b elicited by newly-learned faces (global p = 0.002). Tests of marginal
Fig. 3. (A) ERPs obtained during the discrimination of face familiarity in the acquaintance faces block. ERPs associated with unfamiliar faces (thin line) are overlaid on ERPs elicited by acquaintance faces (thick lines) in selected derivations. (B) Results of permutation over the 120 channels of the individuals ERPs are shown for each time point. Probability is presented in a logarithmic scale. Markers indicate time points where probability was below 0.05.
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
151
Fig. 4. Voltage map representing the scalp distribution of the positive components in the ERPs associated to familiar faces. (A) P3b for newly learned faces. (B) P3b for acquaintance faces. (C) P3a for acquaintance faces. These maps were obtained for the mean amplitude in the time windows including the positive components; expressing amplitudes as a percentage of the maximum. Three different views are depicted: left, frontal view; center, from above with the nose up; right, back view.
hypotheses showed that the earlier frontal-P3 component was significantly larger in amplitude, as compared to the P3b elicited by acquaintance faces, at electrodes Fp1, Fp2, AF7, AF8, AF3, AF4, C4 and CP4 (all p < 0.05). To summarize, the scalp distribution of the P3b associated to acquaintance faces is very similar to the distribution of the P3b elicited by newlylearned faces, but its topography differed from the distribution of frontal-P3 elicited by acquaintance faces. The onset latency of the familiarity effects was estimated in each subject using the permutation test over the single-trial epochs of the EEG. The onset latency in each case was measured as the first point showing significant amplitude difference between familiar and unfamiliar faces ( p < 0.05), considering the 120 channels together. The mean value for acquaintance faces was 314 ms (S.D. = 76), which was significantly earlier than the 420 ms (S.D. = 82) obtained for newly-learned faces (t = 4.48, d.f. = 9, p < 0.0001). 4. Discussion Accurate detection was achieved for both acquaintance and newly-learned face targets. Faces of acquaintances elicited larger SCR responses than did newly-learned and unfamiliar faces. Both types of familiar-faces elicited a robust P3. However, the time-course and scalp distribution of the P3 response differed between the two tasks. Newly-learned faces elicited a positivity around 500 ms (larger over centro-parietal sites), whereas faces of acquaintances elicited two sub-
components: one similar to that elicited by newly-learned faces and another (earlier) sub-component with latency at about 350 ms (larger over frontal sites). The similar recognition accuracy for the two types of familiar faces indicates that the artificial over-training for newly-learned faces created memory traces of similar strength as those existing for acquaintances. These memories were necessarily of a purely visual nature (i.e. limited to face structural knowledge) since no other information was provided about these faces. On the other hand, the difference in emotional significance between newly-learned and acquaintance faces was clearly validated by the SCR results. The SCR reflects emotional arousal, with larger responses to emotional (i.e. faces of acquaintance or famous persons) relative to neutral events (i.e. unknown faces, Bauer, 1984; Stone et al., 2001; Tranel and Damasio, 1985). As expected, here faces of acquaintances elicited large SCRs whereas responses for unknown faces were of low amplitude. Newly-learned faces also elicited small SCRs, which confirms their lack of emotional and social significance. Therefore, the contributions of purely visual familiarity and of emotion-from-identity can be disentangled comparing the two sorts of faces. This is a difficult goal to reach using only naturally learned faces for which a closer acquaintance generates not only more emotional and social memories, but also more visual familiarity with the other person’s face. Newly-learned faces evoked a large P3 component with a centro-parietal scalp topography, which corresponds to what
152
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
has been described as P3b (see Picton, 1992, for a review). The latency of this P3b is somehow delayed compared to the typical 300 ms described for oddball tasks based on the discrimination of simple sensory features. However, it is in the same latency range as previously described P3s obtained with face stimuli (Bobes et al., 2004; Campanella et al., 2002; Gunnar and Nelson, 1994; Renault et al., 1989). This is consistent with the idea that P3b latency varies widely as a function of stimulus material and task difficulty (Kutas et al., 1977; Picton, 1992; Polich, 2003). The P3b component is generated only when active attention is allocated to the oddball stimuli, and is of larger amplitude if the categorization decisions are more confident (Ruchkin et al., 1990; Picton, 1992), and thus is construed as being linked to mental operations such as the explicit updating of working memory (Polich, 2003). As mentioned above, the P3 for the acquaintance targets is comprised by two sub-components with different timings. The scalp distribution of the response in the two time regions was different, suggesting that they originated from different neural sources. The later sub-component, with latency near 500 ms, exhibited a centro-parietal distribution and is probably similar to the component elicited by newly-learned face targets and can be considered a P3b. In contrast, the earlier (and perhaps more interesting) sub-component has a frontal distribution and is more similar to the P3a (Squires et al., 1975), or the novelty P3 (Courchesne et al., 1975) which appears to unusual or arousing distracters. However, further research is needed before one can confidently label the frontal-P3 as P3a. This frontal-P3 is not elicited by newly-learned faces implying that emotion-fromidentity is a necessary condition for its generation. Other late positive components have been reported in response to emotional stimuli, such as the late positive potential (LPP) obtained for arousing pictures in the affective oddball paradigm (Cacioppo et al., 1994; Cuthbert et al., 2000). However, the LPP is of a longer latency and has a centro-parietal distribution which implies that it is different from the frontal-P3 described here. Salient significant stimuli can elicit a frontal positive component under some conditions. When interspersed with sounds from musical instruments, human voices elicit the voice sensitive response (VSR), a positive component reminiscent of the P3a (Levy et al., 2001). More than novelty, this positive component seems to be triggered by the significance of voice stimuli for humans listeners (Levy et al., 2003), and consequently could be related to the frontal-P3 described here, given the emotional significance of acquaintance faces. The fact that only faces of acquaintances elicit a frontal-P3, similar in latency and topography to the P3a is interesting. On one hand, only the faces of acquaintances afforded emotionfrom-identity in the present study. On the other hand, P3a is considered to reflect more automatic and less attentiondependent processes than P3b (Escera et al., 1998; Na¨a¨ta¨nen, 1992; Schroger et al., 2000). This opens the possibility that the frontal-P3 described here reflects a faster, and perhaps more automatic, evaluation of emotion-from-identity. The idea of fast and automatic processing of emotionally significant stimuli (such as facial expressions) has been frequently discussed (Damasio, 1999; LeDoux, 1995; Phelps, 2006; see Pessoa,
2005, for a critical review), although to our knowledge has not been specifically examined for emotion-from-identity. A component similar to the frontal-P3 was found with the same oddball design used here, but in a study of a case with dense, acquired prosopagnosia (Bobes et al., 2004). Despite his severe inability for overt recognition of faces, this case presented larger SCRs to faces of acquaintances than to unfamiliar faces and selected the faces of acquaintances above chance if asked about trustworthiness. The onset latency of the P3 elicited by previously familiar (but now unrecognized) faces was around 400 ms and presented a centro-frontal maximum. This would be consistent with the idea that the frontal-P3 could be more automatic in nature than the P3b (which was absent in the patient). Further studies are necessary to verify this hypothesis. As pointed out in the introduction, onset latencies of the P3 components provide an upper bound for the timing of the memory activations that enable visual familiarity and emotionfrom-identity. The frontal-P3 elicited by acquaintances had an onset latency that was about 100 ms shorter than the P3b elicited by both types of target faces. This precluded the possibility that the memories coding emotion-from-identity were activated in series after the stimulus evaluation leading to the P3b was completed. While not the only possible explanation, this result is consistent with a faster route to emotion-from-identity parallel to the explicit and attentive processing leading to the P3b. The latency of the N170 face-specific component was equivalent for all types of faces used in the study. The N170 is considered to index structural coding of faces in ventral extrastriate cortex (Bentin and Deouell, 2000). Therefore, differences in processing times along these hypothetical routes would have to originate after, or in parallel, to this structural coding. To conclude, our data indicates that the social and emotional significance of faces triggers a relatively fast (and perhaps more automatic) response that is indexed by a frontal-P3, which does not depend on the slower and overt stimulus evaluation of familiarity that leads to a P3b. The neuroanatomy of these processes remains to be worked out.
References Allison, T., Puce, A., Spencer, G., McCarthy, G., 1999. Electrophysiological studies of human face perception. I. Potentials generated in occipitotemporal cortex by face and non-face stimuli. Cerebral Cortex 9, 415–430. Bauer, R.M., 1984. Autonomic recognition of names and faces in prosopagnosia: a neuropsychological application of the guilty knowledge test. Neuropsychologia 22, 457–469. Bentin, S., Allison, T., Puce, A., Perez, A., McCarthy, G., 1996. Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience 8, 551–556. Bentin, S., Deouell, L.Y., 2000. Structural encoding and identification in face processing: ERP evidence for separate mechanisms. Cognitive Neuropsychology 17, 35–54. Blair, R., Karniski, W., 1993. An alternative method for significance testing of waveform difference potential. Psychophysiology 30, 518–524. Bobes, M.A., Lopera, F., Garcia, M., Dı´az Comas, L., Galan, L., Valdes-Sosa, M., 2004. Brain potentials reflect covert recognition in a case of prosopagnosia. Cognitive Neuropsychology 21 (7), 691–718.
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153 Bruce, V., Young, A.W., 1986. Understanding face recognition. British Journal of Psychology 77, 305–327. Cacioppo, J.T., Crites, S.L., Gardner, W.L., Bernston, G.G., 1994. Bioelectrical echoes from evaluative categorizations: I. A late positive brain potential that varies as a function of trait negativity and extremity. Journal of Personality and Social Psychology 67 (1), 115–125. Campanella, S., Gaspard, C., Debatisse, D., Bruyer, R., Crommelinck, M., Guerit, J.M., 2002. Discrimination of emotional facial expressions in a visual oddball task: an ERP study. Biological Psychology 59 (3), 171–186. Courchesne, E., Hillyard, S.A., Galambos, R., 1975. Stimulus novelty, task relevance and the visual evoked potential in man. Electroencephalography and Clinical Neurophysiology 39, 131–143. Cuthbert, B.N., Schupp, H.T., Bradley, M.M., Birbaumer, N., Lang, P.J., 2000. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biological Psychology 52 (2), 95–111. Damasio, A.R., 1999. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Harcourt Brace, New York. Eimer, M., 2000a. Event related brain potentials distinguish processing stages involved in face perception and recognition. Clinical Neurophysiology 111, 694–705. Eimer, M., 2000b. Effects of face inversion on the structural encoding and recognition of faces: evidence from event-related potentials. Cognitive Brain Research 10, 145–158. Escera, C., Alho, K., Winkler, I., Na¨a¨ta¨nen, R., 1998. Neural mechanisms of involuntary attention to acoustic novelty and change. Journal of Cognitive Neuroscience 10, 590–604. Gala´n, L., Biscay, R., Rodriguez, J.L., Pe´rez-Abalo, M.C., Rodrı´guez, R., 1997. Testing topographic differences between event related brain potentials by using non-parametric combinations of permutation tests. Electroencephalography Clinical Neurophysiology 102 (3), 240–247. Gobbini, I., Haxby, J.V., 2007. Neural systems for recognition of familiar faces. Neuropsychologia 45 (1), 32–41. Gorno-Tempini, M.L., Price, C.J., 2001. Identification of famous faces and buildings: a functional neuroimaging study of semantically unique items. Brain 124, 2087–2097. Gunnar, M.R., Nelson, C.A., 1994. Event-related potentials in year-old infants: relations with emotionality and cortisol. Child Development 65 (1), 80–94. Haxby, J., Hoffman, E., Gobbini, I., 2000. The distributed human neural system for face perception. Trends in Cognitive Science 4, 223–233. Haxby, J., Horwitz, B., Ungerleider, L.G., Maisog, J.M., Pietrini, P., Grady, C.L., 1994. The functional organization of human extrastriate cortex: s PETrCBF study of selective attention to face and locations. Journal of Neuroscience 14, 6336–6353. Henson, R.N., Goshen-Gottstein, Y., Ganel, T., Otten, L.J., Quayle, A., Rugg, M.D., 2003. Electrophysiological and haemodynamic correlates of face perception, recognition and priming. Cerebral Cortex 13 (7), 793–805. Kanwisher, N., McDermott, J., Chun, M.M., 1997. The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 17, 4302–4311. Knight, R.T., 1984. Decreased response to novel stimuli after prefrontal lesions in man. Electroencephalography Clinical Neurophysiology 59 (1), 9–20. Kosaka, H., Omori, M., Iidaka, T., Murata, T., Shimoyama, T., Sadato, N., Yonekura, Y., Wada, Y., 2003. Neural substrates participating in acquisition of facial familiarity: an fMRI study. NeuroImage 20, 1734–1742. Kutas, M., McCarthy, G., Donchin, E., 1977. Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time. Science 197, 792– 795. Lazarus, R.S., 1984. On the primacy of cognition. The American Psychologist 39, 124–129. LeDoux, J.E., 1995. In search of an emotional system in the brain: leaping from fear to emotion consciousness. In: Gazzaniga, M. (Ed.), The Cognitive Neuroscience. MIT Press, Cambridge, MA, pp. 1049–1062. Leube, D.T., Erb, M., Grodd, W., Bartels, M., Kircher, T., 2003. Successful episodic memory retrieval of newly learned faces activates a left frontoparietal network. Cognitive Brain Research 18, 97–101. Leveroni, C.L., Seidenberg, M., Mayer, A.R., Mead, L.A., Binder, J.R., Rao, S.M., 2000. Neural systems underlying the recognition of familiar and newly learned faces. Journal of Neuroscience 20 (2), 878–886.
153
Levy, D.A., Granot, R., Bentin, S., 2001. Processing specificity for human voice stimuli: electrophysiological evidence. Neuroreport 12, 2653–2657. Levy, D.A., Granot, R., Bentin, S., 2003. Neural sensitivity to human voices: ERP evidence to task and attentional influences. Psychophysiology 40, 291–305. McCarthy, G., Wood, C.C., 1985. Scalp distributions of event related potentials: an ambiguity associated with analysis of variance models. Electroencephalography Clinical Neurophysiology 62, 203–208. Na¨a¨ta¨nen, R., 1992. Attention and Brain Function. Lawrence Erlbaum Associates, Inc., Hillsdale, New Jersey. Nakamura, K., Kawashima, R., Sato, N., Nakamura, A., Sugiura, M., Kato, T., Hatano, K., Ito, K., Fukuda, H., Schormann, T., Zilles, K., 2000. Functional delineation of the human occipito-temporal areas related to face and scene processing. A PET Study. Brain 123, 1903–1912. Olivares, E., Bobes, M.A., Aubert, E., Valdes-Sosa, M., 1994. Associative ERP effects with memories of artificial faces. Cognitive Brain Research 2, 39–48. Pessoa, L., 2005. To what extent are emotional visual stimuli processed without attention and awareness? Current Opinions in Neurobiology 15 (2), 188– 196. Phelps, E.A., 2006. Emotion and cognition: insights from studies of the human amygdale. Annual Review of Psychology 57, 27–53. Picton, T.W., 1992. The P300 wave of the human event-related potentials. Journal Clinical Neurophysiology 9, 456–479. Polich, J., 2003. Overview of P3a and P3b. In: Polich, J. (Ed.), Detection of Change: Event-Related Potential and fMRI Findings. Kluwer Academic Press, Boston, pp. 83–98. Ranganath, C., Paller, K.A., 1999. Frontal brain activity during episodic and semantic retrieval: insights from event-related potentials. Journal of Cognitive Neuroscience 11 (6), 598–609. Ranganath, C., Rainer, G., 2003. Neural mechanisms for detecting and remembering novel events. Nature Reviews. Neuroscience 4 (3), 193–202. Renault, B., Signoret, J., Debruille, B., Breton, F., Bolgert, F., 1989. Brain potentials reveal covert facial recognition in prosopagnosia. Neuropsychologia 27, 905–912. Rolls, E.T., 2007. The representation of information about faces in the temporal and frontal lobes. Neuropsychologia 45 (1), 124–143. Rossion, B., Schiltz, C., Crommelinck, M., 2003. The functionally defined right occipital and fusiform ‘‘face areas’’ discriminate novel from visually familiar faces. NeuroImage 19, 877–883. Ruchkin, D.S., Johnson, R., Canoune, H.L., Ritter, W., Hammer, M., 1990. Multiple source of P3b associated with different types of information. Psychophysiology 27, 157–176. Shah, N.J., Marshall, J.C., Zafiris, O., Schwab, A., Zilles, K., Markowitsch, H.J., Fink, G.R., 2001. The neural correlates of person familiarity. A functional magnetic resonance imaging study with clinical implications. Brain 124, 804–815. Schweinberger, S.R., Pickering, E.C., Jentzsch, I., Burton, M., Kaufmann, J.M., 2002. Event-related brain potential evidence for a response of inferior temporal cortex of familiar face repetitions. Cognitive Brain Research 14, 398–409. Schroger, E., Giard, M.H., Wolff, C., 2000. Auditory distraction: event-related potential and behavioral indices. Clinical Neurophysiology 111, 1450–1460. Squires, N., Squires, K., Hillyard, S.A., 1975. Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. Electroencphalography Clinical Neurophysiology 38, 387–401. Soltani, M., Knight, R.T., 2000. Neural origins of the P300. Critical Reviews in Neurobiology 14 (3/4), 199–224. Stone, A., Valentine, T., Davis, R., 2001. Face recognition and emotional valence: processing without awareness by neurologically intact participants does not simulate covert recognition in prosopagnosia. Cognitive, Affective and Behavioral Neuroscience 1 (2), 183–191. Sugiura, R., Kawashima, M., Nakamura, K., Sato, N., Nakamura, A., Kato, T., Hatano, K., Schormann, T., Zilles, K., Sato, K., Ito, K., Fukuda, H., 2001. Activation reduction in anterior temporal cortices during repeated recognition of faces of personal acquaintances. NeuroImage 13, 877–890. Tranel, D., Damasio, A.R., 1985. Knowledge without awareness: an autonomic index of facial recognition by prosopagnosics. Science 228, 1453–1454. Verleger, R., 1997. On the utility of P3 latency as an index of mental chronometry. Psychophysiology 34 (2), 131–156.
Brain potentials reflect access to visual and emotional memories for faces Maria A. Bobes a,*, Ileana Quin˜onez a, Jhoanna Perez a, Inmaculada Leon b, Mitchell Valde´s-Sosa a a
Cognitive Neuroscience Department, Cuban Center for Neuroscience, Ave. 25 y 158, Cubanacan, Apartado 6880, Havana, Cuba b Psychology Faculty, University of La Laguna, Campus de Guajara 38071, Tenerife, Spain Received 15 July 2006; accepted 19 January 2007 Available online 1 February 2007
Abstract Familiar faces convey different types of information, unlocking memories related to social–emotional significance. Here, the availability over time of different types of memory was evaluated using the time-course of P3 event related potentials. Two oddball paradigms were employed, both using unfamiliar faces as standards. The infrequent targets were, respectively, artificially-learned faces (devoid of social–emotional content) and faces of acquaintances. Although in both tasks targets were detected accurately, the corresponding time-course and scalp distribution of the P3 responses differed. Artificially-learned and acquaintance faces both elicited a P3b, maximal over centro-parietal sites, and a latency of 500 ms. Faces of acquaintances elicited an additional component, an early P3 maximal over frontal sites: with a latency of 350 ms. This suggests that visual familiarity can only trigger the overt recognition processes leading to the slower P3b, whereas emotional–social information can also elicit fast and automatic assessments (indexed by the frontal-P3) crucial for successful social interactions. # 2007 Elsevier B.V. All rights reserved. Keywords: P3b; P3a; Faces; Memory; Emotion; Familiarity; Social
1. Introduction Faces afford different types of useful social information. From faces, we can infer age, race, or sex, as well as the emotional states and intentions of others. Also important is the ability to recognize person-identity from familiar faces, which is at the heart of important social transactions. Familiar faces unlock memories related to hierarchical status, degree of friendship (or otherwise), and personality traits among other aspects, plus a host of biographical details all of which go beyond the memory traces leading to purely visual familiarity. This emotional and social information associated with specific individuals will be dubbed here as ‘‘emotion-from-identity’’ to distinguish it from the more thoroughly studied emotional information that is derived from facial expressions. Existing models of face processing (i.e. Bruce and Young, 1986; Haxby et al., 2000) accept the idea that different kinds of face-derived information are processed in distinct and
* Corresponding author. Tel.: +53 7 2083990; fax: +53 7 2086321. E-mail address: [email protected] (M.A. Bobes). 0301-0511/$ – see front matter # 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.biopsycho.2007.01.006
specialized brain sub-systems. Originally, models postulated separate routes for identity and emotional expression recognition, with little emphasis on emotion-from-identity, although more recent models are beginning to remedy this (Gobbini and Haxby, 2007; Rolls, 2007). Neuroimaging studies have begun to identify the neural structures comprising these sub-systems (Haxby et al., 1994; Kanwisher et al., 1997), with a number of studies examining the neural basis of face familiarity. Several studies have compared brain responses to previously known faces and new unfamiliar faces, but of course the results depend of the type and degree of knowledge acquired for the familiar faces used in the comparison. Studies with functional magnetic resonance imaging (fMRI) have evinced brain regions activated more by faces that are familiar at a purely visual level (i.e. only structural memories are available) than by completely unfamiliar faces. These areas include the medial and posterior right face fusiform area (FFA), the left posterior FFA and the posterior cingulate gyrus (Kosaka et al., 2003; Leube et al., 2003; Rossion et al., 2003). On the other hand, for familiar faces that, beyond visual familiarity, and also are associated with biographical, emotional and social cues (such as famous faces and faces of acquaintances)
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
additional brain regions are recruited. Such additional brain regions include areas in the temporal lobe, the occipitotemporal gyrus including the FFAs, the anterior and posterior cingulate cortex, medial and inferior frontal gyrus, and the orbitofrontal cortex (for famous > unfamiliar, see Gorno-Tempini and Price, 2001; Henson et al., 2003; Leveroni et al., 2000; for acquaintances > unfamiliar, see Nakamura et al., 2000; Shah et al., 2001; Sugiura et al., 2001). Ecological considerations suggest that brain areas handling emotionally relevant memories should activate rapidly (Phelps, 2006). However, it is difficult to infer the timing of these activations from neuroimaging studies due to their poor temporal resolution. Information about these time-courses could potentially be obtained with the higher temporal resolution of event related potentials (ERPs). However, even so several studies have been performed, conclusive information is not available yet. Early ERP components, highly sensitive to faces, such as the intracranial negativity at 200 ms (Allison et al., 1999), and the scalprecorded N170 (Bentin et al., 1996), are not sensitive to face familiarity (Eimer, 2000a,b; Henson et al., 2003). Potentially relevant ERPs effects related to familiarity have been reported at about 250–300 ms (Bentin and Deouell, 2000; Schweinberger et al., 2002). To our knowledge no previous studies have used ERPs to compare the time-course of activations for different types of memories related to faces, nor have they specifically examined the time-course of memory activation for emotion-from-identity. One approach to this issue is to use the well-known P3 ERP, which has previously been used to study face familiarity (Bobes et al., 2004; Gunnar and Nelson, 1994; Renault et al., 1989). The P3 is usually examined within an oddball paradigm, which consists of presenting rare target stimuli amidst a train of more frequent standard stimuli, the former evoking the P3. Although the P3 is not specifically related to faces processing (in contrast to the N170), it has been frequently used to index stimulus evaluation time (Kutas et al., 1977; Picton, 1992; Polich, 2003; but see Verleger, 1997). Since P3 can only arise after the categorization of the target stimuli as different from the standard, its onset latency offers an upper bound on the time needed for this processes. Hence, if infrequent familiar faces are interspersed amidst frequent familiar faces, the P3 can be used to index the availability of the information distinguishing these two types of faces, in other words, relevant memories for people. This strategy has even evinced a P3 in patients with prosopagnosia, who were unable to explicitly recognize familiar faces, thus demonstrating covert recognition (Renault et al., 1989; Bobes et al., 2004). In the present study, the availability over time of different types of memory related to person-identity was examined using two variants of the oddball paradigm. Both variants contained frequent unfamiliar faces as standard stimuli. In the first variant the infrequent target stimuli were newly-learned faces devoid of emotional and social content. These newly-learned were previously unfamiliar faces that had been studied to achieve a high degree of recognition accuracy. In the second variant, the target stimuli were faces of acquaintances with emotional significance (i.e., their close relatives and friends), which were
147
tailored to each subject. In both variants the target and standard stimuli differed purely in visual familiarity. In the case of acquaintances, the targets also afforded emotion-from-identity. Therefore, visual familiarity and emotion-from identity, confounded in most natural situations, could be dissociated. To validate the difference in emotional content for these two types of target faces, skin conductance responses (SCR) were measured. Previous studies have shown that SCR responses elicited by familiar (acquaintance and famous) faces are larger than those triggered by unfamiliar faces (Bauer, 1984; Tranel and Damasio, 1985). The P3 elicited by the targets were used as an index of the availability of the relevant memories associated to the target faces: visual familiarity for newly-learned faces and both visual familiarity and emotion-from-identity for acquaintance faces. The onset latencies of the P3 could provide an upper bound to the availability of the memory traces. An important question was thus whether emotion-from-identity becomes available only after visual familiarity is overtly recognized, as predicted by some theories of emotion (Lazarus, 1984), or whether emotion-from-identity can be processed in parallel with (or even before) explicit recognition of visual familiarity, as suggested by recent models for facial expressions (Gobbini and Haxby, 2007). Moreover, consideration of the scalp topography of the P3 elicited by the two kinds of familiar faces was interesting in itself. Previous work has shown that P3 is really a family of distinct late positive components, each sensitive to different experimental factors, and with divergent distributions over the scalp, thus reflecting distinct mental operations (Squires et al., 1975; Soltani and Knight, 2000). For example, the parietocentral P3b is generated when active attention is given to the stimuli (Ruchkin et al., 1990; Picton, 1992), whereas the frontally distributed P3a (Squires et al., 1975) and the novelty P3 (Courchesne et al., 1975) are considered to be more automatic and less dependent on attention (Escera et al., 1998; Na¨a¨ta¨nen, 1992; Schroger et al., 2000). This novelty P3 is related to the stimulus prediction and is not modality specific (Knight, 1984; Ranganath and Paller, 1999, and see Ranganath and Rainer, 2003 for a review). Therefore, the scalp distribution of the P3 in the two paradigms used in the present study could indicate if different neural systems are involved in the processing of visual familiarity and of emotion-from-identity. 2. Methods 2.1. Participants Ten healthy adults, six males and four females, with ages ranging from 23 to 33 years (mean = 27) participated in the study as non-paid volunteers. All the participants had university degrees, were right-handed (as ascertained by personal report) and had normal or corrected-to-normal vision.
2.2. Face stimuli The stimuli consisted of digitized photographs of faces, which were all software-edited using Adobe Photoshop. All photographs were converted to grayscale, and their background removed. An attempt was made to homogenize the stimuli with respect to average luminance contrast using this software. Each
148
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
face was displayed within a circular border, which masked the external features, clothes, and background. All faces consisted of frontal views and could be either familiar or unfamiliar to the subjects (see Bobes et al., 2004, for a similar procedure and testing for low level physical difference between known and unknown faces). Two different sets of familiar faces were obtained, acquaintance faces, which were selected among the family and close friends of each subject (i.e. this was different for each subject), and newly-learned faces, which were previously unknown faces, learned by practicing in the lab (as explained below). Each of the familiar faces sets, randomly mixed with the unfamiliar faces, comprised a stimulation block for an oddball paradigm: Block I, acquaintance-unknown faces, in which the familiar target faces were acquaintances and Block II, newlylearned-unknown faces, in which the learned faces was the target stimuli. Each block contained 15 different familiar faces and 75 different unfamiliar faces, and every face was presented twice, meaning that each stimuli block comprised a total of 30 familiar (17%) and 150 unfamiliar (83%) faces for each stimuli block. Different unfamiliar faces were used as standards in each block and the order of face presentation in the two blocks was counterbalanced across subjects. Face stimuli were presented for 1000 ms, with an inter-stimulus interval (ISI) of 1000 ms. Subjects were required to discriminate between familiar and unfamiliar faces and to respond via the computer keyboard during the ISI (a go/ go design). Task performance was evaluated by measuring hits percent corrected by chance level: PHits PFalseAlarms 1 PFalseAlarms in each subject.
2.3. Training procedure for newly-learned faces Learning took place on six sessions (two session per day). Each session consist of a study and a test phase. In the study phase, the subject viewed a training set of 15 initially unknown faces (the same in all sessions), presented on the CRT monitor in a random order, with onset and offset times under subject’s control. The test phase consisted of a familiarity decision task. The 15 ‘‘old’’ faces for the training set were all presented again, randomly mixed with 15 unfamiliar ‘‘new’’ faces from the unknown face pool. The unfamiliar faces were never repeated. The subjects’ responses were measured via two mouse-keys (familiar or unfamiliar). Immediate acoustic feedback on the accuracy of the response was provided. The faces in the training set were thus seen 12 times during training. (This procedure was modified from Olivares et al., 1994.)
2.4. ERP recording and analysis Subjects sat in front of a sVGA monitor (about 1 m from the observer) and were instructed to minimize body and eye movements during the experiments. Stimuli on the CRT subtended a vertical visual angle of 3.78 and a horizontal visual angle of 2.58. Data acquisition was carried out with 120 monopolar derivations, using electrodes mounted in an elastic cap homogeneously distributed over the scalp, as well as two channels of EOG signals on a MEDICID-128 system (Neuronic, SA, Havana). A notch filter with peak at 60 Hz was used and the signals were amplified by a factor of 10,000 and filtered between 0.5 and 30 Hz (3 dB down). All electrodes were referred to linked earlobes and then re-referenced to an average reference, and the inter-electrode impedance was always below 10 kV. The onset of the stimuli served to synchronize data collection. The EEG was digitally recorded at a sampling rate of 200 Hz, with 700 ms epochs, and a 300 ms pre-stimulus baseline. Each EEG epoch was stored on magnetic disk, and was visually inspected offline. Those epochs with generalized artifacts or detectable eye-movement in the EOG were eliminated (19% on average). In the 120-channel recording, electrodes with excessive noise were eliminated and substituted by an interpolation of the 11 closest neighbors. A mean of 19 channels (1.58% on average) was interpolated across subjects. Averaged evoked responses were digitally low-pass filtered (18 Hz cut-off). ERP amplitudes were corrected by subtracting the average pre-stimulus amplitude value. Only standard trials preceding deviant stimuli were averaged for this condition in
order to equalize the signal-to-noise ratio of the resulting ERPs with those related to the standard ones. For display purposes, scalp voltage maps were calculated for the average amplitude in a time window centered around the maximum of the P3 peak (see Fig. 4). These voltages were expressed as a percentage of the largest electrode measurement. Thus maps from the two tasks could be compared after eliminating possible differences in amplitude scale. These topographic 3D maps are plotted over the surface of the MNI average brain (ICBM 305). An 11-neighbors interpolation method was used for obtaining intermediate points in the map. Electrophysiological data were analyzed with permutation tests. Permutation tests have the following advantages for ERP research: the tests are distribution free, no assumptions of an underlying correlation structure between time points or electrodes are required, and they provide exact p-values for any number of subjects, time points and recording sites (see Blair and Karniski, 1993; Gala´n et al., 1997). For tests of differences between the ERPs associated with two conditions in a group of subjects, the analysis consists of the following steps: 1. The null hypothesis (H0) of equal mean values for the two groups of ERPs is decomposed into the marginal hypotheses H0dt: mdt1 = mdt2, where mdtj is the mean value of the ERP obtained in the d site and t time for the j experimental condition. 2. All the marginal hypotheses are tested by:
a. Computing the t statistic for the original ERP data contrasting the two conditions in n subjects. b. Obtaining a large number of permutation resample from the data (without replacement) and constructing the permutation distribution of the t statistic (the complete permutation space, 2n). c. Finding the p-value by locating the original statistic on this t statistic permutation distribution. A set of nested decisions about hypotheses H0 is performed by using the maximum of several combinations of marginal hypothesis permutation tests thus controlling the experiment wise error for the simultaneous univariate comparisons. Combining all sites and time points give a global test; combining all time points for each electrode gives a test for sites, combining for each time point across sites gives a test for temporal differences. This last option was used to estimate the P3 onset latencies in each subject by contrasting single-trial epochs associated with standards and those epochs associated with targets. A similar procedure was used with the average ERPs for all subjects in order to estimate the amplitude differences between two experimental conditions. The statistical difference between two scalp distributions was also tested applying permutation techniques. In this case, the vector of amplitudes used to obtain the scalp maps (see above) was firstly normalized as suggested by McCarthy and Wood (1985) to eliminate differences in amplitudes and allow for testing of topographic distributions. Then the global test based on permutations (described above) was performed.
2.5. SCR In a separate session, SCRs were measured for two stimuli blocks, including acquaintance or newly-learned faces respectively. Each block comprised a total of 52 faces, 39 unfamiliar faces randomly mixed with 13 familiar faces (a subset of those presented in the ERP paradigm). The stimuli were presented on a sVGA monitor as explained before. Each stimulus was presented for a duration of 2 s, and the next stimulus was delayed until the SCR recording related to the previous one had returned to baseline. This resulted in inter-stimulus intervals longer than 20 s. Subjects were instructed to passively view the faces. Electrodermal activity was recorded by using Ag/AgCl electrodes taped to the palmar surface of the proximal phalanx of left index and middle fingers after
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153 cleaning the attachment sites and applying conductive gel, and the signal was fed via a skin conductance-processing unit (GSR-2100, Nihon Kohden, Japan) to a channel of the MEDICID III/E system. The filtered analog output of the SCR was displayed online and recorded digitally (sample rate, 100 Hz), in synchronism with the onset of the face, using custom-made software following the procedure described in Bobes et al. (2004).
3. Results 3.1. SCR The amplitudes of the SCR responses for the two experimental blocks (acquaintance versus unknown faces, and newlylearned versus unknown faces) are presented in Fig. 1. In the acquaintance versus unknown faces block, the mean SCR response to acquaintance faces (mean = 0.6 mS, S.D. = 0.39) was larger than for unknown faces (0.42 mS, S.D. = 0.29). However, in the newly-learned versus unknown faces block, the SCR response to newly-learned (mean = 0.449 mS, S.D. = 0.3) and unknown faces (mean = 0.461 mS, S.D. = 0.4) were very similar. A repeated measures ANOVA was performed, with two within-subject factors: FAMILIARITY (known versus unknown) and EXPERIMENTAL BLOCK (newly-learned versus acquaintance). Although the two mains effects were not significant, the interaction between FAMILIARITY and EXPERIMENTAL BLOCK was marginally significant (F(1,9) = 4.4, p = 0.06). Planned comparisons showed a highly significant difference between the SCR to acquaintance faces and unknown faces (F(1,9) = 7.3, p = 0.025), whereas no difference was found between the SCR to newly-learned and unknown faces (F(1,9) = 0.06, p = 0.8). These results confirm the lack of emotional valence of newly-learned faces. 3.2. Behavioral data during the ERP recording session Familiar faces were accurately detected in the two experimental ERP blocks. The mean percentage of hits for
Fig. 1. Skin conductance response (SCR) obtained in the two stimulation blocks. On the left, the responses obtained to acquaintance faces randomly interspersed with unknown faces. On the right, the responses obtained to newlylearned faces randomly interspersed with unknown faces. Means across the subject sample and S.E.M.s (error bars) are presented.
149
newly-learned faces was 94% (S.D. = 0.07) and 96% (S.D. = 0.05) for acquaintances. Accuracy was not different for the two tasks (t = 88, d.f. = 18, p = 0.39). 3.3. ERPs The grand average ERPs elicited by newly-learned and unfamiliar faces are overlaid in Fig. 2A. The amplitudes of the earlier ERP components elicited by these two stimuli were not different. Both responses exhibited the N170 component with similar latency and amplitude. For newly-learned faces, the mean latency of the N170 at T6 was 180 ms (S.D. = 36.6) and for unfamiliar faces it was 176 ms (S.D. = 25.5). After around 400 ms, the recordings corresponding to the two stimuli differed; the ERPs associated with the newly-learned faces presented an enhanced positivity widely distributed over the head, with larger amplitudes at central and posterior sites. The peak latency of this positivity is about 505 ms at Pz. We considered this component to be a P3b. The effect of familiarity was tested statistically by performing permutation analysis for dependent samples over the averaged ERPs obtained in each subject. The comparison between ERPs associated with newlylearned and unfamiliar faces are shown in Fig. 2B, displaying the probability values obtained by the permutation test for each time point (amplitudes for targets > amplitudes for standards). The test was significant in the time window between 450 and 600 ms ( p < 0.05), which corresponds with the timing of the P3b elicited by newly-learned faces. The ERPs’ waveforms evoked by acquaintance faces are shown in Fig. 3A. No difference was observed in components earlier than 300 ms between the ERPs elicited by the two stimuli types. The mean N170 latency at T6 was 183 ms (S.D. = 37) for acquaintance faces and 176 ms (S.D. = 25.1) for unfamiliar faces. A repeated measures ANOVA including TYPE of familiar faces (newly-learned versus acquaintances) and FAMILIARITY (familiar versus unfamiliar) did not find significant differences in N170 latency due to TYPE (F(1,8.1) = 0.11, p = 0.747) or to FAMILIARITY (F(1,292) = 0.306, p = 0.594). No significant interaction between these two factors was found (F(1,32) = 1.92, p = 0.199). A centro-parietal positive component was elicited by acquaintance faces (Fig. 3A). Similar to that found for newly-learned faces, this component reached a maximum around 500 ms at Pz, and was also considered a P3b. However, in the case of acquaintance faces an additional positivity was produced at frontal sites, which shall be referred to as the frontal-P3. The peak latency of this frontal positivity was 355 ms at Fz. Results of the permutation tests for the P3 effect (amplitudes for targets > amplitudes for standards) are shown in Fig. 3B ( p < 0.05). The tests were significant in two time windows: one between 355 and 425 ms and another from 450 and 550 ms. This time windows correspond with the timing of the two P3b and the frontal-P3. The scalp distributions of the P3 components elicited by both types of familiar faces are shown in more detail in Fig. 4. The P3b component elicited by newly-learned faces (in a time window between 500 and 580 ms) was of maximum amplitude
150
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
Fig. 2. (A) ERPs obtained during the discrimination of face familiarity in the newly-learned faces block. ERPs associated with unfamiliar faces (thin line) are overlaid on ERPs elicited by newly-learned faces (thick lines) in selected derivations. (B) Results of permutation over the 120 channels of the individuals ERPs are shown for each time point. Probability is presented in a logarithmic scale. Markers indicate time points where probability was below 0.05.
at central–parietal sites, although widely spread across the scalp (Fig. 4A). A similar topography was exhibited by the P3b elicited by acquaintance faces in the 460–570 ms time window. The topography of these two P3b was not statistically different as evinced by the permutation test (global p = 0.29). In contrast, the earlier frontal-P3 – elicited by acquaintance faces – had a
distribution restricted to frontal sites, and was of low amplitude at central and posterior sites (Fig. 4C). The statistical analysis confirmed that this topography was different from the P3b component elicited by acquaintances’ faces (global p = 0.0001), and also different from the P3b elicited by newly-learned faces (global p = 0.002). Tests of marginal
Fig. 3. (A) ERPs obtained during the discrimination of face familiarity in the acquaintance faces block. ERPs associated with unfamiliar faces (thin line) are overlaid on ERPs elicited by acquaintance faces (thick lines) in selected derivations. (B) Results of permutation over the 120 channels of the individuals ERPs are shown for each time point. Probability is presented in a logarithmic scale. Markers indicate time points where probability was below 0.05.
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
151
Fig. 4. Voltage map representing the scalp distribution of the positive components in the ERPs associated to familiar faces. (A) P3b for newly learned faces. (B) P3b for acquaintance faces. (C) P3a for acquaintance faces. These maps were obtained for the mean amplitude in the time windows including the positive components; expressing amplitudes as a percentage of the maximum. Three different views are depicted: left, frontal view; center, from above with the nose up; right, back view.
hypotheses showed that the earlier frontal-P3 component was significantly larger in amplitude, as compared to the P3b elicited by acquaintance faces, at electrodes Fp1, Fp2, AF7, AF8, AF3, AF4, C4 and CP4 (all p < 0.05). To summarize, the scalp distribution of the P3b associated to acquaintance faces is very similar to the distribution of the P3b elicited by newlylearned faces, but its topography differed from the distribution of frontal-P3 elicited by acquaintance faces. The onset latency of the familiarity effects was estimated in each subject using the permutation test over the single-trial epochs of the EEG. The onset latency in each case was measured as the first point showing significant amplitude difference between familiar and unfamiliar faces ( p < 0.05), considering the 120 channels together. The mean value for acquaintance faces was 314 ms (S.D. = 76), which was significantly earlier than the 420 ms (S.D. = 82) obtained for newly-learned faces (t = 4.48, d.f. = 9, p < 0.0001). 4. Discussion Accurate detection was achieved for both acquaintance and newly-learned face targets. Faces of acquaintances elicited larger SCR responses than did newly-learned and unfamiliar faces. Both types of familiar-faces elicited a robust P3. However, the time-course and scalp distribution of the P3 response differed between the two tasks. Newly-learned faces elicited a positivity around 500 ms (larger over centro-parietal sites), whereas faces of acquaintances elicited two sub-
components: one similar to that elicited by newly-learned faces and another (earlier) sub-component with latency at about 350 ms (larger over frontal sites). The similar recognition accuracy for the two types of familiar faces indicates that the artificial over-training for newly-learned faces created memory traces of similar strength as those existing for acquaintances. These memories were necessarily of a purely visual nature (i.e. limited to face structural knowledge) since no other information was provided about these faces. On the other hand, the difference in emotional significance between newly-learned and acquaintance faces was clearly validated by the SCR results. The SCR reflects emotional arousal, with larger responses to emotional (i.e. faces of acquaintance or famous persons) relative to neutral events (i.e. unknown faces, Bauer, 1984; Stone et al., 2001; Tranel and Damasio, 1985). As expected, here faces of acquaintances elicited large SCRs whereas responses for unknown faces were of low amplitude. Newly-learned faces also elicited small SCRs, which confirms their lack of emotional and social significance. Therefore, the contributions of purely visual familiarity and of emotion-from-identity can be disentangled comparing the two sorts of faces. This is a difficult goal to reach using only naturally learned faces for which a closer acquaintance generates not only more emotional and social memories, but also more visual familiarity with the other person’s face. Newly-learned faces evoked a large P3 component with a centro-parietal scalp topography, which corresponds to what
152
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153
has been described as P3b (see Picton, 1992, for a review). The latency of this P3b is somehow delayed compared to the typical 300 ms described for oddball tasks based on the discrimination of simple sensory features. However, it is in the same latency range as previously described P3s obtained with face stimuli (Bobes et al., 2004; Campanella et al., 2002; Gunnar and Nelson, 1994; Renault et al., 1989). This is consistent with the idea that P3b latency varies widely as a function of stimulus material and task difficulty (Kutas et al., 1977; Picton, 1992; Polich, 2003). The P3b component is generated only when active attention is allocated to the oddball stimuli, and is of larger amplitude if the categorization decisions are more confident (Ruchkin et al., 1990; Picton, 1992), and thus is construed as being linked to mental operations such as the explicit updating of working memory (Polich, 2003). As mentioned above, the P3 for the acquaintance targets is comprised by two sub-components with different timings. The scalp distribution of the response in the two time regions was different, suggesting that they originated from different neural sources. The later sub-component, with latency near 500 ms, exhibited a centro-parietal distribution and is probably similar to the component elicited by newly-learned face targets and can be considered a P3b. In contrast, the earlier (and perhaps more interesting) sub-component has a frontal distribution and is more similar to the P3a (Squires et al., 1975), or the novelty P3 (Courchesne et al., 1975) which appears to unusual or arousing distracters. However, further research is needed before one can confidently label the frontal-P3 as P3a. This frontal-P3 is not elicited by newly-learned faces implying that emotion-fromidentity is a necessary condition for its generation. Other late positive components have been reported in response to emotional stimuli, such as the late positive potential (LPP) obtained for arousing pictures in the affective oddball paradigm (Cacioppo et al., 1994; Cuthbert et al., 2000). However, the LPP is of a longer latency and has a centro-parietal distribution which implies that it is different from the frontal-P3 described here. Salient significant stimuli can elicit a frontal positive component under some conditions. When interspersed with sounds from musical instruments, human voices elicit the voice sensitive response (VSR), a positive component reminiscent of the P3a (Levy et al., 2001). More than novelty, this positive component seems to be triggered by the significance of voice stimuli for humans listeners (Levy et al., 2003), and consequently could be related to the frontal-P3 described here, given the emotional significance of acquaintance faces. The fact that only faces of acquaintances elicit a frontal-P3, similar in latency and topography to the P3a is interesting. On one hand, only the faces of acquaintances afforded emotionfrom-identity in the present study. On the other hand, P3a is considered to reflect more automatic and less attentiondependent processes than P3b (Escera et al., 1998; Na¨a¨ta¨nen, 1992; Schroger et al., 2000). This opens the possibility that the frontal-P3 described here reflects a faster, and perhaps more automatic, evaluation of emotion-from-identity. The idea of fast and automatic processing of emotionally significant stimuli (such as facial expressions) has been frequently discussed (Damasio, 1999; LeDoux, 1995; Phelps, 2006; see Pessoa,
2005, for a critical review), although to our knowledge has not been specifically examined for emotion-from-identity. A component similar to the frontal-P3 was found with the same oddball design used here, but in a study of a case with dense, acquired prosopagnosia (Bobes et al., 2004). Despite his severe inability for overt recognition of faces, this case presented larger SCRs to faces of acquaintances than to unfamiliar faces and selected the faces of acquaintances above chance if asked about trustworthiness. The onset latency of the P3 elicited by previously familiar (but now unrecognized) faces was around 400 ms and presented a centro-frontal maximum. This would be consistent with the idea that the frontal-P3 could be more automatic in nature than the P3b (which was absent in the patient). Further studies are necessary to verify this hypothesis. As pointed out in the introduction, onset latencies of the P3 components provide an upper bound for the timing of the memory activations that enable visual familiarity and emotionfrom-identity. The frontal-P3 elicited by acquaintances had an onset latency that was about 100 ms shorter than the P3b elicited by both types of target faces. This precluded the possibility that the memories coding emotion-from-identity were activated in series after the stimulus evaluation leading to the P3b was completed. While not the only possible explanation, this result is consistent with a faster route to emotion-from-identity parallel to the explicit and attentive processing leading to the P3b. The latency of the N170 face-specific component was equivalent for all types of faces used in the study. The N170 is considered to index structural coding of faces in ventral extrastriate cortex (Bentin and Deouell, 2000). Therefore, differences in processing times along these hypothetical routes would have to originate after, or in parallel, to this structural coding. To conclude, our data indicates that the social and emotional significance of faces triggers a relatively fast (and perhaps more automatic) response that is indexed by a frontal-P3, which does not depend on the slower and overt stimulus evaluation of familiarity that leads to a P3b. The neuroanatomy of these processes remains to be worked out.
References Allison, T., Puce, A., Spencer, G., McCarthy, G., 1999. Electrophysiological studies of human face perception. I. Potentials generated in occipitotemporal cortex by face and non-face stimuli. Cerebral Cortex 9, 415–430. Bauer, R.M., 1984. Autonomic recognition of names and faces in prosopagnosia: a neuropsychological application of the guilty knowledge test. Neuropsychologia 22, 457–469. Bentin, S., Allison, T., Puce, A., Perez, A., McCarthy, G., 1996. Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience 8, 551–556. Bentin, S., Deouell, L.Y., 2000. Structural encoding and identification in face processing: ERP evidence for separate mechanisms. Cognitive Neuropsychology 17, 35–54. Blair, R., Karniski, W., 1993. An alternative method for significance testing of waveform difference potential. Psychophysiology 30, 518–524. Bobes, M.A., Lopera, F., Garcia, M., Dı´az Comas, L., Galan, L., Valdes-Sosa, M., 2004. Brain potentials reflect covert recognition in a case of prosopagnosia. Cognitive Neuropsychology 21 (7), 691–718.
M.A. Bobes et al. / Biological Psychology 75 (2007) 146–153 Bruce, V., Young, A.W., 1986. Understanding face recognition. British Journal of Psychology 77, 305–327. Cacioppo, J.T., Crites, S.L., Gardner, W.L., Bernston, G.G., 1994. Bioelectrical echoes from evaluative categorizations: I. A late positive brain potential that varies as a function of trait negativity and extremity. Journal of Personality and Social Psychology 67 (1), 115–125. Campanella, S., Gaspard, C., Debatisse, D., Bruyer, R., Crommelinck, M., Guerit, J.M., 2002. Discrimination of emotional facial expressions in a visual oddball task: an ERP study. Biological Psychology 59 (3), 171–186. Courchesne, E., Hillyard, S.A., Galambos, R., 1975. Stimulus novelty, task relevance and the visual evoked potential in man. Electroencephalography and Clinical Neurophysiology 39, 131–143. Cuthbert, B.N., Schupp, H.T., Bradley, M.M., Birbaumer, N., Lang, P.J., 2000. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biological Psychology 52 (2), 95–111. Damasio, A.R., 1999. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Harcourt Brace, New York. Eimer, M., 2000a. Event related brain potentials distinguish processing stages involved in face perception and recognition. Clinical Neurophysiology 111, 694–705. Eimer, M., 2000b. Effects of face inversion on the structural encoding and recognition of faces: evidence from event-related potentials. Cognitive Brain Research 10, 145–158. Escera, C., Alho, K., Winkler, I., Na¨a¨ta¨nen, R., 1998. Neural mechanisms of involuntary attention to acoustic novelty and change. Journal of Cognitive Neuroscience 10, 590–604. Gala´n, L., Biscay, R., Rodriguez, J.L., Pe´rez-Abalo, M.C., Rodrı´guez, R., 1997. Testing topographic differences between event related brain potentials by using non-parametric combinations of permutation tests. Electroencephalography Clinical Neurophysiology 102 (3), 240–247. Gobbini, I., Haxby, J.V., 2007. Neural systems for recognition of familiar faces. Neuropsychologia 45 (1), 32–41. Gorno-Tempini, M.L., Price, C.J., 2001. Identification of famous faces and buildings: a functional neuroimaging study of semantically unique items. Brain 124, 2087–2097. Gunnar, M.R., Nelson, C.A., 1994. Event-related potentials in year-old infants: relations with emotionality and cortisol. Child Development 65 (1), 80–94. Haxby, J., Hoffman, E., Gobbini, I., 2000. The distributed human neural system for face perception. Trends in Cognitive Science 4, 223–233. Haxby, J., Horwitz, B., Ungerleider, L.G., Maisog, J.M., Pietrini, P., Grady, C.L., 1994. The functional organization of human extrastriate cortex: s PETrCBF study of selective attention to face and locations. Journal of Neuroscience 14, 6336–6353. Henson, R.N., Goshen-Gottstein, Y., Ganel, T., Otten, L.J., Quayle, A., Rugg, M.D., 2003. Electrophysiological and haemodynamic correlates of face perception, recognition and priming. Cerebral Cortex 13 (7), 793–805. Kanwisher, N., McDermott, J., Chun, M.M., 1997. The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 17, 4302–4311. Knight, R.T., 1984. Decreased response to novel stimuli after prefrontal lesions in man. Electroencephalography Clinical Neurophysiology 59 (1), 9–20. Kosaka, H., Omori, M., Iidaka, T., Murata, T., Shimoyama, T., Sadato, N., Yonekura, Y., Wada, Y., 2003. Neural substrates participating in acquisition of facial familiarity: an fMRI study. NeuroImage 20, 1734–1742. Kutas, M., McCarthy, G., Donchin, E., 1977. Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time. Science 197, 792– 795. Lazarus, R.S., 1984. On the primacy of cognition. The American Psychologist 39, 124–129. LeDoux, J.E., 1995. In search of an emotional system in the brain: leaping from fear to emotion consciousness. In: Gazzaniga, M. (Ed.), The Cognitive Neuroscience. MIT Press, Cambridge, MA, pp. 1049–1062. Leube, D.T., Erb, M., Grodd, W., Bartels, M., Kircher, T., 2003. Successful episodic memory retrieval of newly learned faces activates a left frontoparietal network. Cognitive Brain Research 18, 97–101. Leveroni, C.L., Seidenberg, M., Mayer, A.R., Mead, L.A., Binder, J.R., Rao, S.M., 2000. Neural systems underlying the recognition of familiar and newly learned faces. Journal of Neuroscience 20 (2), 878–886.
153
Levy, D.A., Granot, R., Bentin, S., 2001. Processing specificity for human voice stimuli: electrophysiological evidence. Neuroreport 12, 2653–2657. Levy, D.A., Granot, R., Bentin, S., 2003. Neural sensitivity to human voices: ERP evidence to task and attentional influences. Psychophysiology 40, 291–305. McCarthy, G., Wood, C.C., 1985. Scalp distributions of event related potentials: an ambiguity associated with analysis of variance models. Electroencephalography Clinical Neurophysiology 62, 203–208. Na¨a¨ta¨nen, R., 1992. Attention and Brain Function. Lawrence Erlbaum Associates, Inc., Hillsdale, New Jersey. Nakamura, K., Kawashima, R., Sato, N., Nakamura, A., Sugiura, M., Kato, T., Hatano, K., Ito, K., Fukuda, H., Schormann, T., Zilles, K., 2000. Functional delineation of the human occipito-temporal areas related to face and scene processing. A PET Study. Brain 123, 1903–1912. Olivares, E., Bobes, M.A., Aubert, E., Valdes-Sosa, M., 1994. Associative ERP effects with memories of artificial faces. Cognitive Brain Research 2, 39–48. Pessoa, L., 2005. To what extent are emotional visual stimuli processed without attention and awareness? Current Opinions in Neurobiology 15 (2), 188– 196. Phelps, E.A., 2006. Emotion and cognition: insights from studies of the human amygdale. Annual Review of Psychology 57, 27–53. Picton, T.W., 1992. The P300 wave of the human event-related potentials. Journal Clinical Neurophysiology 9, 456–479. Polich, J., 2003. Overview of P3a and P3b. In: Polich, J. (Ed.), Detection of Change: Event-Related Potential and fMRI Findings. Kluwer Academic Press, Boston, pp. 83–98. Ranganath, C., Paller, K.A., 1999. Frontal brain activity during episodic and semantic retrieval: insights from event-related potentials. Journal of Cognitive Neuroscience 11 (6), 598–609. Ranganath, C., Rainer, G., 2003. Neural mechanisms for detecting and remembering novel events. Nature Reviews. Neuroscience 4 (3), 193–202. Renault, B., Signoret, J., Debruille, B., Breton, F., Bolgert, F., 1989. Brain potentials reveal covert facial recognition in prosopagnosia. Neuropsychologia 27, 905–912. Rolls, E.T., 2007. The representation of information about faces in the temporal and frontal lobes. Neuropsychologia 45 (1), 124–143. Rossion, B., Schiltz, C., Crommelinck, M., 2003. The functionally defined right occipital and fusiform ‘‘face areas’’ discriminate novel from visually familiar faces. NeuroImage 19, 877–883. Ruchkin, D.S., Johnson, R., Canoune, H.L., Ritter, W., Hammer, M., 1990. Multiple source of P3b associated with different types of information. Psychophysiology 27, 157–176. Shah, N.J., Marshall, J.C., Zafiris, O., Schwab, A., Zilles, K., Markowitsch, H.J., Fink, G.R., 2001. The neural correlates of person familiarity. A functional magnetic resonance imaging study with clinical implications. Brain 124, 804–815. Schweinberger, S.R., Pickering, E.C., Jentzsch, I., Burton, M., Kaufmann, J.M., 2002. Event-related brain potential evidence for a response of inferior temporal cortex of familiar face repetitions. Cognitive Brain Research 14, 398–409. Schroger, E., Giard, M.H., Wolff, C., 2000. Auditory distraction: event-related potential and behavioral indices. Clinical Neurophysiology 111, 1450–1460. Squires, N., Squires, K., Hillyard, S.A., 1975. Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. Electroencphalography Clinical Neurophysiology 38, 387–401. Soltani, M., Knight, R.T., 2000. Neural origins of the P300. Critical Reviews in Neurobiology 14 (3/4), 199–224. Stone, A., Valentine, T., Davis, R., 2001. Face recognition and emotional valence: processing without awareness by neurologically intact participants does not simulate covert recognition in prosopagnosia. Cognitive, Affective and Behavioral Neuroscience 1 (2), 183–191. Sugiura, R., Kawashima, M., Nakamura, K., Sato, N., Nakamura, A., Kato, T., Hatano, K., Schormann, T., Zilles, K., Sato, K., Ito, K., Fukuda, H., 2001. Activation reduction in anterior temporal cortices during repeated recognition of faces of personal acquaintances. NeuroImage 13, 877–890. Tranel, D., Damasio, A.R., 1985. Knowledge without awareness: an autonomic index of facial recognition by prosopagnosics. Science 228, 1453–1454. Verleger, R., 1997. On the utility of P3 latency as an index of mental chronometry. Psychophysiology 34 (2), 131–156.