Deep and shallow encoding effects on face recognition: An ERP study

International Journal of Psychophysiology 78 (2010) 239–250

Contents lists available at ScienceDirect

International Journal of Psychophysiology j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / i j p s yc h o

Deep and shallow encoding effects on face recognition: An ERP study Tessa Marzi ⁎, Maria Pia Viggiano 1 Department of Psychology, University of Florence, Via di San Salvi 12, 50100 Firenze, Italy

a r t i c l e

i n f o

Article history: Received 29 March 2010 Received in revised form 17 August 2010 Accepted 17 August 2010 Available online 24 August 2010 Keywords: Event related potentials (ERPs) Face inversion Levels of processing Recognition memory Old/new effect

a b s t r a c t Event related potentials (ERPs) were employed to investigate whether and when brain activity related to face recognition varies according to the processing level undertaken at encoding. Recognition was assessed when preceded by a “shallow” (orientation judgement) or by a “deep” study task (occupation judgement). Moreover, we included a further manipulation by presenting at encoding faces either in the upright or inverted orientation. As expected, deeply encoded faces were recognized more accurately and more quickly with respect to shallowly encoded faces. The ERP showed three main findings: i) as witnessed by more positive-going potentials for deeply encoded faces, at early and later processing stage, face recognition was influenced by the processing strategy adopted during encoding; ii) structural encoding, indexed by the N170, turned out to be “cognitively penetrable” showing repetition priming effects for deeply encoded faces; iii) face inversion, by disrupting configural processing during encoding, influenced memory related processes for deeply encoded faces and impaired the recognition of faces shallowly processed. The present study adds weight to the concept that the depth of processing during memory encoding affects retrieval. We found that successful retrieval following deep encoding involved both familiarity- and recollection-related processes showing from 500 ms a fronto-parietal distribution, whereas shallow encoding affected only earlier processing stages reflecting perceptual priming. © 2010 Elsevier B.V. All rights reserved.

1. Introduction The ability to efficiently encode and recognize a face is an essential component of successful social interaction. Yet, it is not well understood how different levels of processing during encoding influence recognition memory. Face recognition refers to the judgement that a face has been previously experienced and correct recognition crucially depends on a match between initial and subsequent encounter. This process includes structural encoding, that is, the extraction of an invariant representation, and a comparison with stored representations of previously encountered faces. An important question is whether and when during the time course of face recognition the retrieval of a face is influenced by different encoding strategies. Answering this question is the primary aim of this study. A great deal of research has demonstrated that recognition memory depends on the encoding mode; stimuli processed more deeply are better remembered than those processed in a perceptual or shallow fashion (Craik and Lockhart, 1972; Craik, 2002, Craik and Tulving, 2004). The classical framework proposed by Craik and Lockhart (1972) suggests that individuals are better at remembering an item that has been encoded with a semantic (deep encoding) rather with a more ⁎ Corresponding author. Tel.: + 39 055 6237830; fax: + 39 055 6236047. E-mail addresses: tessa.marzi@unifi.it (T. Marzi), [email protected]fi.it (M.P. Viggiano). 1 Tel.: + 39 055 6237830; fax: + 39 055 6236047. 0167-8760/$ – see front matter © 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.ijpsycho.2010.08.005

superficial processing (shallow encoding). Deeper levels of processing involve the extraction of meaning and the deeper the level, i.e., the greater the degree of semantic or abstract processing, the more robust the memory trace. Ultimately, the level at which a stimulus is analyzed depends on factors such as the meaningfulness of the stimulus to the subject, the amount of attention devoted to its analysis, and the subject's purpose and intentions. In line with the levels of processing (LOP) approach, several neuroimaging studies have shown an association between semantic processing and memory performance (Otten et al., 2002; 2001; Henson et al., 2002; Kapur et al., 1994). In one of these studies Henson et al. (2002) found that prefrontal and medial temporal regions show greater functional MRI activation for semantically encoded words relative to alphabetically encoded words. Furthermore, it has been shown that deep semantic processing at encoding selectively activates areas in left prefrontal cortex (Otten et al 2001; Otten and Rugg, 2001). These findings suggest that both the content and the type of processing engaged while the information is encountered influence subsequent recognition (Paller and Wagner, 2002). Useful information on the neural underpinnings and temporal dynamics of memory processes can be provided by event-related potentials (ERPs), an electrophysiological technique that allows a precise description of the time course of neural events and therefore represents a suitable method for investigating recognition memory processes. In general, ERP correlates of memory retrieval can be observed as differences between responses to repeated and new stimuli (Friedman

240

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

and Johnson, 2000). Many studies have been focused on trying to disentangle the different processes related to memory processes (Rugg and Curran, 2007; Paller et al., 2007) by teasing apart those involved in familiarity from those involved in recollection. Specifically, it was found a mid-frontal effect (300–500 ms) linked to familiarity, known as FN400, elicited by words, pictures and faces (Curran, 2000; Curran and Cleary, 2003; Curran and Hancock, 2007) and using verbal material a left parietal old/new effect (500–700 ms) related to recollection (Donaldson and Rugg, 1998; Curran, 2000). Importantly, Donaldson and Rugg (1998) found ERP evidence that old/new effects were influenced by the task performed at encoding with subjects performing either a deep (sentence generation) or a shallow (alphabetic judgments) encoding. The parietal old/new effect was elicited exclusively by old items subjected to deep analysis reflecting recognition processes based on recollection, whereas shallowly studied words were recognized largely on the basis of familiarity. Depth of processing was manipulated using words also in the study of Guo et al. (2004) by requiring subjects to detect animal names (deep encoding) or boldface (shallow encoding) in a series of Chinese words. Recognition was more accurate with deep than shallow encoding and ERP responses starting from 200 ms until 800 ms differed as a function of encoding level. Most studies that investigated the role of LOP used verbal material while relatively little is known about the effects of encoding level on face recognition. Faces were used in a positron emission tomography (PET) study in which the effect of encoding strategy on recognition was investigated using a deep task (judging pleasantness), a shallow task (judging left/right orientation) and an intentional learning task (Bernstein et al., 2002). The results showed an improved performance for deep versus shallow encoding and this effect was associated with increased activation in left inferior temporal and frontal areas. In general keeping with the electrophysiological results, behavioural studies showed that memory for faces is more efficient when the study task involves judgements about abstract rather than physical features (Coin and Tiberghien, 1997). Furthermore, it has been shown that the accuracy with which previously unfamiliar faces are recognised is increased by the presentation of a stereotypecongruent occupation label (Hills et al., 2008). Thus, various encoding strategies that supposedly promote deeper processing of human faces (e.g., character judgments) lead to better recognition than shallow processing tasks (e.g. judging the width of the nose). The present study was designed to examine whether the manner in which unfamiliar faces are encoded influences the neural responses involved in recognition. Although the neural correlates of face perception and memory have been investigated quite extensively no study has yet assessed the ERPs' response in relation to different encoding strategies. We contrasted a shallow versus a deep encoding strategy. In the former, participants were asked to make an orientation judgment, i.e. whether a face was upright or inverted, while in the latter they were asked to report whether a face belonged to an actor/actress or to a politician. After each encoding phase participants were tested in an old/new decision task. The orientation task was chosen because it does not undergo a holistic, configural analysis of the face. In contrast, the actor/politician task requires an abstract judgment and faces are to be processed holistically and taking into account configural information. It has been shown that making judgements about the occupation of a person that can be stereotypically congruent or incongruent with a face (Light et al., 1979) requires the processing of a great number of features and yields greater recognition memory. This kind of categorization depends on comparing a test item with stored exemplars, such as the stereotype of an actor or a politician, and classifying the face as belonging to the most likely category. Perceptual categorization of orientation and occupation are thus likely to rely on different perceptual and cognitive operations operating at different levels during face processing. An important feature of our experimental design is that we included the face-inversion effect in both the two different encoding tasks. Upside-down inversion disrupts the processing of spatial

relations among the features of a face, while largely preserving local feature analysis. Evidence that inversion dramatically disrupts the ability to extract configural information supports the view that inversion makes some of the processes activated by upright faces largely ineffective. Processing configural information, that is, the spacing between the parts, is thought to be critical for structural encoding and is disrupted by face inversion (Rossion and Gauthier, 2002; Leder and Bruce, 2000; Maurer et al., 2002; Rossion, 2009). The present design enabled us to shed light on the differences in recognition memory as a function of shallowly or deeply encoded faces, and to use face inversion as a tool to understand how configural (or holistic) processing is implicated in the deep compared to the shallow encoding of stimuli. These questions were tackled by considering the timing, amplitude, and scalp distribution of various ERP components elicited during the retrieval phase. The earliest components on which we focused our interest were: the N170 (Bentin et al., 1996; Jeffreys, 1996; Rossion and Jacques, 2008), a negative going potential at occipito-temporal sites that is thought to reflect structural encoding and the VPP (Vertex Positive Potential), which is considered the positive counterpart of the N170 (Botzel and Grusser, 1989; Jeffreys, 1989, 1996; Joyce and Rossion, 2005). The N170 effect reflects the earliest stage at which individual faces are discriminated (Jacques and Rossion, 2007). In addition, we considered the N250 component that has been associated to the recognition of facial identity linked to face memory (Schweinberger et al., 2002). The later components we have considered are those related to familiarity, i.e. the FN400 (Curran, 2000; Curran and Hancock, 2007; Paller et al., 2007) and recollection related processes, i.e. Parietal old/ new effect (for a review see Rugg and Curran, 2007). 2. Material and methods 2.1. Participants Fourteen naïve volunteers participated in the experiment. Two were excluded because of a large number of eye blinks during electroencephalogram (EEG) recording. The remaining participants (six males and six females, mean age 27.4) were all right-handed according to the Italian version of the Edinburgh Handedness Inventory (Oldfield, 1971), had normal or corrected to normal vision and no neurological or psychiatric history. All participants gave informed written consent, and the study was approved by the departmental ethics committee. 2.2. Stimuli The stimuli were 600 black-and-white photographs of unfamiliar Caucasian faces. The stimuli were digitally scanned, processed by graphics software, and displayed against a light gray background. The photographs were digitally edited using Adobe Photoshop. All faces were in frontal view, subtended a visual angle of 4.19 × 5.08 degrees and were presented in central vision. The faces had direct eye gaze, neutral expressions and no odd feature. Faces were without paraphernalia, glasses, earrings, make-up that could help recognition; background and clothes were removed and only the hairline remained visible. Gender and age were equally distributed among conditions. All images used represented faces of adult persons aging approximately from 25 to 35 years. At the end of the experiment we asked the participants if they had previously seen some of the faces and none of them reported any familiarity with the faces. All faces were equated for mean luminance and contrast and were selected from a database of photographs of unfamiliar actors and politicians. 2.3. Experimental procedure Prior to the experiment task instructions were given and a short practice session was carried out. Participants were then fitted with an

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

electrode cap and seated in front of a computer monitor at a distance of 100 cm in a dimly illuminated and sound attenuated room. The experimental design consisted of 12 trial blocks each with an encoding phase followed by a retrieval phase. In the encoding phase there were two task conditions, shallow (6 blocks–25 faces for each block) and deep (6 blocks–25 faces for each block). On each trial participants were shown an image of a face that could be upright or inverted and were asked to perform an orientation judgment task in the shallow condition and an actor/politician categorization in the deep condition. Each trial consisted of the following events: a fixation cross appeared for 1200 ms followed by the target stimulus (800 ms duration) and a interstimulus interval (ISI) randomized within a temporal window of 1500–2000 ms. Participants had 1000 ms time for response before the fixation cross was presented again. In the retrieval phase 300 old faces intermixed with 300 new faces were presented. They all appeared in upright orientation and participants had to press one of two buttons according to whether they thought the face had appeared in the study phase or not (i.e., old or new). Participants had to respond both in the encoding and retrieval phase by pressing one of two buttons on a response box with the index or middle finger of the right hand. The assignment of response button to finger was counterbalanced across subjects. Each retrieval phase trial included fixation (1200 ms duration), face exposure (500 ms duration), ISI (randomized interval 1300–1800 ms), and then response. We used a blocked design, the rationale being that attention can be focused better in the blocked than in the mixed condition because an a priori-determined task knowledge reduces trial-by-trial uncertainty (Bentin et al., 1998). After each block there was a short resting period, see Fig. 1 for the sequence of events in a trial. Trials within a block were intermingled in a randomized order. Blocks were administered in an order counterbalanced across participants. All faces used were balanced between conditions and across participants. 2.4. Electrophysiological recordings The EEG was recorded from 32 electrodes mounted on an elastic cap according to the standard system 10–20; the scalp locations were: F7, F3, FZ, F4, F8, FT7, FC3, FCZ, FC4, FT8, T3, C3, CZ, C4, T4, TP7, CP3, CPZ, CP4, TP8, P3, PZ, P4, T5, T6, O1, OZ, and O2. A linked-mastoid served as reference. The electrooculogram (EOG) was recorded with two electrodes placed, in a bipolar montage, at about 1 cm from the external canthus of both eyes. Vertical eye movements and eye blinks were detected by an electrode positioned below the right eye and referenced to the right mastoid. The EEG signals were amplified, filtered (0.01–100 Hz), and digitized at 1000 Hz. The impedance of all electrodes was kept below 5 k. Trials with eye blinks, eye movements (frontal electrodes exceeding 60 V in the 800 ms interval following stimulus onset) and muscular or other artifacts (defined as a voltage

241

deviation on any recording electrode exceeding ±60 μV in the 800 ms following stimulus onset) were excluded. The same was true for trials in which horizontal or non-blink vertical movements occurred, as well as for trials containing EEG drifts or A/D saturation. After removal of EEG and EOG artifacts, epochs beginning 200 ms prior to stimulus onset and continuing for 1000 ms were made. On each trial waveforms were baseline corrected using the signal recorded during a time interval of 200 ms that preceded the onset of the stimulus. The EEG was averaged separately for all conditions of the retrieval phase (old encoded deeply upright, old encoded deeply inverted, new from the deep blocks, old encoded shallowly upright, old encoded shallowly inverted, new from the shallow blocks). The averaged waveforms computed for the different conditions were low pass filtered at 30 Hz. They were based on a minimum of 50 artefact-free trials. 2.5. Data analysis Accuracy and reaction time (RT) of the encoding phase were analyzed with a repeated measure ANOVA factoring: Encoding (Shallow, Deep) and Orientation (Upright, Inverted). For the retrieval phase recognition accuracy was assessed with the discrimination measure indexed by Pr, calculated by computing the proportion of hits minus the proportion of false alarms, p(Hit) − p (False Alarm). Moreover, response bias (Br) was assessed by calculating Br = [p(false alarm)/1 − Pr], (Snodgrass and Corwin, 1988). Pr provides an unbiased estimate of the accuracy in the response to old and new items, with higher values corresponding to more accurate recognition memory. Br indicates the overall tendency to respond “old” or “new” regardless of accuracy; a Br b 0.5 represents a novelty bias (a propensity to say “new”). Pr and Br values were submitted to a repeated measures ANOVA with Encoding (Shallow, Deep) and Orientation (Upright, Inverted) as factors. Correct RTs for old responses were submitted to a repeatedmeasures 2 × 2 ANOVA, with Encoding (Shallow, Deep), and Orientation (Upright, Inverted) as within-participant factors. Moreover, T-test was employed to contrast responses to old versus new faces for each condition. The use of a blocked design enabled us to compare the responses to old faces encoded deeply with those to new faces presented at retrieval in the “deep encoding” blocks, and the responses to old faces encoded shallowly with those to new faces presented at retrieval in the “shallow encoding” blocks. The statistical reliability of the ERPs effects was evaluated by measuring mean amplitudes in the 80–130, 150–200, 200–350, 300– 500 and 500–700 ms time intervals. These intervals were chosen on the basis of the differences visible in the grand average waveforms and the continuous and long-lasting nature of memory effects

Fig. 1. Experimental procedure. The experiment consisted of an encoding phase (depicted on the left) followed by a retrieval phase (depicted on the right). During the study phase a shallow (Task1) or a deep task (Task 2) was employed. During the test phase an Old/New discrimination task was performed.

242

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

Table 1 Recognition memory performance during retrieval. Proportions of hits, correct rejections, false alarms, Pr and Br are reported for each condition. Moreover, means of RT are shown. Behavioral results

Hits Correct rejections False alarms Pr Br Response time (ms)

Deep encoding

Shallow encoding

Old upright

Old inverted

0.89 (0.0)

0.59 (0.1)

New

Old upright

Old inverted

0.69 (0.1)

0.44 (0.1)

0.73 (0.2) 0.27 (0.0) 0.62 (0.0) 0.72 (0.0) 752 (47)

0.33 (0.0) 0.39 (0.1 743 (66)

identified in previous encoding studies. The window were centered on the peak latency. The peak latency analyses were focused only on the early ERP components such as P1, N170 and VPP. Peak measures were taken within a 30 ms window centred on the grand-average means. P1 was analysed on occipito-temporal electrodes (O1, Oz, O2, T5, and T6), the N170 on temporal electrodes (T5 and T6) and the VPP on FZ, FCZ and CZ. Mean amplitude of the N250 component was analysed from 200 to 350 ms on temporal electrodes. Later ERP components were analyzed on frontal-frontocentral (F3, Fz, F4, FC3, FCZ, and FC4) and centroparietal-parietal (CP3, CPZ, CP4, P3, PZ, and P4). Peak latencies and mean amplitude values of each component were submitted to separate repeated-measures ANOVAs. For the ERP data correct responses, i.e. hits and correct rejections, were analysed and subjected to two sets of analyses. The first was an ANOVA conducted only on old correct responses (hits), carried out for each time window with: Encoding (Shallow, Deep), Orientation (Upright, Inverted), Hemisphere (Left, Midline, and Right) and Electrode (number and sites varied depending on the components analyzed) as factors. The second set of ANOVAs were performed to study the old-new effects by comparing each old condition (deep up, deep inverted; shallow up, and shallow inverted) with the respective correct response to new faces (correct rejections) from the deep or shallow blocks; the factors considered were: Memory (Old, New), Side (Left, Midline, and Right) and Electrode (number and sites varied depending on the components analyzed). Only significant results are reported. All effects with two or more degrees of freedom were adjusted for violations of sphericity according to the Greenhouse–Geisser correction.

727 (70)

New 0.72 (0.1) 0.28 (0.0)

0.41 (0.0) 0.47 (0.1) 747 (38)

0.16 (0.1) 0.33 (0.1) 831 (59)

795 (60)

faster response for inverted compared to upright faces in the shallow task, F(1, 11) = 15.2, p b .004. Overall, these results suggest that the deep processing of faces increases RT and lowers performance probably reflecting an increased task demand.

3.1. Behavioral performance

3.1.2. Retrieval phase Recognition accuracy data (Pr) showed a main effect of Encoding, F(1, 11) = 48.6, p b .0001, with enhanced recognition accuracy for deeply versus shallowly encoded faces. Moreover, there was a significant main effect of Orientation, F(1, 11) = 117, 9, p b .0001, with recognition greater for faces that during encoding were presented upright. Analysis of Br showed as significant the main effects of Orientation, F(1, 11) = 101.1, p b .001, and Encoding, F(1, 11) = 8.5, p b .02; moreover the significant interaction Encoding x Orientation, F(1, 11) = 15.5, p b .003, revealed a higher tendency, regardless of accuracy, to classify faces encoded upright deeply as “old” with respect to shallowly upright encoded faces, F(1, 11) = 15.7, p b .003. Furthermore, higher tendency to respond “old” was found for upright compared to inverted faces in both the deep and shallow task, [F(1, 11) = 64.3, p b .001; F(1, 11) = 32.7, p b .001]. An overall tendency to respond “new” was found for faces that during the encoding were presented inverted (a Br b 0.5 represents a novelty bias). Table 1 shows performance for each condition. Analysis of RTs showed a significant interaction Encoding × Orientation, F(1, 11) = 9.4, p b .02, indicating that responses to deeply processed faces were reliably longer for faces encoded inverted with respect to old faces encoded upright F(1, 11) = 9.7, p b .02. No significant differences emerged by comparing old versus new responses. These results show that participants were more accurate in recognizing faces that had been previously encoded at a deep level

3.1.1. Encoding phase Orientation judgements in the shallow task were performed with an accuracy of 95% (SD = 4) and a mean RT of 598 ms (SD = 39) for upright faces and an accuracy of 98% (SD = 2) and a mean RT of 562 ms (SD = 32) for inverted faces. Actor/politician judgements in the deep task were made with an accuracy of 77% (SD = 8) and a mean RT of 837 ms (SD = 55) for upright faces and an accuracy of 66% (SD = 9) and a mean RT of 849 ms (SD = 56) for inverted faces. The ANOVA on accuracy revealed a significant effect of Encoding, F(1, 11) = 87.1, p b .0001, and of Orientation, F(1, 11) = 15.6, p b .003. The significant interaction Encoding× Orientation, F(1, 11) = 24.8, p b .001, revealed a higher accuracy for the shallow task with respect to the deep task for both upright and inverted faces, F(1, 11) = 38.5, p b .001; F(1, 11) = 108.7, p b .0001; moreover the results showed that for the deep encoding task accuracy was reduced for inverted faces, F(1, 11) = 29.9, p b .001. The ANOVA conducted on RT showed again as significant the factor Encoding, F(1, 11) = 22.9, p b .002. The significant interaction Encoding × Orientation, F(1, 11) = 13.4, p b .005, yielded faster RT for faces processed shallowly than deeply, in upright and inverted orientation, F(1, 11) = 19.2, p b .001; F(1, 11) = 26.7, p b .001; and

Fig. 2. Behavioral results. Top: accuracy in Pr values (proportion of hits minus proportion of false alarms). Bottom: RTs in ms for all conditions.

3. Results

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

243

in a task that required more effort or additional processing with respect to the orientation judgement. Recognition performance was also affected by inversion with RTs slower for faces encoded deeply and with inverted orientation. Results of the behavioural data are shown in Fig. 2. Mean RTs are reported in Table 1.

The following section is structured by reporting for each ERP component first the results of the ANOVAs carried out on old responses (Encoding and Orientation effects) only and then the results of the ANOVAs in which responses to old and new faces were compared (Old/New effects).

3.2. ERPs

3.3. P1

The ERP results reported below concern the responses to the oldnew task during the retrieval phase. Grand average waveforms recorded during the retrieval phase are shown for selected electrodes in Fig. 3 which shows responses to faces processed with different encoding levels and orientation and in Fig. 4 where only old faces with shallow or deep encoding and different orientation are directly compared.

3.3.1. Encoding and orientation effects 3.3.1.1. Latency. The analysis of the P1 peak latency showed only a significant main effect of Orientation, F (1, 11) = 11.8, p b .008, with longer latencies for faces encoded inverted with respect to faces encoded upright.

Fig. 3. Grand averages waveforms for selected electrodes. On the left part ERP responses to old (encoded previously upright or inverted) and new faces during retrieval after the deep encoding task; on the right ERP responses to old (encoded previously upright or inverted) and new faces during retrieval after the shallow encoding task.

244

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

Fig. 4. Grand averages of ERP responses to correctly recognized old faces. On the temporal electrodes (T5 and T6) the N170 can be seen, larger for shallowly encoded faces and longer for faces encoded inverted. Similar effects can be found on the VPP on electrodes CPZ and FCZ.

3.3.1.2. Amplitude. No significant results emerged. 3.3.2. Old/new effects 3.3.2.1. Latency. Old-new effects were found for both shallow and deep encoding with old upright faces yielding shorter latencies with respect to new faces, F (1, 11) = 8.3, p b .02; F (1, 11) = 9.3, p b .02]. No significant effects were found for inverted faces. 3.3.2.2. Amplitude. No significant results emerged.

3.4.1.2. Amplitude. The ANOVA on mean amplitude values showed a main effect of Encoding, F(1, 11) = 11.1, p b 0.008, with faces deeply encoded yielding enhanced amplitude compared to faces shallowly encoded. 3.4.2. Old/new effects 3.4.2.1. Latency. The old-new comparisons showed that old faces encoded with a deep task and upright orientation yielded shorter latencies with respect to new faces, F(1, 11) = 29.1, p b .001. No other significant differences were found. 3.4.2.2. Amplitude. No significant differences.

3.4. VPP 3.5. N170 3.4.1. Encoding and orientation effects 3.5.1. Encoding and orientation effects 3.4.1.1. Latency. The ANOVA on the peak latency of the VPP component showed a main effect of Orientation, F(1, 11) = 21.4, p b .002, with longer latencies for old faces encoded inverted compared to old face encoded upright.

3.5.1.1. Latency. The analysis on N170 latency revealed a main effect of Orientation, F(1, 11) = 40.9, p b .001, with earlier peak latencies in response to old upright than to old inverted faces.

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

3.5.1.2. Amplitude. The analysis on the N170 amplitude revealed a main effect of Encoding, F(1, 11) = 7.5, p b .02, reflecting an enhanced negativity for shallowly encoded faces.

245

significant, [F(1, 11) = 23.8, p b .001; F(1, 11) = 19.2, p b .002], reflecting an enhanced negativity for old faces encoded shallowly with respect to old faces encoded deeply and for old upright orientation in comparison with old faces encoded with inverted orientation.

3.5.2. Old/new effects 3.5.2.1. Latency. Old-new comparisons showed that faces encoded upright and inverted in the shallow encoding task elicited shorter latencies with respect to new faces [F(1, 11) = 40.9, p b .001; F(1.6, 17.7) = 7.4, p b .03]; moreover faces encoded upright in the deep encoding task showed earlier latencies with respect to new faces presented in the deep encoding blocks, F(1, 11) = 24.2, p b .001. 3.5.2.2. Amplitude. No significant effects were found by analysing mean amplitudes. 3.6. Early positivity (150–200) 3.6.1. Encoding and orientation effects Similarly but with polarity reversal data with respect to the N170, the ANOVA on mean amplitude on the frontal and fronto-central electrodes revealed a significant main effect of Encoding, F(1, 11) = 12.2, p b .006, with more positive responses for deeply than shallowly encoded faces. By the same token, also the mean amplitude values of the centroparietal and parietal electrodes yielded a greater positivity for deeply than shallowly encoded faces, F(1, 11) = 16.7, p b .003. No significant old–new effects emerged. 3.7. Time window 200–350 ms 3.7.1. Encoding and orientation effects On temporal sites the analysis on mean amplitude of the N250 component showed a significant effect Encoding, F(1, 11) = 13.1, p b .005, with reduced amplitude for deep with respect to shallow encoded faces. In addition, from 200 to 300 ms the results on frontal and frontocentral sites showed that both Encoding and Orientation were

3.7.2. Old/new effects Old–new comparisons did not reach significance on temporal sites: old up deep versus new deep, p b .07; old inverted deep versus new deep p b .06. Moreover, the old–new comparisons on frontal sites showed that old faces shallowly encoded with upright orientation elicited enhanced negativity with respect to new faces, [F(1, 11) = 11.9, p b .006; F(1, 11) = 11.4, p b .007]. 3.8. Middle latency ERPs component 3.8.1. Encoding and orientation effects From approximately 300 to 500 ms following stimulus onset the main effect of Encoding emerged on frontal and fronto-central sites, F(1, 11) = 16.8, p b .003, showing that faces encoded deeply elicited ERPs that were more positive-going than those to shallowly encoded stimuli. On centro-parietal and parietal electrodes again a main effect of Encoding was found, F(1, 11) = 14.5, p b .004, showing enhanced positivity for faces retrieved after the deep encoding task in comparison with the shallow task. 3.8.2. Old/new effects Moreover, considering frontal sites faces retrieved after deep encoding, both upright and inverted, yielded more positive responses with respect to new faces, [F(1, 11) = 10.4, p b .009; F(1, 11) = 6.8, p b .03]. No differences were found as far as new faces are concerned. Moreover, enhanced amplitudes were found on parietal sites by comparing deeply upright and inverted encoded with new faces, [F(1, 11) = 10.1, p b .01; F(1, 11) = 6.5, p = .03], see Fig. 3, 4. Mean amplitudes values are reported in Fig. 5.

Fig. 5. Mean amplitudes recorded on frontal and parietal sites. The left section shows the mean values for the latency window 300–500 and the right section shows the latency window from 500 to 700 ms.

246

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

3.9. Late latency ERPs components 3.9.1. Encoding and orientation effects From 500 to 700 ms on frontal and fronto-central electrodes the effect of Encoding remained significant, F(1, 11) = 9.9, p b .01, showing an amplitude enhancement for deeply with respect to shallowly studied faces both for upright and inverted orientation. On centro-parietal and parietal electrodes the significant interaction Encoding × Inversion, F(1.5, 16.6) = 5.4, p b .03, showed for upright orientation at encoding that deeply learned faces elicited enhanced amplitudes with respect to shallow learned faces, F(1, 11) = 18.7, p b .002. The same pattern of results was found considering the retrieval of faces that were presented inverted during encoding, F(1, 11) = 6.7, p b .04. Furthermore, in the shallow condition a significant interaction Inversion × Side, F(1.3, 14.5) = 8.9, p b .007, with an enhanced amplitude for upright encoded with respect to inverted encoded in the right hemisphere, F(1, 11) = 24.1, p b .001; (this effect is visible on electrode P4, see Figs. 3 and 4. 3.9.2. Old/new effects Old–new comparisons did not reach significance on frontal and fronto-central electrodes. However, on centro-parietal electrodes significant old/new effect revealed that old upright deep encoded faces elicited more positive-going ERPs than new faces, F(1, 11) = 24.2, p b .001, and more surprisingly also faces presented inverted during encoding and deeply processed showed greater amplitude than new faces, F(1, 11) = 17.6, p b .003; See Figs. 3–5. It is worth noting that in all the time windows analysed no significant difference was found by contrasting new faces presented in the blocks with shallow encoding in comparison with new faces presented during the deep encoding blocks. 3.9.3. Summary of the ERPs results Overall, the ERP responses showed a consistent effect of type of encoding on recognition memory beginning quite early, at about 150 ms on frontal and central sites with an amplitude enhancement for deeply compared to shallowly encoded faces. On temporal sites encoding affected early perceptual processing at the level of the P1 and face structural encoding processes indexed by the N170 component. Remarkably, the old–new effect, from 300 to 700 ms showed enhanced positivity as a function of the level of processing engaged during encoding, with an early fronto-central enhancement for faces encoded deeply in both orientations, and a late parietal amplitude enhancement for old (upright and inverted) deeply encoded faces compared to new faces. An effect for shallowly encoded faces was found earlier between 200 and 300 ms with faces encoded shallowly and upright yielding enhanced amplitude with respect to deeply encoded faces. In sum, throughout the entire course of face processing we found a robust effect of encoding, continuously interacting with memory processes. 4. Discussion The present study has been inspired by the well known effects of “depth” of study processing and its effect on subsequent retrieval (Craik and Lockhart, 1972). Our specific aim was to determine how the processing level at encoding contributes to face recognition memory. To do that we contrasted the electrophysiological responses associated with the recognition of faces previously studied either under a deep or a shallow encoding mode and we included a further manipulation by presenting faces either upright or upside-down at encoding. We analysed the ERP components related to structural encoding, i.e. the N170 and those components related to later memory old/new recognition processes.

During the encoding phase participants were more accurate and faster in the shallow compared to the deep encoding task, suggesting a greater cognitive demand for the occupation with respect to the orientation task. The importance of global face processing for the deep task was confirmed by the fact that performance was reduced for inverted faces in which the holistic processing is disrupted (Rossion, 2009). During retrieval, participants recognized previously seen faces more accurately after deep encoding and upright orientation with respect to inverted and shallowly encoded faces. It is of interest that with the deep encoding task RTs for inverted faces were significantly longer than those for upright faces, reflecting greater retrieval processing demands. Furthermore, we found a general tendency to classify faces encoded deeply as old, regardless of accuracy, together with a novelty bias for faces encoded shallowly and presented inverted. Confirming several lines of evidence (Bernstein et al., 2002; Otten et al., 2001; Rugg et al., 2000, 1998a,b), these behavioural data clearly show that the manner in which faces are processed during encoding influences subsequent recognition. The ERP data yielded several major findings. Orientation during the encoding phase influenced subsequent retrieval processes beginning very early, at about 120 ms. The N170 amplitude, during retrieval, was affected by depth of encoding and its latency differed according to whether faces were old or new and upright or upsidedown at encoding. The second important aspect of the ERP findings concerns the memory effect that occurred from 200 to 500 ms with an enhanced negativity for upright and shallowly encoded faces, followed by a sustained and long-lasting greater positivity for faces encoded deeply and presented in either orientation. The last noteworthy effect was the sensitivity of the late parietal effect (N500 ms) to depth of processing. The amplitude of this electrophysiological effect varied in a graded manner with the largest positivity for faces encoded upright and deeply, followed by faces encoded inverted and deeply and finally faces encoded upright and shallowly. One intriguing finding was that at about 120 ms, at the processing level indexed by the P1, a modulation of latency was found with shorter-latency responses for faces that during encoding were presented in upright orientation with respect to faces presented inverted. This result is quite surprising in the light of the assumption that P1 is associated with a low level features analysis although there is evidence that P1 might be influenced also by top-down processes (Schendan et al., 1998; Taylor, 2002; Heisz et al., 2006; Halit et al. 2000) or by task demands (Rossion et al., 1999; Stahl et al., 2010). In this regard, it is important to note that by virtue of the counterbalanced design all faces were similar with respect to physical stimulus characteristics. This early effect is likely to reflect one of the earliest points when perceptual detection and categorization of faces begins (Liu et al., 2002). If the processes indexed by the P1 rely more on global information, reflecting a holistic processing stage (Itier and Taylor, 2002), it is reasonable to assume that a longer processing is required for faces that have been previously encoded inverted and later retrieved upright for which a feature-based processing was carried out during encoding. This early modulation might also be related to an attentional modulatory effect on occipital visual areas as previously found by Rossion et al. (1999). The most important findings concern the N170 component. ERPs studies have demonstrated that face stimuli elicit a greater negative brain potential in right posterior recording sites 170 ms after stimulus onset (N170) relative to non-face stimuli, and its positive counterpart is named VPP (Joyce and Rossion, 2005). We found that the amplitude of N170 in response to deeply encoded faces was smaller with respect to faces shallowly encoded. This is in keeping with evidence that a repeated stimulus presentation produces a decreased response in brain regions specifically associated with processing that stimulus, a phenomenon named “repetition suppression” effect (Grill-Spector et al., 2006). Neural repetition suppression or adaptation effects reflect the reduction in the activity of neural populations in response

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

to stimuli that are preceded by physically or categorically identical adaptor stimuli. Several recent ERP studies using adaptation procedures have found that the N170 or its magnetoencephalographic counterpart M170, is reduced in response to repeated faces compared to new faces (Caharel et al., 2009; Eimer et al., 2010; Jacques and Rossion, 2007, Kloth et al, 2010, Kovacs et al, 2006, and for the M170 Harris and Nakayama 2007). The possibility that the encoding task during the presentation of the first stimulus might produce adaptation effects on the ERP components is just a hypothesis which would require further studies. The enhanced amplitude found for shallowly encoded faces might index processes engaged when attempting to retrieve structural information that has not been thoroughly processed. A better consolidation with deep encoding might facilitate a subsequent encoding of the face hence give rise to priming or repetition effects with a consequent reduction in amplitude. A repetition effect can also explain a shorter latency of the N170 component for faces encoded upright with respect to new faces. A reduced N170 latency and amplitude for repeated faces were also found by Itier and Taylor (2004), which hypothesized a perceptual priming effect. Moreover, the shorter latency for faces presented during the encoding phase upright with respect to faces presented inverted might suggest that it takes longer to access facial representation when the learning process is influenced by inversion (Itier and Taylor, 2004). In this context, it is noteworthy to point out a recent finding by Stahl et al. (2010) who found later peaks of the N170 for other-race compared to own-race faces during a face recognition memory paradigm. Whereas there is evidence of cognitive impenetrability of these early components (Bentin et al., 1996; Bentin and Golland, 2002; Eimer, 2000), several lines of research show reliable effects of familiarity, repetition priming, and recognition on N170 responses to faces (Campanella et al., 2000; Itier and Taylor, 2002, 2004; Guillaume et al., 2009; Jemel et al., 2005, 2003; Caharel et al., 2005; Heisz et al., 2006; Marzi and Viggiano, 2007, 2010), suggesting that as early as 170 ms after stimulus onset, the brain is individuating previously encoded faces. Some evidence of the influence of previous experience on the N170 was found comparing well known and newly learned faces (Herzmann and Sommer, in press); moreover, after semantic learning, differences on the N170 were found by Heisz and Shedden (2009) indicating that semantic learning influences perceptual processing. Taken together all these findings, one is justified to suggest that faces encoded deeply promote subsequent facilitation leading to a more efficient structural encoding. A similar pattern of response albeit with polarity reversal was found in the same latency window of the N170 on frontal and central sites, probably reflecting the VPP component (Joyce and Rossion, 2005). Later on, during the time window from 200 to 300 ms, an intriguing negativity was found for faces encoded upright and shallow in comparison with deeply encoded and new faces. This effect might reflect implicit recognition and related neural processing due to perceptual learning that occurred despite minimal conscious memory. This effect was not found for inverted shallowly encoded faces, for which face inversion disrupted encoding processing and the formation of a global face representation. This could represent a processing stage in which it is possible to discriminate old from new stimuli without access to consciousness. One should point out, however, that our experimental procedure did not enable us to disentangle electrophysiological manifestations of priming and conscious memory. Therefore, further investigations are needed to further clarify this effect. One ERP study found that responses associated with memory for briefly versus long studied faces differed qualitatively (Paller et al., 2003). The former produced a distinctive brain potential, i.e. a frontal negativity beginning at about 270 ms, while the latter faces activated different response patterns producing positive potentials with a

247

maximum at 600–700 ms. In keeping with that, the present results might suggest that the recognition of faces previously processed during a shallow task relies more on implicit recognition while faces studied deep might involve later explicit memory processes. Several studies, using words as stimuli, showed that depth of processing affects especially recollection mechanisms (Rugg et al., 1998a,b; Paller and Kutas, 1992) although these findings are somewhat controversial showing also effects on priming (Bentin et al., 1998; RichardsonKlavehn and Gardiner; 1998; Ramponi et al. 2007). Moreover, Schott et al. (2002) suggested that perceptual priming and explicit memory have distinct neural correlates during encoding as a function of LOP. At about 300 ms level of encoding influenced the ERP responses also on temporal sites. In this time range several studies have shown that repeatedly presented faces elicit a negative component (N250) that indicates the access to face representations and the activation of stored perceptual face representations (Tanaka et al., 2006, Schweinberger et al., 2004, 2002). The N250 has been associated to the recognition of facial identity linked to face memory and familiarity or priming can increase or decrease its amplitude (Schweinberger et al., 2002, 2004). These observations might indicate that the discrimination of individual faces builds up until a matching with memory representations takes place. Moreover, Scott et al. (2006) provided evidence that the N250 might reflect expertise in the processing of stimuli at the subordinate level. On the basis of these findings it is possible to hypothesize that the level of processing during encoding has an influence on the N250 that reflects the processing stage responsible for categorizing a face as familiar. Such a process could nonetheless benefit from a deep encoding level. The reduced amplitude found for old faces encoded deeply might reflect familiarity or priming effects. The different amplitude modulations that we found in comparison to other studies (Scott et al., 2006; Kaufmann et al., 2009; Joyce and Kutas, 2005) might be due to the use of the mastoids as reference site with respect to an average reference. From 300 to 500 the ERP pattern was characterized by an enhanced positivity for deeply encoded faces, while no effects were found for the shallow condition. Previous ERP studies of recognition memory found differences between studied (old) and unstudied (new) stimuli, referred to as ERP old/new effect (see Friedman and Johnson, 2000 and Rugg and Curran, 2007). Moreover, several previous studies have demonstrated that correct recognition of deeply encoded items produces greater parietal positivity than shallowly encoded items (Rugg et al., 1998a,b, 2000; Allan et al., 2000; Paller et al., 2003) The present study extend those results by showing that recognition of faces that were previously deeply encoded produced responses of increased amplitude in a widespread network of frontal, temporal and parietal regions beginning at early latencies. Responses to faces encoded deeply showed a pattern of positive modulations probably engaging functionally distinct processes. Between approximately 300 and 500 ms after stimulus onset, memory effects were found for deeply encoded faces, regardless from the orientation in which the faces were presented. From 500 ms onwards, responses to deeply upright studied faces differed from those to deeply inverted and shallowly studied items on parietal sites. Together, these findings indicate that the depth of processing at encoding influences memory-related neural activity associated with familiarity and recollection of previously seen faces. It has been shown that during retrieval, recollection is indexed by an ERP positivity that is maximal starting from 500 ms over parietal sites, whereas familiarity is related to an earlier positivity that has a fronto-central distribution, indexed by the FN400 (Rugg and Curran, 2007). In line with these findings we interpret the greater frontal positivity found for deeply encoded faces as indexing recognition memory processes based on familiarity, i.e. the feeling that an item has recently been experienced. Moreover, the present findings concerning the parietal old/new differences following deep as compared to perceptual encoding are

248

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

consistent with recollection processes (Rugg et al., 2000; Yonelinas, 2001), assuming that recollection might be reflected by the comparison of deep hits with respect to shallow hits, as found by Henson et al. (2005). Different findings were reported by MacKenzie and Donaldson (2007) showing that face familiarity can be associated with a posterior old/new effect. Furthermore, MacKenzie and Donaldson (2009) found that during 500–700 ms faces were associated with an anterior ERP old-new effect whereas names were associated with the classic left parietal effect linked with recollection, providing evidence that remembering can be associated with different underlying cognitive operations for faces and names. Recently, an ERP study by Nyhus and Curran (2009) provided evidence that semantic encoding task and perceptual matching processing for distinctive items can influence both familiarity and recollection processes. By the same token, semantic compared with perceptual processing led to a large increase in recollection and a moderate increase in familiarity (Yonelinas, 2001). Manipulating levels of processing (LOP) across blocks Rugg et al. (2000) found differences between old and new words diverging between shallow and deep encoding conditions between 300–500 ms and 500–800 ms. In particular, Rugg et al. (1998a) found a left parietal effect for deeply but not for shallow studied items, consistent with the view that this effect might be related to recollection. The study by Rugg et al. (2000) warrants additional mention, because they used, as in the present experiment, a blocked design. Usually, randomized rather than blocked designs are used (Rugg et al., 2000) in which the recognition of deeply and shallowly studied words were assessed with respect to a single new-word baseline. By separating encoding tasks into separate blocks they found that the earlier familiarity and the later parietal recollection effects were affected by LOP (Rugg et al., 2000). It is possible that blocking encoding conditions enhances LOP effects whereas mixing reduces them (Nyhus and Curran, 2009). It is worth noticing that the encoding manipulation and the blocked design used in the present study did not give rise to differences, during the whole time course analysed, between the ERPs elicited by correctly classified new faces. Furthermore, beyond 500 ms although faces studied under the deepencoding condition evoked greater parietal activation than those studied under the shallow-encoding condition, also the latter condition was associated with retrieval success, indexed by a greater positivity for upright compared to inverted faces belonging to the shallow blocks. Neuroimaging studies have contributed to shed light on the processes related to the effects of LOP and retrieval. In an fMRI experiment Iidaka et al. (2006) using pictures of objects, showed that among the fronto-parietal regions involved in retrieval success, the inferior frontal gyrus and intraparietal sulcus were crucial to conscious recollection because the activity of these regions was influenced by the depth of memory at encoding. In addition, retrieval of deeply encoded pictures resulted most notably in a greater hippocampal activation (Mandzia et al., 2004). Some inconsistent results have been found in that no difference in activation patterns associated with different level of processing at encoding was found during recognition in the study by Bernstein et al. (2002), while other studies have reported regions associated with retrieval processes (Buckner and Wheeler, 2001; Rugg et al., 1998a,b). To further characterize the neural processes underlying face memory we studied the effect of orientation during encoding (face inversion effect). Individual faces are notoriously difficult to recognize when they are presented upside-down (Rossion and Gauthier, 2002). Interestingly, whereas inverted deeply encoded faces yielded greater activations with respect to shallowly encoded and new faces during the time window supposed to reflect familiarity, a smaller enhancement was found at longer latencies. Inversion affects face processing dramatically in the shallow condition and interacts with memory processes in the deep condition.

We are particularly sensitive to the configuration of upright faces and inversion removes this sensitivity and the ability to match stimuli successfully which is a prerequisite for accurate memory retrieval. The failure to recognize a face can be due to a lack of compatibility between the images of the face at encoding and at retrieval or to the disruption of configural information (Jacques and Rossion, 2007). There is abundant evidence that upright faces are processed holistically in such a way that face parts are processed interactively rather than independently (Maurer et al., 2002; Rossion, 2009; Yovel, 2009). The holistic interpretation of the face inversion effect proposes that the larger effect of inversion for processing configural rather than featural cues is a consequence of the disruption of the ability to perceive the multiple features of a face as a whole (Rossion, 2008; Rossion, 2009). Inverting faces impairs the integration of features into a gestalt or holistic representation (Tanaka and Farah 1993). When faces are upright the internal features are so strongly integrated that it becomes difficult to parse the face into isolated features while this is not the case following inversion. Because all faces share the same firstorder relations, recognition of individual faces requires the encoding of information about the shape and spacing of the features (Leder and Bruce, 2000). In the light of the above findings we might suggest that face inversion, by disrupting the observers' ability to integrate configural information, and process the face holistically, impaired and delayed recognition memory as indexed by the latency effect found on the N170 and the VPP. Moreover, considering later memory related effects it could be that deep encoding was sufficient to diminish the disruptive effects of inversion, similar results were found in a repetition paradigm by Itier and Taylor (2004). This is in line also with the behavioural results showing a significant drop in accuracy for inverted compared to upright faces. A physical judgement about a face during encoding results in poorer subsequent recognition performance than an abstract judgement. These two kinds of judgements might rely on different perceptual and cognitive processes: the occupation/categorization task requires both perceptual and configural processing of the whole face, whereas judging if a face is presented upright or inverted probably relies more on the processing of first-order configuration (Maurer et al., 2002) reflected in the facial features arrangement (i.e. eyes above the nose). It could be the case that processing the configural information facilitates a correct encoding, storage and subsequent retrieval. All in all, the present ERP results might be interpreted as reflecting distinct kinds of memory- related neural activity which differ as a function of previous encoding demands (Tsivilis et al., 2001; Paller et al., 2003). This is shown by the fact that shallowly studied faces differ qualitatively from deep studied faces and probably engage different processes in different temporal sequence. It might be possible that shallow studied faces activate early view-based representations (the enhanced negativity was found only for upright encoded faces), whereas deep faces activate more abstract representations underlying recognition memory. In conclusion, the present study supports the evidence that, as previously found for pictures and words, face recognition memory both during early and later processing stages is influenced by the strategies adopted during encoding. Effects of deeply studied faces were found throughout the entire time course of recording, whereas old/new effects for shallowly studied faces were confined at early latency, probably reflecting implicit memory. Furthermore, our findings show that structural encoding, indexed by the N170, turns out to be “cognitive penetrable” yielding repetition priming effects for deeply encoded faces. Finally, face inversion, by disrupting configural processing, interacts with memory related processes for deeply studied faces and impairs face recognition after shallow encoding.

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

Acknowledgment We thank Giulia Bagni for precious help during recording.

References Allan, K., Robb, W.G., Rugg, M.D., 2000. The effect of encoding manipulations on neural correlates of episodic retrieval. Neuropsychologia 38, 1188–1205. Bentin, S., Golland, Y., 2002. Meaningful processing of meaningless stimuli: the influence of perceptual experience on early visual processing of faces. Cognition 86, 1–14. Bentin, S., Allison, T., Puce, A., Perez, E., McCarthy, G., 1996. Electrophysiological studies of human face perception. J. Cogn. Neurosci. 8, 551–565. Bentin, S., Moscovitch, M., Nirhod, O., 1998. Levels of processing and selective attention effects on encoding in memory. Acta Psychol. 98, 311–341. Bernstein, L.J., Beig, S., Siegenthaler, A.L., Grady, C.L., 2002. The effect of encoding strategy on the neural correlates of memory for faces. Neuropsychologia 40, 86–98. Botzel, K., Grusser, O.J., 1989. Electric brain potentials-evoked by pictures of faces and non-faces—a search for face-specific EEG-potentials. Exp. Brain Res. 77, 349–360. Buckner, R.L., Wheeler, M.E., 2001. The cognitive neuroscience of remembering. Nat. Rev. Neurosci. 2, 624–634. Caharel, S., Courtay, N., Bernard, C., Lalonde, R., Rebai, M., 2005. Familiarity and emotional expression influence an early stage of face processing: an electrophysiological study. Brain Cogn. 59, 96–100. Caharel, S., d’Arripe, O., Ramon, M., Jacques, C., Rossion, B., 2009. Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: evidence from the N170 ERP component. Neuropsychologia 47, 639–643. Campanella, S., Hanoteau, C., Depy, D., Rossion, B., Bruyer, R., Crommelinck, M., Guerit, J.M., 2000. Right N170 modulation in a face discrimination task: an account for categorical perception of familiar faces. Psychophysiology 37, 796–806. Coin, C., Tiberghien, G., 1997. Encoding activity and face recognition. Memory 5, 545–568. Craik, F.I., 2002. Levels of processing: past, present and future? Memory 10, 305–318. Craik, F., Lockhart, R., 1972. Levels of processing: a framework for memory research. J. Verbal Learn. Verbal Behav. 11, 671–684. Craik, F., Tulving, E., 2004. Depth of processing and the retention of words in episodic memory. Cognitive Psychology: Key Readings, p. 296. Curran, T., 2000. Brain potentials of recollection and familiarity. Mem. Cogn. 28, 923–938. Curran, T., Cleary, A.M., 2003. Using erps to dissociate recollection from familiarity in picture recognition. Brain Res. Cogn. Brain Res. 15, 191–205. Curran, T., Hancock, J., 2007. The FN400 indexes familiarity-based recognition of faces. Neuroimage 36, 464–471. Donaldson, D.I., Rugg, M.D., 1998. Recognition memory for new associations: electrophysiological evidence for the role of recollection. Neuropsychologia 36, 377–395. Eimer, M., 2000. Effects of face inversion on the structural encoding and recognition of faces. Evidence from event-related brain potentials. Brain Res. Cogn. Brain Res. 10, 145–158. Eimer, M., Kiss, M., Nicholas, S., 2010. Response Profile of the Face-Sensitive N170 Component: A Rapid Adaptation Study. Cerebral Cortex 20, 2442–2452. Friedman, D., Johnson, R.J., 2000. Event-related potential (ERP) studies of memory encoding and retrieval: a selective review. Microsc. Res. Tech. 51, 6–28. Grill-Spector, K., Henson, R., Martin, A., 2006. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23. Guillaume, C., Guillery-Girard, B., Chaby, L., Lebreton, K., Hugueville, L., Eustache, F., Fiori, N., 2009. The time course of repetition effects for familiar faces and objects: an ERP study. Brain Res. 1248, 149–161. Guo, C., Zhu, Y., Ding, J., Fan, S., Paller, K.A., 2004. An electrophysiological investigation of memory encoding, depth of processing, and word frequency in humans. Neurosci. Lett. 356, 279–282. Halit, H., de Haan, M., Johnson, M.H., 2000. Modulation of event-related potentials by prototypical and atypical faces. NeuroReport 11, 1871–1875. Harris, A., Nakayama, K., 2007. Rapid face-selective adaptation of an early extrastriate component in MEG. Cereb. Cortex 17, 63–70. Heisz, J.J., Shedden, J.M., 2009. Semantic learning modifies perceptual face processing. J. Cogn. Neurosci. 21, 1127–1134. Heisz, J.J., Watter, S., Shedden, J.M., 2006. Automatic face identity encoding at the n170. Vis. Res. 46, 4604–4614. Henson, R.N., Shallice, T., Gorno-Tempini, M.L., Dolan, R.J., 2002. Face repetition effects in implicit and explicit memory tests as measured by fMRI. Cereb. Cortex 12, 178–186. Henson, R.N., Hornberger, M., Rugg, M.D., 2005. Further dissociating the processes involved in recognition memory: an fMRI study. J. Cogn. Neurosci. 17, 1058–1073. Herzmann, G., Sommer, W., 2010. Effects of previous experience and associated knowledge on retrieval processes of faces: an ERP investigation of newly learned faces. Brain Res. 1356, 54–72. Hills, P.J., Lewis, M.B., Honey, R.C., 2008. Stereotype priming in face recognition: interactions between semantic and visual information in face encoding. Cognition 108, 185–200. Iidaka, T., Matsumoto, A., Nogawa, J., Yamamoto, Y., Sadato, N., 2006. Frontoparietal network involved in successful retrieval from episodic memory. spatial and temporal analyses using fMRI and ERP. Cereb. Cortex 16, 1349–1360. Itier, R.J., Taylor, M.J., 2002. Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs. Neuroimage 15, 353–372.

249

Itier, R.J., Taylor, M.J., 2004. Effects of repetition learning on upright, inverted and contrast-reversed face processing using ERPs. Neuroimage 21, 1518–1532. Jacques, C., Rossion, B., 2007. Early electrophysiological responses to multiple face orientations correlate with individual discrimination performance in humans. Neuroimage 36, 863–876. Jeffreys, D.A., 1989. A face-responsive potential recorded from the human scalp. Exp. Brain Res. 78, 193–202. Jeffreys, D.A., 1996. Evoked potential studies of face and object processing. Vis. Cogn. 3, 1–38. Jemel, B., Pisani, M., Calabria, M., Crommelinck, M., Bruyer, R., 2003. Is the N170 for faces cognitively penetrable? Evidence from repetition priming of mooney faces of familiar and unfamiliar persons. Brain Res. Cogn. Brain Res. 17, 431–446. Jemel, B., Pisani, M., Rousselle, L., Crommelinck, M., Bruyer, R., 2005. Exploring the functional architecture of person recognition system with event-related potentials in a within- and cross-domain self-priming of faces. Neuropsychologia 43, 2024–2040. Joyce, C.A., Kutas, M., 2005. Event-related potential correlates of long-term memory for briefly presented faces. J. Cogn. Neurosci. 17, 757–767. Joyce, C., Rossion, B., 2005. The face-sensitive N170 and vpp components manifest the same brain processes: the effect of reference electrode site. Clin. Neurophysiol. 116, 2613–2631. Kapur, S., Craik, F.I., Tulving, E., Wilson, A.A., Houle, S., Brown, G.M., 1994. Neuroanatomical correlates of encoding in episodic memory: levels of processing effect. Proc. Natl Acad. Sci. USA 91, 2008–2011. Kaufmann, J.M., Schweinberger, S.R., Burton, A.M., 2009. N250 ERP correlates of the acquisition of face representations across different images. J. Cogn. Neurosci. 21, 625–641. Kloth, N., Schweinberger, S.R., Kovacs, G., 2010. Neural correlates of generic versus gender-specific face adaptation. J. Cogn. Neurosci. 22, 2345–2356. Kovacs, G., Zimmer, M., Banko, E., Harza, I., Antal, A., Vidnyanszky, Z., 2006. Electrophysiological correlates of visual adaptation to faces and body parts in humans. Cereb. Cortex 16, 742–753. Leder, H., Bruce, V., 2000. When inverted faces are recognized: the role of configural information in face recognition. Q. J. Exp. Psychol. A 53, 513–536. Light, L.L., Kayra-Stuart, F., Hollander, S., 1979. Recognition memory for typical and unusual faces. J. Exp. Psychol. Hum. Learn. 5, 212–228. Liu, J., Harris, A., Kanwisher, N., 2002. Stages of processing in face perception: an MEG study. Nat. Neurosci. 5, 910–916. MacKenzie, G., Donaldson, D.I., 2007. Dissociating recollection from familiarity: electrophysiological evidence that familiarity for faces is associated with a posterior old/new effect. Neuroimage 36, 454–463. MacKenzie, G., Donaldson, D.I., 2009. Examining the neural basis of episodic memory: ERP evidence that faces are recollected differently from names. Neuropsychologia 47, 2756–2765. Mandzia, J.L., Black, S.E., McAndrews, M.P., Grady, C., Graham, S., 2004. fMRI differences in encoding and retrieval of pictures due to encoding strategy in the elderly. Hum. Brain Mapp. 21, 1–14. Marzi, T., Viggiano, M.P., 2007. Interplay between familiarity and orientation in face processing: an ERP study. Int. J. Psychophysiol. 65, 182–192. Marzi, T., Viggiano, M.P., 2010. When memory meets beauty: insights from eventrelated potentials. Biol. Psychol. 84, 192–205. Maurer, D., Grand, R.L., Mondloch, C.J., 2002. The many faces of configural processing. Trends Cogn. Sci. 6, 255–260. Nyhus, E., Curran, T., 2009. Semantic and perceptual effects on recognition memory: evidence from ERP. Brain Res. 1283, 102–114. Oldfield, R.C., 1971. The assessment and analysis of handedness: the edinburgh inventory. Neuropsychologia 9, 97–113. Otten, L.J., Rugg, M.D., 2001. Task-dependency of the neural correlates of episodic encoding as measured by fMRI. Cereb. Cortex 11, 1150–1160. Otten, L.J., Henson, R.N., Rugg, M.D., 2001. Depth of processing effects on neural correlates of memory encoding: relationship between findings from across- and within-task comparisons. Brain 124, 399–412. Otten, L.J., Henson, R.N.A., Rugg, M.D., 2002. State-related and itemrelated neural correlates of successful memory encoding. Nat. Neurosci. 5, 1339–1344. Paller, K.A., Kutas, 1992. Brain potentials during memory retrieval provide neurophysiological support for the distinction between conscious recollection and priming. J. Cogn. Neurosci. 4, 375–391. Paller, K.A., Wagner, A.D., 2002. Observing the transformation of experience into memory. Trends Cogn. Sci. 6, 93–102. Paller, K.A., Hutson, C.A., Miller, B.B., Boehm, S.G., 2003. Neural manifestations of memory with and without awareness. Neuron 38, 507–516. Paller, K.A., Voss, J.L., Boehm, S.G., 2007. Validating neural correlates of familiarity. Trends Cogn. Sci. 11, 243–250. Ramponi, C., Richardson-Klavehn, A., Gardiner, J.M., 2007. Component processes of conceptual priming and associative cued recall: the roles of preexisting representation and depth of processing. J. Exp. Psychol. Learn. Mem. Cogn. 33, 843–862. Richardson-Klavehn, A., Gardiner, J.M., 1998. Depth-of-processing effects on priming in stem completion: tests of the voluntary-contamination, conceptual-processing, and lexical-processing hypotheses. J. Exp. Psychol. Learn. Mem. Cogn. 24, 593–609. Rossion, B., 2008. Picture-plane inversion leads to qualitative changes of face perception. Acta Psychol. 128, 274–289. Rossion, B., 2009. Distinguishing the cause and consequence of face inversion: the perceptual field hypothesis. Acta Psychol. (Amst.) 132, 300–312. Rossion, B., Gauthier, I., 2002. How does the brain process upright and inverted faces? Behav. Cogn. Neurosci. Rev. 1, 63–75. Rossion, B., Jacques, C., 2008. Does physical interstimulus variance account for early electrophysiological face sensitive responses in the human brain? Ten lessons on the N170. Neuroimage 39, 1959–1979.

250

T. Marzi, M.P. Viggiano / International Journal of Psychophysiology 78 (2010) 239–250

Rossion, B., Campanella, S., Gomez, C.M., Delinte, A., Debatisse, D., Liard, L., Dubois, S., Bruyer, R., Crommelinck, M., Guerit, J.M., 1999. Task modulation of brain activity related to familiar and unfamiliar face processing: an ERP study. Clin. Neurophysiol. 110, 449–462. Rugg, M.D., Curran, T., 2007. Event-related potentials and recognition memory. Trends Cogn. Sci. 11, 251–257. Rugg, M.D., Mark, R.E., Walla, P., Schloerscheidt, A.M., Birch, C.S., Allan, K., 1998a. Dissociation of the neural correlates of implicit and explicit memory. Nature 392, 595–598. Rugg, M.D., Walla, P., Schloerscheidt, A.M., Fletcher, P.C., Frith, C.D., Dolan, R.J., 1998b. Neural correlates of depth of processing effects on recollection: evidence from brain potentials and positron emission tomography. Exp. Brain Res. 123, 18–23. Rugg, M.D., Allan, K., Birch, C.S., 2000. Electrophysiological evidence for the modulation of retrieval orientation by depth of study processing. J. Cogn. Neurosci. 12, 664–678. Schendan, H.E., Ganis, G., Kutas, M., 1998. Neurophysiological evidence for visual perceptual categorization of words and faces within 150 ms. Psychophysiology 35, 240–251. Schott, B., Richardson-Klavehn, A., Heinze, H.-J., Duzel, E., 2002. Perceptual priming versus explicit memory: dissociable neural correlates at encoding. J. Cogn. Neurosci. 14, 578–592. Schweinberger, S.R., Pickering, E.C., Jentzsch, I., Burton, A.M., Kaufmann, J.M., 2002. Event-related brain potential evidence for a response of inferior temporal cortex to familiar face repetitions. Brain Res. Cogn. Brain Res. 14, 398–409.

Schweinberger, S.R., Huddy, V., Burton, A.M., 2004. N250r: a face-selective brain response to stimulus repetitions. NeuroReport 15, 1501–1505. Scott, L.S., Tanaka, J.W., Sheinberg, D.L., Curran, T., 2006. A reevaluation of the electrophysiological correlates of expert object processing. J. Cogn. Neurosci. 18, 1453–1465. Snodgrass, J.G., Corwin, J., 1988. Pragmatics of measuring recognition memory: applications to dementia and amnesia. J. Exp. Psychol. Gen. 117, 34–50. Stahl, J., Wiese, H., Schweinberger, S.R., 2010. Learning task affects ERP-correlates of the own-race bias, but not recognition memory performance. Neuropsychologia 48, 2027–2040. Tanaka, J.W., Farah, M.J., 1993. Parts and wholes in face recognition. Q. J. Exp. Psychol. A 46, 225–245. Tanaka, J.W., Curran, T., Porterfield, A.L., Collins, D., 2006. Activation of preexisting and acquired face representations: the N250 event-related potential as an index of face familiarity. J. Cogn. Neurosci. 18, 1488–1497. Taylor, M.J., 2002. Non-spatial attentional effects on P1. Clin. Neurophysiol. 113, 1903–1908. Tsivilis, D., Otten, L.J., Rugg, M.D., 2001. Context effects on the neural correlates of recognition memory: an electrophysiological study. Neuron 31, 497–505. Yonelinas, A.P., 2001. Components of episodic memory: the contribution of recollection and familiarity. Philos. Trans. R. Soc. Lond. B Biol. Sci. 356, 1363–1374. Yovel, G., 2009. The shape of facial features and the spacing among them generate similar inversion effects: A reply to Rossion (2008). Acta Psychol. (Amst.) 132, 293–299.