- Email: [email protected]
Event-related potentials and recognition memory
Review
TRENDS in Cognitive Sciences
Vol.11 No.6
Event-related potentials and recognition memory Michael D. Rugg1 and Tim Curran2 1
Center for the Neurobiology of Learning and Memory, and Department of Neurobiology and Behavior, University of California, Irvine, Irvine, CA 92697, USA 2 Department of Psychology and Center for Neuroscience, University of Colorado, Boulder, CO 80309-0345, USA
According to dual-process models, recognition memory is supported by distinct retrieval processes known as familiarity and recollection. Important evidence supporting the dual-process framework has come from studies using event-related brain potentials (ERPs). These studies have identified two topographically distinct ERP correlates of recognition memory –the ‘parietal’ and ‘mid-frontal’ old/new effects – that are dissociated by variables that selectively modulate recollection and familiarity, respectively. We evaluate the extent to which ERP data support dual-process models in light of the proposal that recollection is a continuous rather than a discrete memory process. We also examine the claim that the putative ERP index of familiarity is a reflection of implicit rather than explicit memory. We conclude that ERP findings continue to offer strong support for the dual-process perspective. Introduction Recognition memory – the judgment that a stimulus event has been previously experienced – has been a major focus of interest among experimental psychologists and cognitive neuroscientists for more than two decades [1,2]. Among the issues addressed during this period one has stood out: whether recognition memory judgments are supported by one or by multiple kinds of mnemonic information. According to single-process accounts [3–5], recognition judgments are based on the evaluation of a single type of evidence, and a positive judgment is made when the strength of the evidence exceeds a criterion level. In common with the single-process account, dual-process accounts [6–8] propose that recognition is supported by an undifferentiated, strength-like form of information (usually referred to in the context of these accounts as familiarity). However, advocates of dualprocess accounts argue that recognition relies on a second, functionally distinct, memory signal that results from the retrieval of qualitative information about the study episode. The retrieval of episodic information in response to a recognition test item, which is usually assumed to rely on a subset of the processes that also support recall, is referred to as recollection. Although the relative merits of single- versus dual-process accounts of recognition continue to be debated, especially in the experimental psychology literature, it is Corresponding authors: Rugg, M.D. ([email protected]); Curran, T. ([email protected]). Available online 3 May 2007. www.sciencedirect.com
probably fair to say that the majority of contemporary researchers adhere to some kind of dual-process model, although the details of the different models differ markedly [6,7,9–11]. Most of these researchers would ascribe to the notion that familiarity-based recognition is fast-acting, relatively automatic and does not provide qualitative information about the study episode (see Box 1 for differing perspectives). Recollection, by contrast, is conceived as a slower, more effortful process that gives rise to consciously accessible information about both the prior occurrence of the test item and the context of that occurrence. Thus, recollection is often operationally defined as recognition accompanied by accurate source memory – memory for a specific feature of the study context, such as the location or color of an item. Another popular, albeit controversial [3,11], method for segregating recollection- and familiaritydriven recognition relies on the ‘remember/know’ procedure, which requires the subject to report whether recognition is accompanied by specific details of the study episode [12]. The broad acceptance of dual-process models stems not only from the findings of psychological experiments, some of which are still the subject of vigorous debate (e.g. [13,14]). Dual process models are also favored because converging evidence suggests that recollection and familiarity can be dissociated at the neural level. This evidence includes investigations of patients with lesions thought to be restricted to the hippocampus [15,16] (see Refs [17,18] for opposing evidence), differential patterns of neural activity revealed by functional magnetic resonance imaging (fMRI) (e.g. Refs [19–21]), and dissociations among retrieval-related event-related potential (ERP) effects. In this article, we briefly review the reasons why ERP findings are thought to support the dual-process perspective. We go on to consider how well these findings fare in the face of recent proposals that, one, recollection, in common with familiarity, is supported by a continuous rather than a discontinuous memory signal, and, two, a putative ERP correlate of familiarity has been mischaracterized and is in fact an index of implicit memory. ERP correlates of recollection and familiarity It is more than 25 years since the first reports that ERPs elicited by correctly classified old (studied) items are more positive-going than those elicited by correctly classified new (unstudied) test items [22,23]. It is these ERP ‘old/ new effects’ that are the subject of this article. We focus here on studies conducted since the mid-1990s that have
1364-6613/$ – see front matter ß 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2007.04.004
252
Review
TRENDS in Cognitive Sciences
Box 1. Theoretical conceptualization of familiarity The dual-process perspective, with its two processes of familiarity and recollection, is widely accepted, but the nature of familiarity is disputed. Familiarity is often operationally defined as information that supports recognition in the absence of recollection. This exclusionary definition leaves open the possibility that familiarity has more than one basis, so the alternatives considered below are not mutually exclusive. Most researchers agree that familiarity represents a content-free ‘strength of evidence’ signal. According to global matching models (e.g. Refs [63,64]), familiarity varies with the closeness of the match between test cues and previously studied information. Similar matching computations have been implemented in neural network models simulating cortical learning mechanisms [7]. Other perspectives suggest that familiarity arises from the facilitated processing accorded repeated stimuli (the ‘fluency heuristic’ [65,66]). Insofar as fluent processing is a reflection of implicit memory, this view might be compatible with accounts equating familiarity with perceptual [8] or conceptual [1] priming. Although these views have been challenged by evidence held to dissociate familiarity from implicit memory, it might be premature to dismiss this perspective altogether. If, as we believe, familiarity is multiply determined, then dissociations between implicit memory and familiarity might be observed under some conditions but not others (e.g. Ref. [67]).
employed manipulations aimed at separating the ERP effects according to whether recognition was accompanied by recollection or was based on familiarity (see Box 2). We limit our review to studies of retrieval-related activity rather than encoding (e.g. [24,25]), because retrieval
Vol.11 No.6
Box 2. ERPs and the neural dissociation of cognitive processes Unlike studies using fMRI or other functional neuroimaging methods, with ERP data it is rarely possible to dissociate cognitive processes on the basis of the differential localization of associated neural activations. Instead, qualitatively distinct ERP scalp topographies indicate that the distribution of neural activity within the brain differs in the respective experimental conditions. Topographic dissociations could reflect the engagement of distinct neural populations or differences in the relative activity levels in the members of a common population. Such differences potentially constitute evidence for a neural double dissociation, which indicates that two experimental manipulations engaged functionally distinct cognitive processes (for additional caveats, see Ref. [68]).
studies have provided the bulk of the evidence for ERP dissociations between recollection and familiarity. Findings from several early studies suggested that recollection has a distinct ERP signature, now often termed the ‘parietal’ old/new effect (Figures 1 and 2). The effect takes the form of a phasic, positive-going, parietally maximal ERP modulation that onsets around 400–500 ms post-stimulus onset and frequently exhibits a left-sided maximum. Perhaps the most compelling evidence linking the effect to recollection is its sensitivity to two common procedures for operationally defining recollection (see above). Thus, the effect is modulated according to whether the eliciting items are associated
Figure 1. (a) Grand-average ERPs from left frontal (F3) and left parietal (P3) electrodes. The left side of the figure depicts ERPs elicited by correctly classified new items and by shallowly studied old items correctly endorsed as old (shallow hits) or incorrectly endorsed as new (shallow misses). The right side depicts waveforms elicited by new items, shallow hits and correctly endorsed deeply studied old items (deep hits). The mid-frontal old/new effect is indicated by the blue arrow; the more posterior old/new effect hypothesized to reflect implicit memory is indicated by the red arrow; and the purple arrow indicates the parietal old/new effect. Reproduced from Ref. [35]. (b) Scalp topographies of the mid-frontal (left) and parietal (right) old/new effects, as revealed by the subtraction of shallow misses from shallow hits (left) and shallow hits from deep hits (right). In each figure, the nose is at the top, and minima and maxima are signaled by dark blue and dark red respectively. www.sciencedirect.com
Review
TRENDS in Cognitive Sciences
Vol.11 No.6
253
Figure 2. (a) Grand-average ERPs from frontal electrodes (average of electrode clusters including F3 and F4) with the mid-frontal old/new effect indicated by the blue arrow. Data from Ref. [36]. (b)Grand-average ERPs from left parietal (average of electrode cluster including P3) with the parietal old/new effect indicated by the purple arrow. Data from Ref. [36].
with successful or unsuccessful source judgments [26,27], and whether the items are endorsed as ‘remembered’ or ‘known’ [28–30] (but see Ref. [31]). Importantly, the effect has been functionally and topographically dissociated from other posteriorly distributed ERP effects that take place in the same time-frame, and which are sensitive to such factors as stimulus probability and response confidence [28,32–34]. Early studies demonstrating a relationship between the parietal old/new effect and recollection either did not address the question of whether ERPs were independently sensitive to familiarity or failed to find evidence of a familiarity correlate [26,30]. The first hint of a specific ERP signature of familiarity came from Du¨zel et al. [29], who reported that test items accorded ‘remember’ or ‘know’ judgments elicited old/new effects with different scalp topographies. Next, Rugg et al. [35] reported that old/new effects could be triply dissociated by a combination of functional and topographical variables. Subjects performed either a ‘deep’ (sentence generation) or a ‘shallow’ (alphabetic judgment) task on each word studied, followed by a recognition test. The key findings are illustrated in Figure 1. The parietal old/new effect was elicited exclusively by old items subjected to deep study, but two other old/new effects were also evident. An earlier (300–500 ms) transient positivity, also maximal over the parietal scalp, was elicited by all studied items irrespective of study task or recognition accuracy. Rugg et al. [35] interpreted this effect as a reflection of implicit memory (priming) that operated independently of explicit memory judgments. A second effect, occurring in the same time range but with a frontal scalp distribution, was also insensitive to depth-of-study processing. Unlike the posterior effect, this ‘mid-frontal’ effect was seen only for items correctly endorsed as old. On the basis of behavioral evidence that depth-of-processing manipulations influence recollection more than familiarity [1], Rugg et al. [35] proposed that the mid-frontal old/new effect was an ERP correlate of familiarity-driven recognition. According to this proposal, recognition of deeply studied words was based on both recollection (indexed by the parietal old/new effect) and familiarity (indexed by the mid-frontal old/new www.sciencedirect.com
effect), whereas shallowly studied words were recognized largely on the basis of their familiarity. Several studies have reported functional dissociations between parietal and mid-frontal old/new effects when using experimental procedures that allowed familiaritydriven recognition judgments to be identified more directly. The first such study was motivated by behavioral evidence that ‘new’ test items that share features with studied items elicit high rates of familiarity-driven false alarms [36]. Some new words on the recognition test were dissimilar to studied items, but others differed from a studied word solely by virtue of a plurality reversal (e.g. TRUCK and TRUCKS). Whereas the parietal old/new effect was elicited only by correctly recognized old items (items that could be correctly classified on the basis of both recollection and familiarity), both studied items and plurality-reversed items incorrectly endorsed as old elicited a 300–500 ms mid-frontal effect (called the ‘FN400 old/new effect’ by Curran [36] and by Paller et al. [37]). This pattern is consistent with the proposal that the 400–800 ms parietal effect is linked to the recollection of specific information, whereas the 300–500 ms mid-frontal effect is linked to familiarity-driven recognition (Figure 2). Similar findings have been reported for false alarms elicited by new words sharing semantic features with studied words [38] and new pictures that were mirror-reversals of originally studied pictures [39]. The mid-frontal effect has also been reported for false alarms in a study where there was no manipulation of similarity between new and studied items [40]. As already noted, it is generally assumed that familiarity is a continuously varying signal and that positive recognition decisions are made when the strength of the signal exceeds a set criterion. Thus, the average familiarity of correctly classified new and old items will increase as the criterion becomes more conservative. Therefore, if the midfrontal effect reflects familiarity, it should be sensitive to criterion placement. This prediction was confirmed in a study by Azimian-Faridani and Wilding [41]. Relative to the ERPs elicited under a liberal response criterion, the waveforms elicited by correctly classified new and old items
254
Review
TRENDS in Cognitive Sciences
both demonstrated mid-frontal effects when subjects were instructed to respond conservatively. Woodruff et al. [32] also investigated whether the magnitude of the mid-frontal old/new effect co-varies with familiarity strength. Subjects performed a modified remember/know task in which test items that were not endorsed as remembered were rated on a four-point scale between ‘definitely old’ and ‘definitely new’ on the assumption that confidence covaried with the strength of the underlying familiarity signal. The magnitude of the mid-frontal effect varied monotonically with confidence that an item was old, but did not differ according to whether test items were endorsed as recollected or were recognized with high confidence in the absence of recollection. Conversely, the parietal old/new effect was selectively enhanced for recollected items and did not vary with confidence for items that were not recollected. Recollection as a continuous process The relationship between familiarity, recollection and confidence is a hotly debated topic that might be informed by ERP studies. According to Yonelinas and colleagues [1,6,42], recollection is best conceptualized as representing a discrete, thresholded mnemonic state (that is, recollection is in some sense all or none’), whereas others have suggested that both familiarity and recollection are continuously varying memory signals that are combined before a recognition judgment is made [11]. From the standpoint of recollection as a continuous signal, the finding that the mid-frontal effect varies with confidence [32] does not necessarily indicate a neural correlate of familiarity. Rather, an ERP effect that varies with confidence that an item is old could reflect gradation in the strength of familiarity, recollection or a signal based on some combination of the two processes. By this view, therefore, ERP effects that differentiate ‘recollected’ and ‘non-recollected’ items (exemplified by remember versus know judgments) might merely reflect differences in overall memory strength rather than in underlying memory processes. However, such an account does not readily explain the double-dissociation between mid-frontal and parietal effects observed by Woodruff et al. [32]. If the memory signal supporting a ‘remember’ response differs quantitatively, but not qualitatively, from the signal that supports ‘sub-recollective’ responding, it is unclear why the midfrontal effect should reflect memory strength for items that are not endorsed as remembered but be insensitive to whether an item is remembered or confidently judged familiar. An analogous issue arises for the parietal effect: it is unclear how the findings that the effect is elicited exclusively by items endorsed as remembered and is topographically dissociable from generic confidence effects can be reconciled with the notion that a remember response is simply a high-confidence recognition decision. In summary, although additional work is needed to examine the relationship between confidence, familiarity and recollection, the existing ERP data are difficult to reconcile with the view that the remember/know distinction merely indicates differences in memory confidence or strength. Rather, the data suggest that these two memory judgments are the outcome of qualitatively distinct memory processes. www.sciencedirect.com
Vol.11 No.6
The midfrontal effect and conceptual priming The proposal that the mid-frontal effect is a neural correlate of familiarity has recently been challenged. An hypothesis has been advanced that the effect is linked to a form of implicit memory known as conceptual priming [24,37,43]. Conceptual priming refers to a form of repetition priming that depends on repeated access to semantic, rather than perceptual, representations [37]. According to Paller and colleagues [37], most of the experimental manipulations used to isolate or modulate the neural correlates of familiarity are not ‘process pure’, and instead exert parallel effects on conceptual priming. Hence, studies employing these manipulations do not provide a sound basis for identifying a specific neural correlate of familiarity. Below, we briefly discuss two issues relevant to this alternative account. First, we discuss whether the mid-frontal effect is modulated exclusively by conceptual overlap between study and test items. And second, we ask whether the correspondences that exist between the magnitude of the effect and familiarity-driven recognition judgments are compatible with a priming account. Evidence relevant to the first of these issues comes from studies using seemingly semantically ‘empty’ items. Robust mid-frontal effects have been reported for successful recognition of pre-experimentally unfamiliar, abstract visual patterns [44,45] and unfamiliar faces [33,46]. As these items are devoid of semantic content, the findings are difficult to reconcile with the view that the mid-frontal effect depends upon a high degree of conceptual overlap between study and test items. However, not all studies using meaningless items have reported a mid-frontal effect. Yovel and Paller [24] taught subjects to associate pre-experimentally unfamiliar faces with fictitious occupations. During the subsequent test, subjects discriminated studied and unstudied faces and, for those faces judged to be old items, attempted to retrieve the occupation paired with the face or, failing that, any other details about the study episode. Faces for which no study information could be reported were considered to have been recognized on the basis of ‘pure familiarity’. Belying the proposal that it reflects familiarity-driven recognition, the ERPs elicited by these items showed no evidence of the mid-frontal effect. The generality of this finding is uncertain. Whereas one follow-up study replicated the original failure to find a midfrontal effect [47], a second study reported a robust effect for faces recognized on the basis of familiarity alone [33] (Figure 3). The explanation for these divergent findings is unclear, but it is noteworthy that recognition performance in the study where the mid-frontal effect was evident [33] was substantially higher than in the two studies where it was not [24,47]. Further research is needed to determine whether these disparities in performance reflect differences in familiarity strength, and hence whether they could account for the divergent ERP findings. Whereas the bulk of the evidence suggests that the mid-frontal effect can be elicited by meaningless items, it can be argued that the effect emerges because subjects typically imbue such items with semantic content [37]. A complementary line of evidence comes from studies that manipulated item format between the study and test in ways that preserve conceptual overlap. One such study [48]
Review
TRENDS in Cognitive Sciences
Vol.11 No.6
255
neither purely conceptual nor limited to supporting implicit memory. As noted in Box 1, there could be circumstances in which conceptual priming contributes to familiarity and, under such circumstances, an ERP correlate of conceptual priming would be expected to co-vary with recognitionmemory accuracy. However, evidence that the mid-frontal effect is not exclusively sensitive to conceptual overlap between study and test items indicates that the construct of conceptual priming is too limited to account for the data, even on the assumption that such priming invariably contributes to familiarity-driven recognition.
Figure 3. Grand-average ERPs from frontal electrodes (average of electrode clusters including F3 and F4) with the mid-frontal old/new effect indicated by the blue arrow. Subjects studied pictures of faces paired with occupation names. Recognition memory judgments were followed by an attempt to recollect the associated occupation. ERPs are plotted for faces that were correctly rejected, missed or correctly recognized (a hit). Hits were broken down according to those that were associated with correct occupation recollection (occupation hit) versus those without correct occupation recollection (familiar hit). Reproduced, with permission, from Ref. [33].
reported that the magnitude of the mid-frontal effect was not significantly affected by a shift in sensory modality (auditory to visual), although a non-significant attenuation was noted in the between-modality condition. In two other studies, format change was associated with a robust attenuation of the effect. In one case [49], the format change was between words and pictures. In the second study [44], the color of the test items (line-drawings) was changed. In light of the results of these and other studies, Groh-Bordin and colleagues [44] proposed that both conceptual and perceptual information can contribute to the familiarity signal reflected by the mid-frontal effect, but that the relative contributions of the two classes of information varies depending on the different tasks and stimuli involved. A second line of evidence relevant to Yovel and Paller’s [24] hypothesis comes from the variety of circumstances where the mid-frontal effect co-varies with behavioral performance in recognition-memory tasks rather with the study status of the test items. Thus, the effect is modulated by recognition accuracy (smaller for misses than for hits) [33,35,41,50], is not evident when ERPs elicited by studied versus unstudied words are equated for rating of familiarity strength [32], varies with response criterion [32] (see also Ref. [51]), is elicited by false alarms in addition to correctly recognized old items [36,40], and demonstrates a positive across-subjects correlation with familiarity-based recognition performance [52]. It is unclear how these findings can be reconciled with any hypothesis proposing that the mid-frontal effect reflects processes uncorrelated with those that support recognition judgments. The evidence reviewed in this section indicates that the processes reflected by the mid-frontal old/new effect are www.sciencedirect.com
Functional significance of old/new effects The evidence linking the mid-frontal and parietal old/new effects to familiarity and recollection is arguably strong; however, the cognitive operations reflected by these effects remain to be identified. In the case of the mid-frontal effect, there are grounds for thinking that the effect does not reflect familiarity directly. Tsivilis et al. [50] had subjects study visual objects superimposed upon background scenes. During the test, old and new objects were superimposed either on studied or unstudied scenes. Correctly classified old objects superimposed on studied scenes elicited a mid-frontal effect, but those paired with a novel scene did not, although a companion behavioral study revealed no difference between these two conditions in estimates of familiarity. Tsivilis et al. proposed that the mid-frontal effect reflects processes downstream from those responsible for computing the familiarity of the different elements in an episode, and that an earlier old/ new effect (onsetting 120 ms) might be a more direct reflection of the accumulation of familiarity information (see also [25,53]). A recent study [54] extended the findings of Tsivilis et al. by demonstrating that a test item superimposed on a novel background elicits a mid-frontal effect when attention is focused exclusively on the item, but not when both item and background are attended to. Thus, the effect reflects attentionally mediated processing of multiple sources of familiarity information. The functional significance of the parietal old/new effect is equally obscure. One early suggestion was that the effect reflects processes that contribute to the representation of recollected information [26]. Alternatively, the effect might index attentional orienting to recollected information [55,56], rather than processes supporting its representation or maintenance. It has recently been argued [53] that findings indicating that the effect varies according to the amount of information recollected [53,57] are more consistent with the first of these two proposals. Neural generators of the mid-frontal and parietal old/new effects Single-neuron studies in non-human primates and human fMRI studies have identified putative familiarity-sensitive neuronal populations in the perirhinal region of the medial temporal lobe [58,59]. However, the onset latency of these neuronal responses is of the order of 90 ms [59], which is considerably earlier than the onset of the mid-frontal ERP effect. Familiarity-sensitive neuronal responses have also been identified in the monkey in a variety of prefrontal regions [60], with onset latencies (at 250 ms) fitting well
256
Review
TRENDS in Cognitive Sciences
Box 3. Questions for future research Can ERP data help adjudicate between threshold and continuous models of recollection? What cognitive operations are directly reflected by the mid-frontal and parietal old/new effects? What is the relationship between the mid-frontal old/new effect and earlier onsetting effects that also seem to track familiarity? How is memory-related fMRI activity that originates in pre-frontal and parietal cortices related to the mid-frontal and parietal ERP old/new effects?
with the time-course of the mid-frontal ERP effect. Together with fMRI evidence that the strength of familiarity modulates lateral prefrontal activity in humans [21], these findings suggest that the mid-frontal effect originates from one or more regions of the prefrontal cortex. It will be of considerable interest to determine how closely the functional characteristics of familiarity-sensitive prefrontal activity, as indexed by fMRI, match the characteristics of the ERP effect. The characteristic scalp distribution of the parietal old/ new effect (Figure 1) suggests that it might reflect neural activity generated in the lateral parietal cortex. fMRI findings of recollection-sensitive activity in this region (reviewed in Refs [55,56]) are consistent with this suggestion. Findings of direct functional parallels between these fMRI and ERP old/new effects [21,32,34,53,61,62] give additional credence to the hypothesis that the ERP effect reflects activity in a recollection-sensitive region of the lateral parietal cortex. Concluding comments ERP studies of recognition memory provided some of the first evidence that recollection and familiarity have qualitatively distinct neural correlates. More recent ERP findings have added to this evidence and to the general weight of the evidence supporting dual-process models of recognition memory. Among the challenges for the future (Box 3) are the elucidation of the cognitive operations reflected by these ERP effects and the identification of their neural generators. Acknowledgements Research described in this article was supported by National Institutes of Health grants MH072966 (M.D.R.) and MH64812 (T.C.).
References 1 Yonelinas, A.P. (2002) The nature of recollection and familiarity: a review of 30 years of research. J. Mem. Lang. 46, 441–517 2 Rugg, M.D. and Yonelinas, A.P. (2003) Human recognition memory: a cognitive neuroscience perspective. Trends Cogn. Sci. 7, 313–319 3 Dunn, J.C. (2004) Remember-know: a matter of confidence. Psychol. Rev. 111, 524–542 4 Donaldson, W. (1996) The role of decision processes in remembering and knowing. Mem. Cognit. 24, 523–533 5 Heathcote, A. et al. (2006) Recollection and familiarity in recognition memory: evidence from ROC curves. J. Mem. Lang, DOI: 10.1016/j.jml. 2006.1007.1001 6 Yonelinas, A.P. (2001) Components of episodic memory: the contribution of recollection and familiarity. Philos. Trans. R. Soc. Lond. B Biol. Sci. 356, 1363–1374 7 Norman, K.A. and O’Reilly, R.C. (2003) Modeling hippocampal and neocortical contributions to recognition memory: a complementary learning systems approach. Psychol. Rev. 110, 611–646 www.sciencedirect.com
Vol.11 No.6
8 Mandler, G. (1980) Recognizing: the judgment of previous occurrence. Psychol. Rev. 87, 252–271 9 Diana, R.A. et al. (2006) Models of recognition: a review of arguments in favor of a dual-process account. Psychon. Bull. Rev. 13, 1–21 10 Rotello, C.M. et al. (2004) Sum-difference theory of remembering and knowing: a two-dimensional signal-detection model. Psychol. Rev. 111, 588–616 11 Wixted, J.T. (2007) Dual-process theory and signal-detection theory of recognition memory. Psychol. Rev. 114, 152–176 12 Tulving, E. (1985) Memory and consciousness. Can. Psychol. 26, 1–12 13 Heathcote, A. (2003) Item recognition memory and the receiver operating characteristic. J. Exp. Psychol. Learn. Mem. Cogn. 29, 1210–1230 14 Hilford, A. et al. (2002) Regularities of source recognition: ROC analysis. J. Exp. Psychol. Gen. 131, 494–510 15 Aggleton, J.P. et al. (2005) Sparing of the familiarity component of recognition memory in a patient with hippocampal pathology. Neuropsychologia 43, 1810–1823 16 Holdstock, J.S. et al. (2002) Under what conditions is recognition spared relative to recall after selective hippocampal damage in humans? Hippocampus 12, 341–351 17 Manns, J.R. et al. (2003) Recognition memory and the human hippocampus. Neuron 37, 171–180 18 Wais, P.E. et al. (2006) The hippocampus supports both the recollection and the familiarity components of recognition memory. Neuron 49, 459–466 19 Davachi, L. et al. (2003) Multiple routes to memory: distinct medial temporal lobe processes build item and source memories. Proc. Natl. Acad. Sci. U. S. A. 100, 2157–2162 20 Eldridge, L.L. (2000) Remembering episodes: a selective role for the hippocampus during retrieval. Nat. Neurosci. 3, 1149–1152 21 Yonelinas, A.P. et al. (2005) Separating the brain regions involved in recollection and familiarity in recognition memory. J. Neurosci. 25, 3002–3008 22 Sanquist, T.F. et al. (1980) Electrocortical signs of levels of processing: perceptual analysis and recognition memory. Psychophysiology 17, 568–576 23 Warren, L.R. (1980) Evoked-potential correlates of recognition memory. Biol. Psychol. 11, 21–35 24 Yovel, G. and Paller, K.A. (2004) The neural basis of the butcher-onthe-bus phenomenon: when a face seems familiar but is not remembered. Neuroimage 21, 789–800 25 Duarte, A. et al. (2004) Dissociable neural correlates for familiarity and recollection during the encoding and retrieval of pictures.. Brain Res. Cogn. Brain Res. 18, 255–272 26 Wilding, E.L. and Rugg, M.D. (1996) An event-related potential study of recognition memory with and without retrieval of source. Brain 119, 889–905 27 Senkfor, A.J. and Van Petten, C. (1998) Who said what? An eventrelated potential investigation of source and item memory. J. Exp. Psychol. Learn. Mem. Cogn. 24, 1005–1025 28 Curran, T. (2004) Effects of attention and confidence on the hypothesized ERP correlates of recollection and familiarity. Neuropsychologia 42, 1088–1106 29 Du¨zel, E. et al. (1997) Event-related potential correlates of two states of conscious awareness in memory. Proc. Natl. Acad. Sci. U. S. A. 94, 5973–5978 30 Smith, M.E. (1993) Neurophysiological manifestations of recollective experience during recognition memory judgments. J. Cogn. Neurosci. 5, 1–13 31 Spencer, K.M. et al. (2000) On the search for the neurophysiological manifestation of recollective experience. Psychophysiology 37, 494– 506 32 Woodruff, C.C. et al. (2006) Electrophysiological dissociation of the neural correlates of recollection and familiarity. Brain Res. 1100, 125– 135 33 Curran, T. and Hancock, J. (2007) The FN400 indexes familiaritybased recognition of faces. Neuroimage 36, 464–471 34 Herron, J.E. et al. (2004) Probability effects on the neural correlates of retrieval success: an fMRI study. Neuroimage 21, 302–310 35 Rugg, M.D. et al. (1998) Dissociation of the neural correlates of implicit and explicit memory. Nature 392, 595–598
Review
TRENDS in Cognitive Sciences
36 Curran, T. (2000) Brain potentials of recollection and familiarity. Mem. Cognit. 28, 923–938 37 Paller, K.A. et al. (2007) Validating neural correlates of familiarity. Trends Cogn. Sci 11, 243–250 38 Nessler, D. et al. (2001) Event related brain potentials and illusory memories: the effects of differential encoding. Brain Res. Cogn. Brain Res. 10, 283–301 39 Curran, T. and Cleary, A.M. (2003) Using ERPs to dissociate recollection from familiarity in picture recognition. Brain Res. Cogn. Brain Res. 15, 191–205 40 Wolk, D.A. et al. (2006) ERP correlates of recognition memory: effects of retention interval and false alarms. Brain Res. 1096, 148–162 41 Azimian-Faridani, N. and Wilding, E.L. (2006) The influence of criterion shifts on electrophysiological correlates of recognition memory. J. Cogn. Neurosci. 18, 1075–1086 42 Parks, C.M. and Yonelinas, A.P. (2007) Moving beyond pure signaldetection models: comment on Wixted (2007). Psychol. Rev. 114, 188– 202 43 Voss, J.L. and Paller, K.A. (2006) Fluent conceptual processing and explicit memory for faces are electrophysiologically distinct. J. Neurosci. 26, 926–933 44 Groh-Bordin, C. et al. (2006) Has the butcher on the bus dyed his hair? When color changes modulate ERP correlates of familiarity and recollection. Neuroimage 32, 1879–1890 45 Curran, T. et al. (2002) An electrophysiological comparison of visual categorization and recognition memory. Cogn. Affect. Behav. Neurosci. 2, 1–18 46 Nessler, D. et al. (2005) Perceptual fluency, semantic familiarity and recognition-related familiarity: an electrophysiological exploration. Brain Res. Cogn. Brain Res. 22, 265–288 47 MacKenzie, G. and Donaldson, D.I. (2007) Dissociating recollection from familiarity: electrophysiological evidence that familiarity for faces is associated with a posterior old/new effect. Neuroimage 36, 454–463 48 Curran, T. and Dien, J. (2003) Differentiating amodal familiarity from modality-specific memory processes: an ERP study. Psychophysiology 40, 979–988 49 Schloerscheidt, A.M. and Rugg, M.D. (2004) The impact of change in stimulus format on the electrophysiological indices of recognition. Neuropsychologia 42, 451–466 50 Tsivilis, D. et al. (2001) Context effects on the neural correlates of recognition memory: an electrophysiological study. Neuron 31, 497–505 51 Bridson, N.C. et al. (2006) Electrophysiological correlates of familiarity in recognition memory and exclusion tasks. Brain Res. 1114, 149–160 52 Curran, T. et al. (2006) Combined pharmacological and electrophysiological dissociation of familiarity and recollection. J. Neurosci. 26, 1979–1985
Vol.11 No.6
53 Vilberg, K.L. et al. (2006) The relationship between electrophysiological correlates of recollection and amount of information retrieved. Brain Res. 1122, 161–170 54 Ecker, U.K. et al. (2007) Context effects on familiarity are familiarity effects of context – an electrophysiological study. Int. J. Psychophysiol 64, 146–156 55 Rugg, M.D. and Henson, R.N.A. (2002) Episodic memory retrieval: an (event-related) functional neuroimaging perspective. In The Cognitive Neuroscience of Memory Encoding and Retrieval (Parker, A.E. et al., eds), pp. 3–38, Psychology Press 56 Wagner, A.D. et al. (2005) Parietal lobe contributions to episodic memory retrieval. Trends Cogn. Sci. 9, 445–453 57 Wilding, E.L. (2000) In what way does the parietal ERP old/new effect index recollection? Int. J. Psychophysiol. 35, 81–87 58 Henson, R.N. et al. (2003) A familiarity signal in human anterior medial temporal cortex? Hippocampus 13, 301–304 59 Xiang, J.Z. and Brown, M.W. (1998) Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe. Neuropharmacology 37, 657–676 60 Xiang, J.Z. and Brown, M.W. (2004) Neuronal responses related to long-term recognition memory processes in prefrontal cortex. Neuron 42, 817–829 61 Herron, J.E. et al. (2003) Probability effects on event-related potential correlates of recognition memory. Brain Res. Cogn. Brain Res. 16, 66–73 62 Vilberg, K.L. and Rugg, M.D. (2007) Dissociation of the neural correlates of recognition memory according to familiarity, recollection, and amount of recollected information. Neuropsychologia 45, 2216– 2225 63 Shiffrin, R.M. and Steyvers, M. (1997) A model of recognition memory: REM—retrieving effectively from memory. Psychol. Bull. Rev. 4, 145– 166 64 Hintzman, D.L. (1988) Judgments of frequency and recognition memory in a multiple-trace memory model. Psychol. Rev. 95, 528– 551 65 Jacoby, L.L. and Dallas, M. (1981) On the relationship between autobiographical memory and perceptual learning. J. Exp. Psychol. Gen. 110, 306–340 66 Whittlesea, B.W.A. et al. (1990) Illusions of immediate memory: evidence of an attributional basis for feelings of familiarity and perceptual quality. J. Mem. Lang. 29, 716–732 67 Whittlesea, B.W.A. and Price, J.R. (2001) Implicit/explicit memory versus analytic/nonanalytic processing: rethinking the mere exposure effect. Mem. Cognit. 29, 234–246 68 Rugg, M.D. and Coles, M.G.H. (1995) The ERP and cognitive psychology: conceptual issues. In Electrophysiology of Mind (Rugg, M.D. and Coles, M.G.H., eds), pp. 27–39, Oxford University Press
The ScienceDirect collection ScienceDirect’s extensive and unique full-text collection covers more than 1900 journals, including titles such as The Lancet, Cell, Tetrahedron and the full suite of Trends, Current Opinion and Drug Discovery Today journals. With ScienceDirect, the research process is enhanced with unsurpassed searching and linking functionality, all on a single, intuitive interface. The rapid growth of the ScienceDirect collection is a result of the integration of several prestigious publications and the ongoing addition to the Backfiles – heritage collections in a number of disciplines. The latest step in this ambitious project to digitize all of Elsevier’s journals back to volume one, issue one, is the addition of the highly cited Cell Press journal collection on ScienceDirect. Also available online for the first time are six Cell titles’ long-awaited Backfiles, containing more than 12,000 articles that highlight important historic developments in the field of life sciences.
For more information, visit www.sciencedirect.com www.sciencedirect.com
257
TRENDS in Cognitive Sciences
Vol.11 No.6
Event-related potentials and recognition memory Michael D. Rugg1 and Tim Curran2 1
Center for the Neurobiology of Learning and Memory, and Department of Neurobiology and Behavior, University of California, Irvine, Irvine, CA 92697, USA 2 Department of Psychology and Center for Neuroscience, University of Colorado, Boulder, CO 80309-0345, USA
According to dual-process models, recognition memory is supported by distinct retrieval processes known as familiarity and recollection. Important evidence supporting the dual-process framework has come from studies using event-related brain potentials (ERPs). These studies have identified two topographically distinct ERP correlates of recognition memory –the ‘parietal’ and ‘mid-frontal’ old/new effects – that are dissociated by variables that selectively modulate recollection and familiarity, respectively. We evaluate the extent to which ERP data support dual-process models in light of the proposal that recollection is a continuous rather than a discrete memory process. We also examine the claim that the putative ERP index of familiarity is a reflection of implicit rather than explicit memory. We conclude that ERP findings continue to offer strong support for the dual-process perspective. Introduction Recognition memory – the judgment that a stimulus event has been previously experienced – has been a major focus of interest among experimental psychologists and cognitive neuroscientists for more than two decades [1,2]. Among the issues addressed during this period one has stood out: whether recognition memory judgments are supported by one or by multiple kinds of mnemonic information. According to single-process accounts [3–5], recognition judgments are based on the evaluation of a single type of evidence, and a positive judgment is made when the strength of the evidence exceeds a criterion level. In common with the single-process account, dual-process accounts [6–8] propose that recognition is supported by an undifferentiated, strength-like form of information (usually referred to in the context of these accounts as familiarity). However, advocates of dualprocess accounts argue that recognition relies on a second, functionally distinct, memory signal that results from the retrieval of qualitative information about the study episode. The retrieval of episodic information in response to a recognition test item, which is usually assumed to rely on a subset of the processes that also support recall, is referred to as recollection. Although the relative merits of single- versus dual-process accounts of recognition continue to be debated, especially in the experimental psychology literature, it is Corresponding authors: Rugg, M.D. ([email protected]); Curran, T. ([email protected]). Available online 3 May 2007. www.sciencedirect.com
probably fair to say that the majority of contemporary researchers adhere to some kind of dual-process model, although the details of the different models differ markedly [6,7,9–11]. Most of these researchers would ascribe to the notion that familiarity-based recognition is fast-acting, relatively automatic and does not provide qualitative information about the study episode (see Box 1 for differing perspectives). Recollection, by contrast, is conceived as a slower, more effortful process that gives rise to consciously accessible information about both the prior occurrence of the test item and the context of that occurrence. Thus, recollection is often operationally defined as recognition accompanied by accurate source memory – memory for a specific feature of the study context, such as the location or color of an item. Another popular, albeit controversial [3,11], method for segregating recollection- and familiaritydriven recognition relies on the ‘remember/know’ procedure, which requires the subject to report whether recognition is accompanied by specific details of the study episode [12]. The broad acceptance of dual-process models stems not only from the findings of psychological experiments, some of which are still the subject of vigorous debate (e.g. [13,14]). Dual process models are also favored because converging evidence suggests that recollection and familiarity can be dissociated at the neural level. This evidence includes investigations of patients with lesions thought to be restricted to the hippocampus [15,16] (see Refs [17,18] for opposing evidence), differential patterns of neural activity revealed by functional magnetic resonance imaging (fMRI) (e.g. Refs [19–21]), and dissociations among retrieval-related event-related potential (ERP) effects. In this article, we briefly review the reasons why ERP findings are thought to support the dual-process perspective. We go on to consider how well these findings fare in the face of recent proposals that, one, recollection, in common with familiarity, is supported by a continuous rather than a discontinuous memory signal, and, two, a putative ERP correlate of familiarity has been mischaracterized and is in fact an index of implicit memory. ERP correlates of recollection and familiarity It is more than 25 years since the first reports that ERPs elicited by correctly classified old (studied) items are more positive-going than those elicited by correctly classified new (unstudied) test items [22,23]. It is these ERP ‘old/ new effects’ that are the subject of this article. We focus here on studies conducted since the mid-1990s that have
1364-6613/$ – see front matter ß 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2007.04.004
252
Review
TRENDS in Cognitive Sciences
Box 1. Theoretical conceptualization of familiarity The dual-process perspective, with its two processes of familiarity and recollection, is widely accepted, but the nature of familiarity is disputed. Familiarity is often operationally defined as information that supports recognition in the absence of recollection. This exclusionary definition leaves open the possibility that familiarity has more than one basis, so the alternatives considered below are not mutually exclusive. Most researchers agree that familiarity represents a content-free ‘strength of evidence’ signal. According to global matching models (e.g. Refs [63,64]), familiarity varies with the closeness of the match between test cues and previously studied information. Similar matching computations have been implemented in neural network models simulating cortical learning mechanisms [7]. Other perspectives suggest that familiarity arises from the facilitated processing accorded repeated stimuli (the ‘fluency heuristic’ [65,66]). Insofar as fluent processing is a reflection of implicit memory, this view might be compatible with accounts equating familiarity with perceptual [8] or conceptual [1] priming. Although these views have been challenged by evidence held to dissociate familiarity from implicit memory, it might be premature to dismiss this perspective altogether. If, as we believe, familiarity is multiply determined, then dissociations between implicit memory and familiarity might be observed under some conditions but not others (e.g. Ref. [67]).
employed manipulations aimed at separating the ERP effects according to whether recognition was accompanied by recollection or was based on familiarity (see Box 2). We limit our review to studies of retrieval-related activity rather than encoding (e.g. [24,25]), because retrieval
Vol.11 No.6
Box 2. ERPs and the neural dissociation of cognitive processes Unlike studies using fMRI or other functional neuroimaging methods, with ERP data it is rarely possible to dissociate cognitive processes on the basis of the differential localization of associated neural activations. Instead, qualitatively distinct ERP scalp topographies indicate that the distribution of neural activity within the brain differs in the respective experimental conditions. Topographic dissociations could reflect the engagement of distinct neural populations or differences in the relative activity levels in the members of a common population. Such differences potentially constitute evidence for a neural double dissociation, which indicates that two experimental manipulations engaged functionally distinct cognitive processes (for additional caveats, see Ref. [68]).
studies have provided the bulk of the evidence for ERP dissociations between recollection and familiarity. Findings from several early studies suggested that recollection has a distinct ERP signature, now often termed the ‘parietal’ old/new effect (Figures 1 and 2). The effect takes the form of a phasic, positive-going, parietally maximal ERP modulation that onsets around 400–500 ms post-stimulus onset and frequently exhibits a left-sided maximum. Perhaps the most compelling evidence linking the effect to recollection is its sensitivity to two common procedures for operationally defining recollection (see above). Thus, the effect is modulated according to whether the eliciting items are associated
Figure 1. (a) Grand-average ERPs from left frontal (F3) and left parietal (P3) electrodes. The left side of the figure depicts ERPs elicited by correctly classified new items and by shallowly studied old items correctly endorsed as old (shallow hits) or incorrectly endorsed as new (shallow misses). The right side depicts waveforms elicited by new items, shallow hits and correctly endorsed deeply studied old items (deep hits). The mid-frontal old/new effect is indicated by the blue arrow; the more posterior old/new effect hypothesized to reflect implicit memory is indicated by the red arrow; and the purple arrow indicates the parietal old/new effect. Reproduced from Ref. [35]. (b) Scalp topographies of the mid-frontal (left) and parietal (right) old/new effects, as revealed by the subtraction of shallow misses from shallow hits (left) and shallow hits from deep hits (right). In each figure, the nose is at the top, and minima and maxima are signaled by dark blue and dark red respectively. www.sciencedirect.com
Review
TRENDS in Cognitive Sciences
Vol.11 No.6
253
Figure 2. (a) Grand-average ERPs from frontal electrodes (average of electrode clusters including F3 and F4) with the mid-frontal old/new effect indicated by the blue arrow. Data from Ref. [36]. (b)Grand-average ERPs from left parietal (average of electrode cluster including P3) with the parietal old/new effect indicated by the purple arrow. Data from Ref. [36].
with successful or unsuccessful source judgments [26,27], and whether the items are endorsed as ‘remembered’ or ‘known’ [28–30] (but see Ref. [31]). Importantly, the effect has been functionally and topographically dissociated from other posteriorly distributed ERP effects that take place in the same time-frame, and which are sensitive to such factors as stimulus probability and response confidence [28,32–34]. Early studies demonstrating a relationship between the parietal old/new effect and recollection either did not address the question of whether ERPs were independently sensitive to familiarity or failed to find evidence of a familiarity correlate [26,30]. The first hint of a specific ERP signature of familiarity came from Du¨zel et al. [29], who reported that test items accorded ‘remember’ or ‘know’ judgments elicited old/new effects with different scalp topographies. Next, Rugg et al. [35] reported that old/new effects could be triply dissociated by a combination of functional and topographical variables. Subjects performed either a ‘deep’ (sentence generation) or a ‘shallow’ (alphabetic judgment) task on each word studied, followed by a recognition test. The key findings are illustrated in Figure 1. The parietal old/new effect was elicited exclusively by old items subjected to deep study, but two other old/new effects were also evident. An earlier (300–500 ms) transient positivity, also maximal over the parietal scalp, was elicited by all studied items irrespective of study task or recognition accuracy. Rugg et al. [35] interpreted this effect as a reflection of implicit memory (priming) that operated independently of explicit memory judgments. A second effect, occurring in the same time range but with a frontal scalp distribution, was also insensitive to depth-of-study processing. Unlike the posterior effect, this ‘mid-frontal’ effect was seen only for items correctly endorsed as old. On the basis of behavioral evidence that depth-of-processing manipulations influence recollection more than familiarity [1], Rugg et al. [35] proposed that the mid-frontal old/new effect was an ERP correlate of familiarity-driven recognition. According to this proposal, recognition of deeply studied words was based on both recollection (indexed by the parietal old/new effect) and familiarity (indexed by the mid-frontal old/new www.sciencedirect.com
effect), whereas shallowly studied words were recognized largely on the basis of their familiarity. Several studies have reported functional dissociations between parietal and mid-frontal old/new effects when using experimental procedures that allowed familiaritydriven recognition judgments to be identified more directly. The first such study was motivated by behavioral evidence that ‘new’ test items that share features with studied items elicit high rates of familiarity-driven false alarms [36]. Some new words on the recognition test were dissimilar to studied items, but others differed from a studied word solely by virtue of a plurality reversal (e.g. TRUCK and TRUCKS). Whereas the parietal old/new effect was elicited only by correctly recognized old items (items that could be correctly classified on the basis of both recollection and familiarity), both studied items and plurality-reversed items incorrectly endorsed as old elicited a 300–500 ms mid-frontal effect (called the ‘FN400 old/new effect’ by Curran [36] and by Paller et al. [37]). This pattern is consistent with the proposal that the 400–800 ms parietal effect is linked to the recollection of specific information, whereas the 300–500 ms mid-frontal effect is linked to familiarity-driven recognition (Figure 2). Similar findings have been reported for false alarms elicited by new words sharing semantic features with studied words [38] and new pictures that were mirror-reversals of originally studied pictures [39]. The mid-frontal effect has also been reported for false alarms in a study where there was no manipulation of similarity between new and studied items [40]. As already noted, it is generally assumed that familiarity is a continuously varying signal and that positive recognition decisions are made when the strength of the signal exceeds a set criterion. Thus, the average familiarity of correctly classified new and old items will increase as the criterion becomes more conservative. Therefore, if the midfrontal effect reflects familiarity, it should be sensitive to criterion placement. This prediction was confirmed in a study by Azimian-Faridani and Wilding [41]. Relative to the ERPs elicited under a liberal response criterion, the waveforms elicited by correctly classified new and old items
254
Review
TRENDS in Cognitive Sciences
both demonstrated mid-frontal effects when subjects were instructed to respond conservatively. Woodruff et al. [32] also investigated whether the magnitude of the mid-frontal old/new effect co-varies with familiarity strength. Subjects performed a modified remember/know task in which test items that were not endorsed as remembered were rated on a four-point scale between ‘definitely old’ and ‘definitely new’ on the assumption that confidence covaried with the strength of the underlying familiarity signal. The magnitude of the mid-frontal effect varied monotonically with confidence that an item was old, but did not differ according to whether test items were endorsed as recollected or were recognized with high confidence in the absence of recollection. Conversely, the parietal old/new effect was selectively enhanced for recollected items and did not vary with confidence for items that were not recollected. Recollection as a continuous process The relationship between familiarity, recollection and confidence is a hotly debated topic that might be informed by ERP studies. According to Yonelinas and colleagues [1,6,42], recollection is best conceptualized as representing a discrete, thresholded mnemonic state (that is, recollection is in some sense all or none’), whereas others have suggested that both familiarity and recollection are continuously varying memory signals that are combined before a recognition judgment is made [11]. From the standpoint of recollection as a continuous signal, the finding that the mid-frontal effect varies with confidence [32] does not necessarily indicate a neural correlate of familiarity. Rather, an ERP effect that varies with confidence that an item is old could reflect gradation in the strength of familiarity, recollection or a signal based on some combination of the two processes. By this view, therefore, ERP effects that differentiate ‘recollected’ and ‘non-recollected’ items (exemplified by remember versus know judgments) might merely reflect differences in overall memory strength rather than in underlying memory processes. However, such an account does not readily explain the double-dissociation between mid-frontal and parietal effects observed by Woodruff et al. [32]. If the memory signal supporting a ‘remember’ response differs quantitatively, but not qualitatively, from the signal that supports ‘sub-recollective’ responding, it is unclear why the midfrontal effect should reflect memory strength for items that are not endorsed as remembered but be insensitive to whether an item is remembered or confidently judged familiar. An analogous issue arises for the parietal effect: it is unclear how the findings that the effect is elicited exclusively by items endorsed as remembered and is topographically dissociable from generic confidence effects can be reconciled with the notion that a remember response is simply a high-confidence recognition decision. In summary, although additional work is needed to examine the relationship between confidence, familiarity and recollection, the existing ERP data are difficult to reconcile with the view that the remember/know distinction merely indicates differences in memory confidence or strength. Rather, the data suggest that these two memory judgments are the outcome of qualitatively distinct memory processes. www.sciencedirect.com
Vol.11 No.6
The midfrontal effect and conceptual priming The proposal that the mid-frontal effect is a neural correlate of familiarity has recently been challenged. An hypothesis has been advanced that the effect is linked to a form of implicit memory known as conceptual priming [24,37,43]. Conceptual priming refers to a form of repetition priming that depends on repeated access to semantic, rather than perceptual, representations [37]. According to Paller and colleagues [37], most of the experimental manipulations used to isolate or modulate the neural correlates of familiarity are not ‘process pure’, and instead exert parallel effects on conceptual priming. Hence, studies employing these manipulations do not provide a sound basis for identifying a specific neural correlate of familiarity. Below, we briefly discuss two issues relevant to this alternative account. First, we discuss whether the mid-frontal effect is modulated exclusively by conceptual overlap between study and test items. And second, we ask whether the correspondences that exist between the magnitude of the effect and familiarity-driven recognition judgments are compatible with a priming account. Evidence relevant to the first of these issues comes from studies using seemingly semantically ‘empty’ items. Robust mid-frontal effects have been reported for successful recognition of pre-experimentally unfamiliar, abstract visual patterns [44,45] and unfamiliar faces [33,46]. As these items are devoid of semantic content, the findings are difficult to reconcile with the view that the mid-frontal effect depends upon a high degree of conceptual overlap between study and test items. However, not all studies using meaningless items have reported a mid-frontal effect. Yovel and Paller [24] taught subjects to associate pre-experimentally unfamiliar faces with fictitious occupations. During the subsequent test, subjects discriminated studied and unstudied faces and, for those faces judged to be old items, attempted to retrieve the occupation paired with the face or, failing that, any other details about the study episode. Faces for which no study information could be reported were considered to have been recognized on the basis of ‘pure familiarity’. Belying the proposal that it reflects familiarity-driven recognition, the ERPs elicited by these items showed no evidence of the mid-frontal effect. The generality of this finding is uncertain. Whereas one follow-up study replicated the original failure to find a midfrontal effect [47], a second study reported a robust effect for faces recognized on the basis of familiarity alone [33] (Figure 3). The explanation for these divergent findings is unclear, but it is noteworthy that recognition performance in the study where the mid-frontal effect was evident [33] was substantially higher than in the two studies where it was not [24,47]. Further research is needed to determine whether these disparities in performance reflect differences in familiarity strength, and hence whether they could account for the divergent ERP findings. Whereas the bulk of the evidence suggests that the mid-frontal effect can be elicited by meaningless items, it can be argued that the effect emerges because subjects typically imbue such items with semantic content [37]. A complementary line of evidence comes from studies that manipulated item format between the study and test in ways that preserve conceptual overlap. One such study [48]
Review
TRENDS in Cognitive Sciences
Vol.11 No.6
255
neither purely conceptual nor limited to supporting implicit memory. As noted in Box 1, there could be circumstances in which conceptual priming contributes to familiarity and, under such circumstances, an ERP correlate of conceptual priming would be expected to co-vary with recognitionmemory accuracy. However, evidence that the mid-frontal effect is not exclusively sensitive to conceptual overlap between study and test items indicates that the construct of conceptual priming is too limited to account for the data, even on the assumption that such priming invariably contributes to familiarity-driven recognition.
Figure 3. Grand-average ERPs from frontal electrodes (average of electrode clusters including F3 and F4) with the mid-frontal old/new effect indicated by the blue arrow. Subjects studied pictures of faces paired with occupation names. Recognition memory judgments were followed by an attempt to recollect the associated occupation. ERPs are plotted for faces that were correctly rejected, missed or correctly recognized (a hit). Hits were broken down according to those that were associated with correct occupation recollection (occupation hit) versus those without correct occupation recollection (familiar hit). Reproduced, with permission, from Ref. [33].
reported that the magnitude of the mid-frontal effect was not significantly affected by a shift in sensory modality (auditory to visual), although a non-significant attenuation was noted in the between-modality condition. In two other studies, format change was associated with a robust attenuation of the effect. In one case [49], the format change was between words and pictures. In the second study [44], the color of the test items (line-drawings) was changed. In light of the results of these and other studies, Groh-Bordin and colleagues [44] proposed that both conceptual and perceptual information can contribute to the familiarity signal reflected by the mid-frontal effect, but that the relative contributions of the two classes of information varies depending on the different tasks and stimuli involved. A second line of evidence relevant to Yovel and Paller’s [24] hypothesis comes from the variety of circumstances where the mid-frontal effect co-varies with behavioral performance in recognition-memory tasks rather with the study status of the test items. Thus, the effect is modulated by recognition accuracy (smaller for misses than for hits) [33,35,41,50], is not evident when ERPs elicited by studied versus unstudied words are equated for rating of familiarity strength [32], varies with response criterion [32] (see also Ref. [51]), is elicited by false alarms in addition to correctly recognized old items [36,40], and demonstrates a positive across-subjects correlation with familiarity-based recognition performance [52]. It is unclear how these findings can be reconciled with any hypothesis proposing that the mid-frontal effect reflects processes uncorrelated with those that support recognition judgments. The evidence reviewed in this section indicates that the processes reflected by the mid-frontal old/new effect are www.sciencedirect.com
Functional significance of old/new effects The evidence linking the mid-frontal and parietal old/new effects to familiarity and recollection is arguably strong; however, the cognitive operations reflected by these effects remain to be identified. In the case of the mid-frontal effect, there are grounds for thinking that the effect does not reflect familiarity directly. Tsivilis et al. [50] had subjects study visual objects superimposed upon background scenes. During the test, old and new objects were superimposed either on studied or unstudied scenes. Correctly classified old objects superimposed on studied scenes elicited a mid-frontal effect, but those paired with a novel scene did not, although a companion behavioral study revealed no difference between these two conditions in estimates of familiarity. Tsivilis et al. proposed that the mid-frontal effect reflects processes downstream from those responsible for computing the familiarity of the different elements in an episode, and that an earlier old/ new effect (onsetting 120 ms) might be a more direct reflection of the accumulation of familiarity information (see also [25,53]). A recent study [54] extended the findings of Tsivilis et al. by demonstrating that a test item superimposed on a novel background elicits a mid-frontal effect when attention is focused exclusively on the item, but not when both item and background are attended to. Thus, the effect reflects attentionally mediated processing of multiple sources of familiarity information. The functional significance of the parietal old/new effect is equally obscure. One early suggestion was that the effect reflects processes that contribute to the representation of recollected information [26]. Alternatively, the effect might index attentional orienting to recollected information [55,56], rather than processes supporting its representation or maintenance. It has recently been argued [53] that findings indicating that the effect varies according to the amount of information recollected [53,57] are more consistent with the first of these two proposals. Neural generators of the mid-frontal and parietal old/new effects Single-neuron studies in non-human primates and human fMRI studies have identified putative familiarity-sensitive neuronal populations in the perirhinal region of the medial temporal lobe [58,59]. However, the onset latency of these neuronal responses is of the order of 90 ms [59], which is considerably earlier than the onset of the mid-frontal ERP effect. Familiarity-sensitive neuronal responses have also been identified in the monkey in a variety of prefrontal regions [60], with onset latencies (at 250 ms) fitting well
256
Review
TRENDS in Cognitive Sciences
Box 3. Questions for future research Can ERP data help adjudicate between threshold and continuous models of recollection? What cognitive operations are directly reflected by the mid-frontal and parietal old/new effects? What is the relationship between the mid-frontal old/new effect and earlier onsetting effects that also seem to track familiarity? How is memory-related fMRI activity that originates in pre-frontal and parietal cortices related to the mid-frontal and parietal ERP old/new effects?
with the time-course of the mid-frontal ERP effect. Together with fMRI evidence that the strength of familiarity modulates lateral prefrontal activity in humans [21], these findings suggest that the mid-frontal effect originates from one or more regions of the prefrontal cortex. It will be of considerable interest to determine how closely the functional characteristics of familiarity-sensitive prefrontal activity, as indexed by fMRI, match the characteristics of the ERP effect. The characteristic scalp distribution of the parietal old/ new effect (Figure 1) suggests that it might reflect neural activity generated in the lateral parietal cortex. fMRI findings of recollection-sensitive activity in this region (reviewed in Refs [55,56]) are consistent with this suggestion. Findings of direct functional parallels between these fMRI and ERP old/new effects [21,32,34,53,61,62] give additional credence to the hypothesis that the ERP effect reflects activity in a recollection-sensitive region of the lateral parietal cortex. Concluding comments ERP studies of recognition memory provided some of the first evidence that recollection and familiarity have qualitatively distinct neural correlates. More recent ERP findings have added to this evidence and to the general weight of the evidence supporting dual-process models of recognition memory. Among the challenges for the future (Box 3) are the elucidation of the cognitive operations reflected by these ERP effects and the identification of their neural generators. Acknowledgements Research described in this article was supported by National Institutes of Health grants MH072966 (M.D.R.) and MH64812 (T.C.).
References 1 Yonelinas, A.P. (2002) The nature of recollection and familiarity: a review of 30 years of research. J. Mem. Lang. 46, 441–517 2 Rugg, M.D. and Yonelinas, A.P. (2003) Human recognition memory: a cognitive neuroscience perspective. Trends Cogn. Sci. 7, 313–319 3 Dunn, J.C. (2004) Remember-know: a matter of confidence. Psychol. Rev. 111, 524–542 4 Donaldson, W. (1996) The role of decision processes in remembering and knowing. Mem. Cognit. 24, 523–533 5 Heathcote, A. et al. (2006) Recollection and familiarity in recognition memory: evidence from ROC curves. J. Mem. Lang, DOI: 10.1016/j.jml. 2006.1007.1001 6 Yonelinas, A.P. (2001) Components of episodic memory: the contribution of recollection and familiarity. Philos. Trans. R. Soc. Lond. B Biol. Sci. 356, 1363–1374 7 Norman, K.A. and O’Reilly, R.C. (2003) Modeling hippocampal and neocortical contributions to recognition memory: a complementary learning systems approach. Psychol. Rev. 110, 611–646 www.sciencedirect.com
Vol.11 No.6
8 Mandler, G. (1980) Recognizing: the judgment of previous occurrence. Psychol. Rev. 87, 252–271 9 Diana, R.A. et al. (2006) Models of recognition: a review of arguments in favor of a dual-process account. Psychon. Bull. Rev. 13, 1–21 10 Rotello, C.M. et al. (2004) Sum-difference theory of remembering and knowing: a two-dimensional signal-detection model. Psychol. Rev. 111, 588–616 11 Wixted, J.T. (2007) Dual-process theory and signal-detection theory of recognition memory. Psychol. Rev. 114, 152–176 12 Tulving, E. (1985) Memory and consciousness. Can. Psychol. 26, 1–12 13 Heathcote, A. (2003) Item recognition memory and the receiver operating characteristic. J. Exp. Psychol. Learn. Mem. Cogn. 29, 1210–1230 14 Hilford, A. et al. (2002) Regularities of source recognition: ROC analysis. J. Exp. Psychol. Gen. 131, 494–510 15 Aggleton, J.P. et al. (2005) Sparing of the familiarity component of recognition memory in a patient with hippocampal pathology. Neuropsychologia 43, 1810–1823 16 Holdstock, J.S. et al. (2002) Under what conditions is recognition spared relative to recall after selective hippocampal damage in humans? Hippocampus 12, 341–351 17 Manns, J.R. et al. (2003) Recognition memory and the human hippocampus. Neuron 37, 171–180 18 Wais, P.E. et al. (2006) The hippocampus supports both the recollection and the familiarity components of recognition memory. Neuron 49, 459–466 19 Davachi, L. et al. (2003) Multiple routes to memory: distinct medial temporal lobe processes build item and source memories. Proc. Natl. Acad. Sci. U. S. A. 100, 2157–2162 20 Eldridge, L.L. (2000) Remembering episodes: a selective role for the hippocampus during retrieval. Nat. Neurosci. 3, 1149–1152 21 Yonelinas, A.P. et al. (2005) Separating the brain regions involved in recollection and familiarity in recognition memory. J. Neurosci. 25, 3002–3008 22 Sanquist, T.F. et al. (1980) Electrocortical signs of levels of processing: perceptual analysis and recognition memory. Psychophysiology 17, 568–576 23 Warren, L.R. (1980) Evoked-potential correlates of recognition memory. Biol. Psychol. 11, 21–35 24 Yovel, G. and Paller, K.A. (2004) The neural basis of the butcher-onthe-bus phenomenon: when a face seems familiar but is not remembered. Neuroimage 21, 789–800 25 Duarte, A. et al. (2004) Dissociable neural correlates for familiarity and recollection during the encoding and retrieval of pictures.. Brain Res. Cogn. Brain Res. 18, 255–272 26 Wilding, E.L. and Rugg, M.D. (1996) An event-related potential study of recognition memory with and without retrieval of source. Brain 119, 889–905 27 Senkfor, A.J. and Van Petten, C. (1998) Who said what? An eventrelated potential investigation of source and item memory. J. Exp. Psychol. Learn. Mem. Cogn. 24, 1005–1025 28 Curran, T. (2004) Effects of attention and confidence on the hypothesized ERP correlates of recollection and familiarity. Neuropsychologia 42, 1088–1106 29 Du¨zel, E. et al. (1997) Event-related potential correlates of two states of conscious awareness in memory. Proc. Natl. Acad. Sci. U. S. A. 94, 5973–5978 30 Smith, M.E. (1993) Neurophysiological manifestations of recollective experience during recognition memory judgments. J. Cogn. Neurosci. 5, 1–13 31 Spencer, K.M. et al. (2000) On the search for the neurophysiological manifestation of recollective experience. Psychophysiology 37, 494– 506 32 Woodruff, C.C. et al. (2006) Electrophysiological dissociation of the neural correlates of recollection and familiarity. Brain Res. 1100, 125– 135 33 Curran, T. and Hancock, J. (2007) The FN400 indexes familiaritybased recognition of faces. Neuroimage 36, 464–471 34 Herron, J.E. et al. (2004) Probability effects on the neural correlates of retrieval success: an fMRI study. Neuroimage 21, 302–310 35 Rugg, M.D. et al. (1998) Dissociation of the neural correlates of implicit and explicit memory. Nature 392, 595–598
Review
TRENDS in Cognitive Sciences
36 Curran, T. (2000) Brain potentials of recollection and familiarity. Mem. Cognit. 28, 923–938 37 Paller, K.A. et al. (2007) Validating neural correlates of familiarity. Trends Cogn. Sci 11, 243–250 38 Nessler, D. et al. (2001) Event related brain potentials and illusory memories: the effects of differential encoding. Brain Res. Cogn. Brain Res. 10, 283–301 39 Curran, T. and Cleary, A.M. (2003) Using ERPs to dissociate recollection from familiarity in picture recognition. Brain Res. Cogn. Brain Res. 15, 191–205 40 Wolk, D.A. et al. (2006) ERP correlates of recognition memory: effects of retention interval and false alarms. Brain Res. 1096, 148–162 41 Azimian-Faridani, N. and Wilding, E.L. (2006) The influence of criterion shifts on electrophysiological correlates of recognition memory. J. Cogn. Neurosci. 18, 1075–1086 42 Parks, C.M. and Yonelinas, A.P. (2007) Moving beyond pure signaldetection models: comment on Wixted (2007). Psychol. Rev. 114, 188– 202 43 Voss, J.L. and Paller, K.A. (2006) Fluent conceptual processing and explicit memory for faces are electrophysiologically distinct. J. Neurosci. 26, 926–933 44 Groh-Bordin, C. et al. (2006) Has the butcher on the bus dyed his hair? When color changes modulate ERP correlates of familiarity and recollection. Neuroimage 32, 1879–1890 45 Curran, T. et al. (2002) An electrophysiological comparison of visual categorization and recognition memory. Cogn. Affect. Behav. Neurosci. 2, 1–18 46 Nessler, D. et al. (2005) Perceptual fluency, semantic familiarity and recognition-related familiarity: an electrophysiological exploration. Brain Res. Cogn. Brain Res. 22, 265–288 47 MacKenzie, G. and Donaldson, D.I. (2007) Dissociating recollection from familiarity: electrophysiological evidence that familiarity for faces is associated with a posterior old/new effect. Neuroimage 36, 454–463 48 Curran, T. and Dien, J. (2003) Differentiating amodal familiarity from modality-specific memory processes: an ERP study. Psychophysiology 40, 979–988 49 Schloerscheidt, A.M. and Rugg, M.D. (2004) The impact of change in stimulus format on the electrophysiological indices of recognition. Neuropsychologia 42, 451–466 50 Tsivilis, D. et al. (2001) Context effects on the neural correlates of recognition memory: an electrophysiological study. Neuron 31, 497–505 51 Bridson, N.C. et al. (2006) Electrophysiological correlates of familiarity in recognition memory and exclusion tasks. Brain Res. 1114, 149–160 52 Curran, T. et al. (2006) Combined pharmacological and electrophysiological dissociation of familiarity and recollection. J. Neurosci. 26, 1979–1985
Vol.11 No.6
53 Vilberg, K.L. et al. (2006) The relationship between electrophysiological correlates of recollection and amount of information retrieved. Brain Res. 1122, 161–170 54 Ecker, U.K. et al. (2007) Context effects on familiarity are familiarity effects of context – an electrophysiological study. Int. J. Psychophysiol 64, 146–156 55 Rugg, M.D. and Henson, R.N.A. (2002) Episodic memory retrieval: an (event-related) functional neuroimaging perspective. In The Cognitive Neuroscience of Memory Encoding and Retrieval (Parker, A.E. et al., eds), pp. 3–38, Psychology Press 56 Wagner, A.D. et al. (2005) Parietal lobe contributions to episodic memory retrieval. Trends Cogn. Sci. 9, 445–453 57 Wilding, E.L. (2000) In what way does the parietal ERP old/new effect index recollection? Int. J. Psychophysiol. 35, 81–87 58 Henson, R.N. et al. (2003) A familiarity signal in human anterior medial temporal cortex? Hippocampus 13, 301–304 59 Xiang, J.Z. and Brown, M.W. (1998) Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe. Neuropharmacology 37, 657–676 60 Xiang, J.Z. and Brown, M.W. (2004) Neuronal responses related to long-term recognition memory processes in prefrontal cortex. Neuron 42, 817–829 61 Herron, J.E. et al. (2003) Probability effects on event-related potential correlates of recognition memory. Brain Res. Cogn. Brain Res. 16, 66–73 62 Vilberg, K.L. and Rugg, M.D. (2007) Dissociation of the neural correlates of recognition memory according to familiarity, recollection, and amount of recollected information. Neuropsychologia 45, 2216– 2225 63 Shiffrin, R.M. and Steyvers, M. (1997) A model of recognition memory: REM—retrieving effectively from memory. Psychol. Bull. Rev. 4, 145– 166 64 Hintzman, D.L. (1988) Judgments of frequency and recognition memory in a multiple-trace memory model. Psychol. Rev. 95, 528– 551 65 Jacoby, L.L. and Dallas, M. (1981) On the relationship between autobiographical memory and perceptual learning. J. Exp. Psychol. Gen. 110, 306–340 66 Whittlesea, B.W.A. et al. (1990) Illusions of immediate memory: evidence of an attributional basis for feelings of familiarity and perceptual quality. J. Mem. Lang. 29, 716–732 67 Whittlesea, B.W.A. and Price, J.R. (2001) Implicit/explicit memory versus analytic/nonanalytic processing: rethinking the mere exposure effect. Mem. Cognit. 29, 234–246 68 Rugg, M.D. and Coles, M.G.H. (1995) The ERP and cognitive psychology: conceptual issues. In Electrophysiology of Mind (Rugg, M.D. and Coles, M.G.H., eds), pp. 27–39, Oxford University Press
The ScienceDirect collection ScienceDirect’s extensive and unique full-text collection covers more than 1900 journals, including titles such as The Lancet, Cell, Tetrahedron and the full suite of Trends, Current Opinion and Drug Discovery Today journals. With ScienceDirect, the research process is enhanced with unsurpassed searching and linking functionality, all on a single, intuitive interface. The rapid growth of the ScienceDirect collection is a result of the integration of several prestigious publications and the ongoing addition to the Backfiles – heritage collections in a number of disciplines. The latest step in this ambitious project to digitize all of Elsevier’s journals back to volume one, issue one, is the addition of the highly cited Cell Press journal collection on ScienceDirect. Also available online for the first time are six Cell titles’ long-awaited Backfiles, containing more than 12,000 articles that highlight important historic developments in the field of life sciences.
For more information, visit www.sciencedirect.com www.sciencedirect.com
257