Temporal texture of associative encoding modulates recall processes

Brain and Cognition 84 (2014) 1–13

Contents lists available at ScienceDirect

Brain and Cognition journal homepage: www.elsevier.com/locate/b&c

Temporal texture of associative encoding modulates recall processes Roni Tibon, Daniel A. Levy ⇑ School of Psychology and Unit for Applied Neuroscience, The Interdisciplinary Center, Herzliya 46683, Israel

a r t i c l e

i n f o

Article history: Accepted 16 October 2013 Available online 8 November 2013 Keywords: Cued recall Familiarity Temporal Association EEG ERP Recollection

a b s t r a c t Binding aspects of an experience that are distributed over time is an important element of episodic memory. In the current study, we examined how the temporal complexity of an experience may govern the processes required for its retrieval. We recorded event-related potentials during episodic cued recall following pair associate learning of concurrently and sequentially presented object-picture pairs. Cued recall success effects over anterior and posterior areas were apparent in several time windows. In anterior locations, these recall success effects were similar for concurrently and sequentially encoded pairs. However, in posterior sites clustered over parietal scalp the effect was larger for the retrieval of sequentially encoded pairs. We suggest that anterior aspects of the mid-latency recall success effects may reflect working-with-memory operations or direct access recall processes, while more posterior aspects reflect recollective processes which are required for retrieval of episodes of greater temporal complexity. Ó 2013 Elsevier Inc. All rights reserved.

1. Introduction Events that we may remember are often richly textured, comprising multiple objects and persons, generally dispersed across numerous spatial locations, and importantly – distributed over time. Understanding how the brain represents and retrieves such temporal structure is a major research goal. Binding experiences over time is important for episodic memories, which are commonly organized by their sequence of occurrence (Tulving & Markowitsch, 1998). Studies regarding the neural substrates of time representation in human memory most commonly address our capacity to remember the relative recency or order of experienced events. That research has identified prefrontal (Cabeza et al., 1997; Dobbins, Rice, Wagner, & Schacter, 2003; Fujii et al., 2004; Hayes, Ryan, Schnyer, & Nadel, 2004; Nyberg et al., 1996; Suzuki et al., 2002), extra-hippocampal medial temporal lobe (Dudukovic & Wagner, 2007; Jenkins & Ranganath, 2010; Rekkas et al., 2005; St. Jacques, Rubin, LaBar, & Cabeza, 2008) and more recently, hippocampal (Hales & Brewer, 2010; Lehn et al., 2009; Tubridy & Davachi, 2011; cf. the primate study of Naya & Suzuki, 2011) and even parietal (Kimura et al., 2010) contributions to that type of memory. However, memory formation over time is required for a much wider range of competences than just temporal order judgments. Episodic associations among objects (and between objects and contexts) enable constructive retrieval of an experience, as presentation of one item as a cue may enable the recall of other items experienced in conjunction with it. Importantly, such ⇑ Corresponding author. E-mail address: [email protected] (D.A. Levy). 0278-2626/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.bandc.2013.10.003

reconstructive processes may link not only objects that were experienced concurrently, but also items that were experienced sequentially. The processes that enable binding objects experienced over time to yield associations, and the use of such associations in active retrieval, still requires explication. In the current study we examine how the temporal complexity of an experience may govern the processes required for its being remembered. We investigated one component process of the retrieval of episodic associations, the cued recall of pair associates (in this case object pictures), which were experienced either concurrently or sequentially. Episodic cued recall involves reconstructing an event to retrieve a memory target given its associated probe. We surmised that more ramified retrieval processes would be required to reconstruct associations having greater temporal complexity due to sequential stimulus presentation, relative to associations between stimuli experienced concurrently. We further predicted that this process difference would modulate brain activity over the time course of retrieval processes. We therefore employed EEG to track differences in cued recall success effects between concurrently and sequentially encoded object picture pairs. While scalp-recorded EEG is not reliably sensitive to medial temporal lobe activity, it seemingly does reflect patterns of retrieval-related cortical activity, with high temporal resolution. Several event-related potential (ERP) components derived from EEG recordings associated with retrieval success have been reported, including the FN400/ early mid-frontal effect (in which the absence of retrieval, either for novel foils probes in a recognition test or for recall failure in a cued recall test, yields more negative deflections than successful retrievals); the late positive component (LPC), exhibited most strongly over parietal scalp, associated with recollective success;

2

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

and later frontal deflections associated with additional mnemonic processes such as monitoring (Friedman & Johnson, 2000; Mecklinger, 2000; Rugg & Curran, 2007; Wilding & Ranganath, 2011). The LPC component, which is generally considered to reflect parietal lobe mnemonic processes also evidenced in hemodynamic studies (Levy, 2012; Rugg & Curran, 2007; Wagner, Shannon, Kahn, & Buckner, 2005), is reported to be sensitive to recollective complexity. That is, to the extent that the retrieved memory is one that is dependent on relational and associative hippocampus-dependent processes, it will elicit a stronger LPC component. Accordingly, the LPC should be modulated by types of temporal complexity for which hippocampal binding is important. A basis for these predictions regarding modulation of these components by temporal factors may be found in the report of Grove and Wilding (2009) that modulation of the FN400 and LPC components predicted accuracy in judgments of recency in a recognition memory test. Thus, explicit reconstruction of temporal order affected recognition-related ERP components. It remains to be determined whether differences in temporal structure of encoded events in a cued recall paradigm, in which temporal complexity is manipulated, leads to differential modulation of those components, even when memory for temporal order is not explicitly examined. Such a finding would indicate that temporal complexity of an encoding episode might influence the retrieval processes needed to recall component representations. An additional goal of the current study is to further characterize the ERP correlates of associative memory expressed by cued recall success. While numerous studies have documented ERP patterns associated with recognition success, examination of the time course of cued recall is much less common. Arguably, earlier studies either did not examine associative recall (as cues employed were word stems or word fragments), or used paradigms in which effects of cue familiarity and associative recall were confounded. Accordingly, in contrast to earlier recognition-then-cued recall paradigms (Donaldson & Rugg, 1998; Donaldson & Rugg, 1999), in the present study, there was no explicit demand to perform recognition judgments. Furthermore, all cues were stipulated to be old, and expected by participants to be old. This paradigm therefore enabled us to explore the time course of associative cued recall minimally confounded by recognition processes. In this study we expand on our earlier examination of ERP correlates of cued recall following concurrent unimodal and crossmodal pair associate learning (Tibon & Levy, 2013) by using a temporal manipulation rather than a modality manipulation, with the goal of determining whether those findings regarding cued recall ERPs can be generalized across encoding factors. Finally, this cued recall paradigm allows examination of hypotheses that have been advanced regarding the nature of retrieval-related posterior parietal activations observed primarily in studies of recognition memory (recently reviewed by Kim, 2011; Levy, 2012). While much recent work has focused on hemodynamic studies of parietal mnemonic effects, it is generally asserted that parietal-maximal ERP retrieval success effects should be attributed to the same underlying processes as indexed by fMRI (e.g., Rugg & Curran, 2007; Vilberg & Rugg, 2009; Wilding & Ranganath, 2011). One suggestion is that parietal activations at retrieval reflect the participation of the angular gyrus in multimodal representation (Shimamura, 2011). If that is the case, parietal ERPs should not differ between concurrent and sequential conditions, which were both comprised of unimodal visually presented stimuli. Another suggestion is that parietal activations reflect a post-retrieval buffer (Guerin & Miller, 2011; Haramati, Soroker, Dudai, & Levy, 2008; Rugg & Wilding, 2000; Vilberg & Rugg, 2007; Vilberg & Rugg, 2008; Vilberg & Rugg, 2009). Such an account is most tenable if the ERP component associated with parietal processes follows an earlier component associated with a process reflecting

recall success. According to the attentional process account (e.g., Cabeza, Ciaramelli, & Moscovitch, 2012; Ciaramelli, Grady, Levine, Ween, & Moscovitch, 2010), parietal ERP components should be found for cued recall, and may reflect either pre-retrieval attentional focus or post-retrieval attentional capture. In contrast, accounts proposing that parietal mnemonic activations reflect signal accumulation in the service of a recognition judgment (e.g., Donaldson, Wheeler, & Petersen, 2010; Wagner et al., 2005), or expectations regarding mnemonic status of a probe and their violations (Buchsbaum, Ye, & D’Esposito, 2011; O’Connor, Han, & Dobbins, 2010) would be challenged by the emergence of parietal ERP effects for cued recall, in which those processes have no substrate on which to act.

2. Methods 2.1. Participants Participants were 35 healthy right handed (all scored positively in the Edinburgh Handedness Inventory; Oldfield, 1971) young adults (22 females; mean age 22.2, SD = 3.11 years, range 18–33), with normal or adjusted-to-normal vision. All were undergraduate students who volunteered in return for academic requirement credit or payment. Informed consent was obtained from all participants for a protocol approved by the Interdisciplinary Center’s Institutional Review Board. Two participants were excluded from the analyses due to computer failure during the experiment, leaving us with 33 subjects whose data was analyzed. 2.2. Materials Stimuli were 480 color drawings (240 pairs) of common objects obtained from various internet sources, including fruits and vegetables, tools, sporting goods, electrical and electronic devices, animals, furniture, and clothing, each approximately 6–8 cm in on-screen size. To form the various experimental conditions, several stimulus lists were created. Overall there were four lists of 120 object pictures. Two lists of 120 pictures served as recall cues. Two additional lists containing 120 pictures served as recall targets. Each entry in the cueing lists was pseudo-randomly assigned to an entry in the target lists. List pairing was counterbalanced across participants, and lists were counterbalanced across conditions, such that each object was equally presented as cue and as target, and each pair was equally presented in concurrent and sequential conditions. The domains of the to-be-associated entries from the cue and target lists were always different (e.g., a picture of a cat from the target list was never matched with a picture of a dog from the cue list). This pair construction process was intended to minimize the degree of pre-existing semantic associative strength between pair members, such that the association to be generated by the participant would constitute a discrete event, leading to a subsequent episodic memory. 2.3. Task procedure The experiment consisted of two sessions that took place in two separate days, with 3–7 days interval between the sessions. EEG recording was conducted over two sessions, as initial feedback indicated that the duration of the experiment in a single session led to participant reports of mental exhaustion and a notable dropoff in performance. In both sessions, participants performed alternating concurrent and sequential blocks. Since there were 8 sequential blocks, but only 6 concurrent blocks (see below), the day always started with the former, thus, in each session

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

participants performed 4 sequential blocks and 3 concurrent blocks, interleaved. Participants were tested individually in a quiet room. Upon arrival at the lab, they signed an informed consent form and filled out the Edinburgh Handedness Inventory (Oldfield, 1971). Following EEG electrode cap preparation (described below), participants were seated at a distance of 70 cm from a computer monitor. Participants were then told that they would be presented with pairs of pictures, and were instructed to remember those pairs. While viewing the pairs, they were instructed to perform two tasks: (1) to decide which object they like better, and to (2) judge which object is more common in their everyday life. They were further told that after each study block, a test phase would ensue, in which cue pictures would be presented alone and they would need to recall and use the keyboard to type the name of the object portrayed in the picture that had accompanied that cue. During the test phase, participants were asked to relax and to avoid eye movements and blinks as much as possible. Pilot data indicated that performance in the concurrent condition was superior to performance in the sequential condition. We therefore employed different block lengths, and presented the liking judgment twice in the sequential condition, in order to match difficulty levels; behavioral results (see below) indicated that this manipulation did indeed equate cued recall success across conditions. The participants viewed six blocks of 20 stimulus pairs each in the concurrent condition, and eight blocks of 15 stimulus pairs each in the sequential condition. During the encoding phase, in the concurrent condition, stimulus pairs were presented adjacently on a gray background for 1 s. In the sequential condition, stimuli were presented centrally one after the other, with each stimulus shown for 1000 ms, and with a 250 ms scrambled colored patch (serving as a mask) presented between the pictures. The presentation of the stimuli was followed by a 300 ms blank screen. This was followed by a screen with the legend ‘‘Which do you like better?’’ in likeness judgment blocks and ‘‘which is more common’’ in commonness judgment blocks, to which participants were instructed to respond by blue and yellow color-marked keys on the right side of a standard keyboard. This triggered a 500 ms visual fixation cross, followed by the next stimuli pair. After all the pairs in the block had been presented twice (in the concurrent condition; once for a liking judgment and once for a commonness judgment) or three times (in the sequential condition; twice for a liking judgment and once for a commonness judgment), the retrieval phase started. In this phase, the cue picture was presented alone. Participants were asked to respond by pressing a green key (if they could recall its paired object) or a red key (if they could not). All participants pressed these keys using their right hand. After key press, a 700 ms blank screen appeared, followed by a screen with the legend ‘‘Pair Associate’’, to which participants were instructed to respond by typing the correct answer and pressing the ‘‘Enter’’ key. This triggered a 800 ms visual fixation cross, and a 700 ms blank screen, followed by the next cue stimulus. After responding to all the cue pictures of the block, the participant was offered a selfpaced rest break of several minutes duration, followed by the next study-test block. A practice block of 3 trials was provided for each encoding task and for each condition. During this practice session, the experimenter verbally confirmed that the participants understood the tasks they were to perform. 2.4. Electrophysiological recording parameters and data processing 2.4.1. EEG recordings The EEG was recorded using the Active II system (BioSemi, The Netherlands) from 64 electrodes mounted in an elastic cap according to the extended 10–20 system. EOG (electro-occulogram) was recorded using four additional external electrodes, located above

3

and below the right eye, and on the outer canthi of both eyes. One electrode was placed on the tip of the nose. Two additional electrodes were placed over the left and right mastoid bones. The ground function during recording was provided by common-mode signal (CMS) and direct right leg (DRL) electrodes forming a feedback loop, placed over parieto-occipital scalp. The online filter settings of the EEG amplifiers were 0.16–100 Hz. Both EEG and EOG were continuously sampled at 512 Hz and stored for offline analysis. 2.4.2. Preprocessing Using the Fieldtrip toolbox for Matlab (Oostenveld, Fries, Maris, & Schoffelen, 2011), stimulus-locked ERPs were segmented into epochs starting 200 ms before cue presentation and up to 1000 ms afterwards. EEG and EOG channels were then re-referenced to the average of the left and right mastoid channels, band-pass filtered with an offline cutoff of 0.1–30 Hz, and baseline-adjusted by subtracting the mean amplitude of the pre-stimulus period (200 ms) of each trial from all the data points in the segment. ICA was employed separately for each session, in order to remove eye movements and blink artifacts (Makeig et al., 1999). Additional trials containing electrode pop artifacts, resulting from a sudden change in the electrical potential between the electrode and the scalp, and muscle artifacts were rejected visually. Channels depicting drifts and other artifacts in individual trials were replaced with interpolated data from adjacent electrodes. Outlier trials, in which response time was more than 3 SDs above or below the mean response time of the subject in the condition, were excluded from computations of ERPs. 3. Results 3.1. Behavioral measures Recall accuracy rates and reaction times (RTs) are shown in Table 1. These were calculated for each of the following recall conditions: (1) Success: participants pressed the green key, thus indicating they remembered the paired probe, and then correctly recalled the probe; (2) Failure-False Alarm (FA): participants pressed the green key, but were mistaken in the pair member they provided, and (3) Failure-Miss: participants pressed either the green or red key, but provided no pair member name and proceeded to the next trial. Trials in which participants initially indicated that they could not recall the paired probe but provided the correct pair member, and RT outliers (3 SDs above or below the participant’s average in each condition) were removed from analysis. We examined differences between conditions in recall success with repeated measures ANOVAs, using recall success (success, failure-FA, failure-miss) and condition (concurrent, sequential) as repeated factors. We found a significant effect of recall success, F(1.14,36.4) = 36.01, p < .001, with success = failuremiss > failure-FA, in both conditions. No other significant effects or interactions were found. For RTs, the analysis also revealed a significant effect of recall success, F(1.6,48.1) = 15.12, p < .001, with failure-miss = failure-FA > success, in both conditions. Importantly, the behavioral analyses revealed that the conditions did not differ with regards to accuracy and reaction times. This indicates that our use of different list lengths for concurrent and sequential conditions was successful in reducing the difficulty differences between those conditions which we had found in pilot studies. Accordingly, ERP differences between conditions cannot be attributed solely to difficulty differences. In order to address the possibility that ERP retrieval success effects reflected not recall but rather cue familiarity, we conducted an ancillary behavioral experiment, with a different set of

4

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

Table 1 Mean performance indices (accuracy and RTs) for the cued recall conditions. Concurrent

% Success Range of trials used for ERP analyses Average # of trials in ERP analyses RTs (ms)

Sequential

Success

Failure-FA

Failure-Miss

Success

Failure-FA

Failure-Miss

42.17 (3.35) 10–85 42.67 (3.31) 1413 (109)

7.78 (1.01) Not included in ERP analyses

47.85 (3.28) 12–87 49.37 (3.49) 2532 (281)

42 (4.03) 1–92 43.89 (4.27) 1559 (143)

8.7 (1.22) Not included in ERP analyses

46.8 (3.76) 10–99 48.19 (3.95) 2670 (324)

2300 (242)

2583 (282)

Note: Standard error is given in parentheses. Here, and in all other statistics, Greenhouse–Geisser correction was applied as appropriate.

comparable participants (n = 15). In this experiment, in each block, participants studied lists of 20 picture pairs, using the same encoding procedure that was used in the concurrent condition. At retrieval, the 20 studied pictures were intermixed with 20 new pictures. Participants were asked to press one of the five buttons on the keyboard according to whether, and how well, they remembered that picture as having been presented in the study phase, by choosing between ‘‘definitely not’’, ‘‘probably not’’, ‘‘don’t know’’, ‘‘probably yes’’, ‘‘definitely yes’’. The task comprised of six encoding-retrieval blocks. For old items, ‘‘definitely yes’’ and ‘‘probably yes’’ responses were classified as hits, and for new items, ‘‘definitely not’’ and ‘‘probably not’’ responses were classified as correct rejections. The average hit rate was 95.3%. Of those hit responses, 98.4% were ‘‘definitely yes’’ responses, and the remaining 1.4% were ‘‘probably yes’’ responses. The average correct rejection rate was 96.3%. Of those correct rejection responses, 95.8% were ‘‘definitely not’’ responses, and the remaining 4.2% were ‘‘probably not’’ responses. These results suggest that the study procedure employed in the main experiment results in almost all cues being recognized with high confidence – i.e., almost all would be regarded as familiar. As we will argue below, this is one of the reasons that it is unlikely that ERP cued recall effects result from differences in the familiarity signal elicited by cues leading to successful recall. 3.2. ERP analyses and results 3.2.1. Data segmentation Trials were averaged to compute four ERP waveforms: (1) Concurrent – recall success; (2) Concurrent – recall failure; (3) Sequential – recall success; and (4) Sequential – recall failure. Due to a negligible number of trials, FA trials were omitted from the ERP analyses. We initially examined the same time windows and locations used in our previous ERP study of cued recall (Tibon & Levy, 2013), in order to allow comparison between previous and current data. These initial analyses were conducted on four time windows (0–200 ms, 200–350 ms, 350–600 ms, and 600–1000 ms), and nine electrode clusters (left anterior – Fp1, AF3, F1, F3, F5; mid anterior – Fpz, AFz, Fz; right anterior – Fp2, AF4, F2, F4, F6; left central – FC1, FC3, FC5, C1, C3, C5; mid central – FCz, Cz; right central – FC2, FC4, FC6, C2, C4, C6; left posterior – CP1, CP3, CP5, P1, P3, P5, PO3; mid-posterior CPz, Pz,POz; and right posterior – CP2, CP4, CP6, P2, P4, P6, PO4) locations, see Fig. 1 for topographical distribution). In the present study, as in the prior one, these selections reflect the latency windows distinguished by ERP component morphology across conditions, and allow extensive inspection of anteriority and laterality factors affecting the electrophysiological expressions of recall success in scalp-recorded EEG (see Fig. 2). 3.2.2. Mixed-effects models analysis To analyze the ERP data, we used a linear mixed-effects models approach, which can take subject-specific variability into account in modeling effects, and accommodate the repeated measures study design. Such models can be considered a generalization of ANOVA, but use maximum likelihood estimation instead of sum

Fig. 1. Topographical distribution of the nine electrodes clusters, covering left anterior (LA), mid anterior (MA), right anterior (RA), left central (LC), mid central (MC), right central (RC), left posterior (LP), mid posterior (MP), and right posterior (RP) locations.

of squares decomposition. An advantage of such an approach over standard repeated measures ANOVA is that mixed-effects models are better suited for complex designs (e.g., Bagiella, Sloan, & Heitjan, 2000). Moreover, such an approach is particularly recommended for unbalanced data (an unequal number of trials in each condition, which we have here due to the post hoc division of trials into success and failure bins). Such models are considered ‘‘mixed’’ as they include two types of statistical effects: (1) fixed effects for which data has been gathered from all levels of the factor of interest, and (2) random effects assumed to be uncorrelated with the independent variables. In our case, inter-individual differences in EEG amplitude dynamics are modeled as a random intercept, which represents an individual ‘‘baseline,’’ in addition to being affected by the fixed factors. The fixed part of the model includes the condition factor (concurrent, sequential), the recall success factor (success, failure), and two spatial factors: anteriority (anterior, central, and posterior) and laterality (left, midline, and right). The fixed part of the model further included all the possible interactions between these four fixed factors. In this mode of analysis, each observation serves as an element of analysis to be modeled; degrees of freedom represent the number of observations and not the number of participants as in customary in grand average ANOVA. These parameters result in increased degrees of freedom compared to traditional designs. Although at first glance this might appear to be an overly liberal approach, as will be shown below, in this approach large inter-observation variance is not tempered by

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

5

Fig. 2. ERP analyses. Grand-average ERP waveforms elicited by recall success and failure trials in (A) the concurrent encoding condition and (B) the sequential encoding condition. Data are shown for the nine electrodes clusters used in the statistical analyses. Shadings mark the four time windows used for statistical analyses.

6

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

Table 2 Outcomes of mixed-effects models analysis. Epoch (in ms)

Recall success Condition Anteriority Laterality Recall success  condition Recall success  laterality Recall success  anteriority Condition  anteriority Laterality  anteriority Recall success  condition  anteriority * **

0–200

200–350

350–600

600–1000

15.41*** 8.63** 44.17*** 4.79**

99.1***

142.25*** 6.52* 635.59*** 26.91***

8.07***

25.36*** 7.71*** 5.57***

87.23*** 6.96** 25.92*** 13.46*** 3.85* 4.67** 6.69**

448.21*** 47.56***

6.54** 14.89*** 5.22*** 3.63*

4.09**

p < .05. p < .01. p < .001.

***

Fig. 3. Topographic maps showing the scalp distributions of the recall success differences (successful minus unsuccessful trials mean amplitudes) for each condition in the four time windows used in the statistical analyses.

averaging within participants, which limits the number of effects that emerge as significant. Furthermore, effects that do emerge from the statistical analyses are reflected by robust differences in mean amplitudes. Model parameters were estimated with the nlme package of the software R (Pinheiro, Bates, DebRoy, Sarkar, & the R Core Team, 2007; freely available at http://www.R-project.org). We ran this analysis separately for each time window. Significant results (p < .05 after a maximum of 1000 iterations, with a convergence criterion of 1e 6) are shown in Table 2. 3.2.3. Follow-up analyses Significant differences arising from interactions that included the recall success factor were subjected to post hoc analyses, which were performed separately for each time window. Since these analysis follow in the wake of the mixed-effects models analysis performed across individual trials, post hoc tests were also conducted across individual trials, using the same procedure that was used in the main analysis; thus, all post hoc analyses included participant as a random factor. The large inter-trial variance untempered by inter-subject averaging is balanced by increases in the number of degrees of freedom. The topographic maps of the recall success effects (successful minus unsuccessful trials mean ampli-

tudes) are shown in Fig. 3. For the first time window (0–200 ms), analysis revealed that in both conditions, successful recall trials elicited more positive deflections than unsuccessful trials, but only in anterior sites, F(1,17894) = 15.24, p < .001. This anterior recall success effect differed significantly from central, F(1,35820) = 10.29, p = .001, and posterior channels, F(1,35820) = 12.26, p < .001, in which the effect did not emerge. In the next time window of interest (200–350 ms), the recall success effect across conditions was present in all loci (all ps < .001), but was more pronounced in anterior compared to central channels, F(1,35820) = 23.45, p < .001, and to posterior channels, F(1,35820) = 43.92, p < .001. The same pattern emerged in the next time window of interest (350–600 ms), in which the recall success effect also emerged in all loci (all ps < .001) but was more pronounced in anterior compared to central channels, F(1,35820) = 13.27, p < .001, and to posterior channels, F(1,35820) = 25.82, p < .001. Additionally, decomposition of the recall success  laterality interaction in this time window revealed the recall success effect was stronger in right side compared to left locations, F(1,35820) = 12.21, p < . 01, and in midline compared to left locations, F(1,35820) = 6.71, p < .01. Importantly, in this time window a significant recall success  condition  anteriority emerged. This interaction seems to reflect the

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

7

Fig. 4. Group mean ERP waveforms elicited by recall success and failure trials in the concurrent encoding condition (left) and the sequential encoding conditions (right). Data is averaged for the three posterior electrodes clusters. Shading indicates the adjusted time window used for post hoc analysis of the posterior effect in the 3rd time window (400–690 ms).

conditions, F(1,26849) = 22.34, p < .001, it was larger in the former. The recall success  anteriority interaction suggested that this effect was more pronounced in anterior compared to central sites, F(1,35820) = 5.83, p < .05, and in anterior compared to posterior sites, F(1,35820) = 11.59, p < .001. Moreover, decomposition of the recall success  laterality interaction revealed that the effect was stronger in right compared to left locations, F(1,35820) = 9.28, p < .01, and in midline compared to left locations, F(1,35820) = 4.46, p < .05.

Fig. 5. Expansion of the three-way interaction in the third (350–600 ms and 400– 690 ms) time window of interest, *p < .05; ***p < .001.

result that in anterior sites the recall success effect is larger in the concurrent condition than in the sequential condition, while the reverse is true for posterior sites. However, decomposition of this 3-way interaction by examining the recall success  condition interaction separately for each location revealed no significant interactions (all ps > .1). This is the case for the time windows of interest initially selected on the basis of our previous cued-recall study (Tibon & Levy, 2013). However, visual examination of the waveforms revealed that the posterior recall success effects in the current study for both concurrent and sequential encoding conditions were better encompassed by the 400–690 ms window. We therefore conducted an additional analysis of the difference between the conditions of interest during that time window (Fig. 4). This revealed a significant recall success  condition interaction in posterior channels, F(1,17892) = 4.45, p < .05. Decomposition of this interaction revealed that the recall success effect is significant in the sequential, F(1,8927) = 23.93, p < .001, but not in the concurrent condition. The results of these analyses are portrayed in Fig. 5. Finally, in the last time window of interest (600–1000 ms) the recall success effect was found in both conditions and in all locations. Decomposition of the recall success  condition interaction revealed that even though the recall success effect was apparent in both sequential, F(1,26849) = 87.94, p < .001, and concurrent

3.2.4. Mixed-effects model analysis versus repeated measures ANOVA. As we explain above, mixed-effects model analysis appears to be the appropriate mode of inspecting data in which bin size is determined post hoc by individual performance. However, since that type of analysis is not yet widespread, we compared the results of that approach with those of a conventional repeated measures (RM) ANOVA. The results of these analyses are reported in Appendix A. The RM ANOVAs do not entirely concur with the results of the mixed effects models analyses. The chief difference is that the recall success  condition interactions seen in the mixed effects models analysis were absent from the third and fourth time windows. As noted in the appendix, these discrepancies appear to be a function of differences between averaging methods. Importantly, both analyses revealed a robust frontally distributed recall success effect that was apparent in both conditions, accompanied by a posterior recall success effect that was more pronounced in the sequential condition. 3. Discussion In the current study, we examined how the temporal complexity of an experience may govern the processes required for remembering its component parts. We recorded EEG while participants engaged in cued recall of object picture pair associates, which had been previously encoded either concurrently or sequentially. Several ERP differences linked to subsequent cued recall success were found in time windows of interest. One of these effects emerged in anterior electrode sites at an early stage, soon after cue onset (0–200 ms). These results converge with our previous finding of recall success modulation of such early components in a cued recall employing unimodal stimulus pairs (Tibon & Levy, 2013). Following a proposal by Addante,

8

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

Watrous, Yonelinas, Ekstrom, and Ranganath (2011) that prestimulus oscillatory brain activity measured at frontal electrode sites may predict subsequent source memory retrieval success, we have suggested that this divergence may reflect a preparatory task set or retrieval orientation to the retrieval cue, allowing for within-domain associations formed at encoding to subsequently encourage a type of pattern completion strategy at retrieval. In the current case, both conditions were comprised of visual stimuli, allowing for this preparatory orientation to facilitate both concurrently and sequentially recalled targets. In the following time window of interest (200–350 ms), recall success effects took the form of more positive (or less negative) deflections in successful than in unsuccessful trials for both concurrent and sequential conditions, most pronounced in anterior electrode sites (in which the divergence continued robustly until 600 ms after cue presentation). This recall success effect, also observed in our prior study, invites comparison with a similar component found in recognition studies, the FN400 or the early midfrontal old–new effect, analogously characterized by a more negative deflection at anterior electrodes for probes judged to be new (e.g., Mecklinger, 2000; Rugg & Curran, 2007). As we have noted in our earlier study (Tibon & Levy, 2013), while in studies of retrieval of verbal materials retrieval success effects of this type emerge at a latency of 300–500 ms post cue, when picture cues are employed as memoranda the effect begins earlier (e.g., Ally & Budson, 2007; Ecker, Zimmer, Groh-Bordin, & Mecklinger, 2007; Herzmann, Jin, Cordes, & Curran, 2012; Iidaka, Matsumoto, Nogawa, Yamamoto, & Sadato, 2006; Jäger, Mecklinger, & Kipp, 2006; Opitz, 2010). Furthermore, in studies of picture recognition, not only the onset latency but the duration of the FN400 component is reported to be the similar to the cued recall effect in the present study (e.g., Iidaka et al., 2006, and Opitz, 2010; in Yu & Rugg, 2010, the effect was stable over the 200–500 ms range). Thus, it appears that the current frontal recall success effect is comparable to the mid-frontal old/new/FN400 recognition effect. That effect is widely held to index familiarity (reviewed by Friedman & Johnson, 2000; Mecklinger, 2000; Rugg & Curran, 2007; Wilding & Ranganath, 2011). However, it seems unlikely that familiarity processes can account for the recall-success effects found in the current study, since in recall, a stored target representation must be activated by the cue and enter consciousness in order to serve as the required response. The target representation is not available to be judged as familiar or not, and therefore target familiarity or even associative familiarity between the probes (Mayes, Montaldi, & Migo, 2007), which can support associative recognition judgments, can play no role in this paradigm. An alternative interpretation of the component herein identified is that it reflects cue familiarity, with cues that are more familiar yielding more recall success. We believe that this is less likely, since all cues were stipulated to be old (no judgments regarding cues were required), and likely to be recognized at a high level of confidence (with a 95% hit rate in the control experiment reported above). Furthermore, in a previous study (Tibon & Levy, 2013), this effect was only found in the unimodal (picture–picture pairs) condition, but not in the cross-modal (picture-sound pairs) condition, despite the fact that pictures were the cues in both conditions such that cue familiarity probability was the same. Given findings from hemodynamic neuroimaging studies indicating that there is substantial overlap between neural substrates of successful cued-recall and recognition (Habib & Nyberg, 2008; Okada, Vilberg, & Rugg, 2012), the emergence of this retrieval-success-related component in a cued recall task poses a challenge to the notion that this early mid-frontal component is indeed fully explained by the process of familiarity. Rather, it is possible that retrieval success effects beginning at 200 ms reported both in recognition and cued recall studies represent a different process common to both forms of retrieval. As we have noted (Tibon &

Levy, 2013), episodic retrieval success has been reported to be reflected electrophysiologically elsewhere in the brain during this time window. Local field potential recording in medial temporal lobes displays divergence of successful from unsuccessful item recognition beginning 200 ms post-cue presentation (Grunwald et al., 2003; Staresina, Fell, Do Lam, Axmacher, & Henson, 2012), and significant divergence related to successful source-memory retrieval in the hippocampus is reported beginning at 250 ms (Staresina et al., 2012). This activity temporally overlaps the recall-success effect over the frontal sites reported in our study, and raises the possibility that this frontal component might be related to attempts at ecphory reflected by hippocampal activity. The greater negativegoing wave observed for unsuccessful trials in the present study may reflect one type of frontal lobe ‘‘working-with-memory’’ operations (Moscovitch, 1992). Frontal mechanisms might engage in querying medial temporal lobe representations, continuing until a decision is made to cease retrieval efforts. It has been suggested that one route to cued recall is a process called ‘‘direct access’’ (Brainerd & Reyna, 2010), a putative automatic activation of a pair associate by a cue which is less richly recollective in nature than reconstructive cued recall. The extension of the frontal recall success effect in both conditions until 600 ms might reflect such direct access. Additionally, the early frontal processes might lead to successful recall via recollective processes, reflected by parietal components that are initiated at longer latencies. In the next time window (350–600 ms), significant recall-success effects were observed in all electrode clusters of interest. Importantly, in this time window, activity patterns in the concurrent condition differed from those of the sequential condition. While in anterior clusters the recall success effect did not differ between the concurrent and sequential conditions, in posterior sites the peak period effect was larger in the sequential condition. The finding that the effect was stronger in sequential vs. concurrent encoding condition was significant in the time window in which the posterior component was most pronounced (400–690 ms; Figs. 4 and 5). While the anterior effect seems to be a continuation of that found in the earlier time window, the posterior dissociation between the conditions suggests that in this time window an additional process contributes to recall success. The concurrent and sequential presentation conditions require encoding and retrieval of the same amount of information. The modality and feature characteristics of the to-be-recalled stimuli are identical across conditions. Why then would the tasks differ with regard to the processes needed to achieve accurate recall? Seemingly, the difference lies in the differential complexity of the encoding episodes. Encoding and retrieval of temporally structured memories, and specifically of sequences of discontiguous events, appear to be the province of the hippocampus. In classical conditioning paradigms, successful acquisition of conditional associations has been shown to rely on the hippocampus in trace conditioning, where CS and US are separated by a temporal gap, but not in delay conditioning, where CS and US overlap in time (Clark & Squire, 1998; Solomon, Vander Schaaf, Thompson, & Weisz, 1986). Closer to the episodic realm, it has been suggested that hippocampal ‘‘time-cells’’ play the role of an associator of discontiguous events (MacDonald, Lepage, Eden, & Eichenbaum, 2011). Aside from supportive findings from animal research, this suggestion is supported by studies in humans (Hales & Brewer, 2010; Hales, Israel, Swann, & Brewer, 2009; Staresina & Davachi, 2009), and is reinforced by a computational model of hippocampal activation (Wallenstein, Eichenbaum, & Hasselmo, 1998). Episodic retrieval in which the hippocampus is implicated is generally considered to be related to recollective processes (Aggleton & Brown, 1999; Eichenbaum, Yonelinas, & Ranganath, 2007). Importantly, the late posterior retrieval success ERP component

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

(LPC) is also considered to be modulated by the degree of recollection (Friedman & Johnson, 2000; Mecklinger, 2000; Rugg & Curran, 2007; Wilding & Ranganath, 2011). The pattern of modulation in the posterior electrode sites in the current study is reminiscent of the LPC as reported in recognition studies (though with a latency shift seemingly a function of the use of pictorial cues as in the FN400 analog, see the studies listed above). In the present case, the addition of the temporal dimension in the sequential condition results in increased complexity of the episodic experience, recollection of which seemingly requires greater engagement of hippocampal recollective processes, which leads to modulation of the LPC component. The modulation of the parietal-maximal LPC recall success effect by the temporal complexity of encoding events may contribute to the recent discussion of the functional significance of parietal retrieval-related hemodynamic activations (recently reviewed by Kim, 2011; Levy, 2012), with which the LPC has been linked (e.g., Rugg & Curran, 2007; Vilberg & Rugg, 2009; Wagner et al., 2005; Wilding & Ranganath, 2011). All extant theories concur that these activations are associated with recollection. Nonetheless, there is a substantial disagreement regarding the exact process they represent. One suggestion is that parietal activations at retrieval reflect the participation of the angular gyrus in multimodal representation (Shimamura, 2011). If that is the case, parietal ERPs should not have differed between concurrent and sequential conditions, as in both cases representations were comprised of unimodal visually presented stimuli. Thus the current findings do not support that account. Another suggestion is that parietal activations reflect a post-retrieval output buffer (Guerin & Miller, 2011; Vilberg & Rugg, 2007; Vilberg & Rugg, 2008; Vilberg & Rugg, 2009), in which retrieved information is maintained, possibly for the purpose of constraining subsequent action (Haramati et al., 2008). This temporary store acts as an interface between episodic memory and the central executive, and contains and integrates retrieved target information from a variety of sources. This suggestion can account for the finding that LPC modulation was increased in the sequential condition when extending this view by suggesting that the buffer includes the entire scope of retrieved information in addition to target information. In this case, the increased complexity of sequential representations might require additional buffering resources. The attentional processes account (Cabeza et al., 2012; Ciaramelli et al., 2010) asserts that dorsal parietal mnemonic activations reflect pre-retrieval top-down attentional processes required for challenging retrieval, while ventral parietal activations reflect bottom-up capture of attention by stronger (and less mnemonically effortful) retrieved representations. The finding of larger LPC in cued recall made challenging by temporal complexity seems consonant with the requirement of greater attentional resources assigned to dorsal parietal function, but less in keeping with the account of ventral parietal activation by easier retrievals. Two additional accounts propose that parietal mnemonic activations reflect signal accumulation in the service of a recognition judgment (e.g., Donaldson et al., 2010; Wagner et al., 2005), and expectations regarding mnemonic status of a probe and their violations (Buchsbaum, Ye & D’Esposito, 2011; O’Connor, Han & Dobbins, 2010). These approaches relate to recognition decisions as signal-detection style evaluation of test probes as being either old or new, analogous to perceptual decisions. In cued recall following pair associate learning of unrelated stimuli, there are no probes to be assessed in a signal detection fashion, as the number of possible recall targets is unbounded. These accounts are therefore challenged by the emergence of parietal ERP effects for cued recall, in which those processes have no substrate on which to act. The current findings are thus in concord with aspects of post-retrieval buffer accounts and one aspect of the attentional account.

9

It is interesting that both frontal recall success divergences and the subsequent parietal recall success divergences seem to taper off at 600 ms, yielding to a different electrophysiological signature for the following time window. That time point might mark the cessation of both working-with-memory and/or direct access operations which we have proposed as being indexed by the FN400-type modulation, and of recollective retrieval processes, indexed by the LPC modulation. Following the marked offset of fronto-parietal differences of the third time-window, an additional recall-success effect, was found to begin at 600 ms post-cue, with more pronounced positivity for successful than for unsuccessful trials. This late recall success effect was more pronounced in the sequential compared to the concurrent condition, in anterior compared to central and posterior locations, and in right and midline compared to left locations. Similar late frontal activity has been explained as reflecting post-retrieval monitoring and verification, in which retrieved information is evaluated to determine if it meets retrieval criteria. The current component may be compared with late frontal components reported in connection with recognition tasks also requiring retrieval of contextual information, such as source retrieval (e.g., Cruse & Wilding, 2011; Wilding & Rugg, 1996; Woodruff, Hayama, & Rugg, 2006). The current study expands on prior studies, demonstrating that comparable late frontal effects may be obtained for successful cued recall following pair associate learning. The significant recall success  condition interaction in this time window may indicate that such post-retrieval monitoring is more demanding in the sequential condition, which requires richer episodic reconstruction due to increased complexity of the encoding episodes. Such additional monitoring and evaluation may be related to the longer latency hit RTs (+146 ms) for the sequential condition. In summary, using a paradigm that minimizes confounds by recognition processes, we have identified several event-related components elicited in conjunction with successful episodic cued recall following pair-associate learning. ERP components analogue to those traditionally found in recognition memory studies were modulated by the temporal structure of the encoded episode, suggesting that the temporal texture of encoding events leads to the modulation of mnemonic processes at several time windows which differentially support the retrieval of concurrent and sequential associations.

Acknowledgments This work was supported by the Israel Science Foundation grant 611/09. We thank Shir Ben-Zvi, Ayelet Peer, Roni Kliger, Lior Levavi, and Efrat Naaman for assistance with stimuli preparation, data collection and analysis.

Appendix A Although for reasons explained in the Methods section we employed a Mixed-Effects Model approach to analysis of the ERP data, herein we report results of conventional repeated measures (RM) ANOVA. The difference between mixed effect models and RM ANOVA approaches is not only in the nature of the statistical tests employed, but more importantly in that the data itself is computed in a different manner. For RM ANOVA grand averages (i.e., a group mean of individual participants’ means) are computed, while for the current mixed effects models analysis an average was computed across all trials of all participants. The differences yielded by these two methods of averaging may be seen in Figs. A1 and A2 below.

10

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

Fig. A1. Comparison between the grand average approach of conventional repeated measures ANOVA (top row) and the global averaging approach of mixed effects models analyses (bottom row) of the mean EEG amplitudes of posterior electrode clusters. The 400–690 ms time window employed for examination of the mid-late posterior recall success effect is highlighted.

For the RM ANOVA analyses we computed four ERP waveforms for each participant: Concurrent – recall success; Concurrent – recall failure; Sequential – recall success; and Sequential – recall failure. We performed the RM ANOVAs both for the entire set of participants, and for participants with at least 10 trials in each bin. Here we report the analyses performed on the entire set of participants. The results of the two analyses were very similar, and all trends were preserved across those two analyses. RM ANOVAs including factors of retrieval success (success/failure), condition (concurrent/sequential), anteriority (frontal, central, parietal), and laterality (left, midline, right) were conducted separately for each epoch. The Geisser–Greenhouse correction for violations of sphericity (Greenhouse & Geisser, 1959) was employed as necessary. Resultant effects and interactions are shown for each time window in Table A1. These results indicate that the RM ANOVAs do not entirely concur with the results of the mixed effects models analyses. Focusing on comparisons in which recall success is involved, it may be seen that new interactions between recall success and laterality (or laterality and anteriority) factors emerged in the three later time windows. On the other hand, the recall success  condition interactions seen in the mixed effects models analysis were absent from the third and fourth time windows. To examine the discrepancy between analytic approaches in recall success  condition interactions in the third time window, we separately analyzed the frontal and posterior ERP effects in the conventional repeated measures approach. We first examined the effects and interaction in the posterior channels in the adjusted

third time window that was used for the mixed effects analysis (400–690 ms). The averages employed for these analyses are portrayed in Fig. A1. The repeated measures analysis revealed a significant effect of recall success, F(1,32) = 5.9, p < .05, and a significant condition  laterality interaction, F(2,64) = 6.78, p < .01. Importantly, this analysis also revealed a borderline recall success  condition interaction, F(1,32) = 4.04, p = .05. Furthermore, RM ANOVA analysis for midline channels, where activation was maximal, revealed a significant recall success  condition interaction, F(1,32) = 4.58, p < .05. Additionally, pairwise comparisons revealed that for all three posterior electrode clusters, the recall success effect was significant in the concurrent condition (all ps < .05), but not in the sequential condition (all ps > .1). Thus, despite the aforementioned discrepancy between analyses, focused examination of posterior effects indicates accordance among the analyses in this time window. RM ANOVA for anterior channels in the third time window (350–600 ms) revealed a significant effect of recall success, F(1,32) = 15.84, p < .001, significant effect of laterality, F(2,64) = 11.97, p < .001 and a significant recall success  laterality interaction, F(2,64) = 9.12, p = .001. As in the mixed effects analysis, here too the recall success  condition interaction was not significant. Nevertheless, in the mixed effects models data averaging, the numerical difference between recall success and recall failure trials in the concurrent condition was larger than in the sequential condition, while in the RM ANOVA data averaging the pattern was reversed. This discrepancy in the waveform is portrayed in Fig. A2.

11

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

Fig. A2. Comparison between the grand average approach of conventional repeated measures ANOVA (top row) and the global averaging approach of mixed effects models analyses (bottom row) of the mean EEG amplitudes of anterior electrode clusters. The 350–600 ms time window employed for examination of the anterior recall success effect is highlighted. Table A1 Outcomes of repeated measures ANOVA models analysis. Epoch (in ms)

Recall success Condition Anteriority Laterality Recall success  laterality Recall success  anteriority Condition  anteriority Condition  laterality Laterality  anteriority Condition  laterality  anteriority Recall success  laterality  anteriority

0–200

200–350

350–600

600–1000

4.09(.051)

10.35(<.01)

13.19(.001)

5.61(<.05)

11.78(.001) 4.44(<.05)

28.23(<.001) 17.64(<.001) 5.97(<.01) 12.12(.001) 7.96(<.01) 5.26(<.01) 6.91(.001) 3.81(<.05)

34.2(<.001) 7.18(<.01) 6.64(<.01) 9.31(<.01)

5.27(<.05) 3.64(<.05)

12.4(<.001) 5.1(.01) 5.9(<.01)

These separate analyses of the posterior and anterior clusters indicate that the absence of the recall success  condition  anteriority interaction in the RM ANOVA most likely results from the impact of averaging methods reversing the pattern of the anterior effect, while the posterior effect remains stable. The main outcome of this inversion of the anterior effect is that in the RM ANOVA data averaging the anterior effect (in which the recall success effect is larger in the concurrent condition) is numerically similar to the posterior effect. This cancels differences related to anteriority, and therefore, the 3-way interaction did not emerge. Another discrepancy between the analyses was revealed in the fourth time window (600–1000 ms), in which the recall success  condition interaction was significant in the mixed effects

4.11(<.05) 8.32(<.001) 3.94(<.05) 5.47(<.01)

10.25(<.001) 7.82(<.001)

models analysis, but not in the RM ANOVA. For comparative purposes, we decomposed the interaction using paired sample ttests for each condition. This revealed a significant difference in the sequential condition, t(32) = 2.72, p = .01, but not in the concurrent condition, t(32) = 0.74, p = .46. Additionally, pairwise comparisons performed separately for each cluster revealed a significant difference between recall success and failure in the sequential condition in all nine locations (ps < .05 except the central left cluster, for which a marginal effect was revealed), but not for the concurrent condition (all ps > .1). It thus seems that the general trend of the dissociation between the conditions remains, even though a significant interaction did not emerge in the RM ANOVA.

12

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13

Interpretations regarding effects that were not consistent across analyses should be treated with caution. Nonetheless, the discrepancy between the results of mixed effect models and of the RM ANOVA requires further discussion. Importantly, as can be seen in the figures above, this discrepancy should not be attributed to the statistical analyses alone, but also (and perhaps mostly) to the approach by which the data is organized, i.e., whether the average is computed across participants or across all trials. Both SSsubject, corresponding to the variability among subject means, and SSRM, corresponding to the variability among the repeated factor means, contribute to the SStotal of the RM ANOVA design. Nevertheless, both of these components are biased as a result of the design employed in the current study. In many other studies the number of trials for each participant in each bin also differs, since usually only correct responses are analyzed. This dependency between accuracy rates and number of trials used for ERP analysis is usually ignored, even though SNR substantially depends on the number of trials that are used in the analysis, and variance decreases with the addition of trials. In our case the variations are even more dramatic, since the allocation of trials to bins is done post hoc, and bins are mutually depended (i.e., a participant who had 100 correct responses, can only have 20 error responses, and vice versa). This means that the SSRM component is markedly biased: the average difference between the number of success and failure trial was 31.24 (STD = 21.6, maximum: 72) in the concurrent condition and 38.21 (STD = 25.54, maximum: 98) in the sequential condition. The SSsubject component is also biased, since it does not account for the large variability in the number of trials each participant has in each condition (STD of the number of trials ranging between 19.3 and 24.9 for the different conditions). The RM ANOVA design does not include different weightings that depend on the amount of information represented by the bin (i.e., the number of trials comprising it) and is therefore inadequate for this kind of data. On the other hand, in the mixed effects models design used in the current study, each observation serves as an element of analysis to be modeled, and therefore each trial is weighted equally; degrees of freedom represent the number of observations and not the number of participants as is customary in grand average ANOVA. Although at first glance this might appear to be an overly liberal approach, in this approach large inter-observation variance is not tempered by averaging within participants, which limits the number of effects that emerge as significant. Furthermore, effects that do emerge from the statistical analyses are reflected by robust differences in mean amplitudes. Additionally, the emergence of some significant effects in the RM ANOVA, which were not apparent in the mixed effects analyses, indicate that this analysis is not necessarily an ‘‘overly liberal’’ approach. We therefore believe that mixed effects models are more appropriate for the analysis of the data of the current study. References Addante, R. J., Watrous, A. J., Yonelinas, A. P., Ekstrom, A. D., & Ranganath, C. (2011). Prestimulus theta activity predicts correct source memory retrieval. Proceedings of the National Academy of Sciences, 108, 10702–10707. http://dx.doi.org/ 10.1073/pnas.1014528108. Aggleton, J. P., & Brown, M. W. (1999). Episodic memory, amnesia, and the hippocampal-anterior thalamic axis. Behavioral and Brain Sciences, 22, 425–444. http://dx.doi.org/10.1017/S0140525X99002034. discussion 444–489. Ally, B. A., & Budson, A. E. (2007). The worth of pictures: Using high density eventrelated potentials to understand the memorial power of pictures and the dynamics of recognition memory. NeuroImage, 35, 378–395. http://dx.doi.org/ 10.1016/j.neuroimage.2006.11.023. Bagiella, E., Sloan, R. P., & Heitjan, D. F. (2000). Mixed-effects models in psychophysiology. Psychophysiology, 37, 13–20. http://dx.doi.org/10.1111/ 1469-8986.3710013. Brainerd, C. J., & Reyna, V. F. (2010). Recollective and nonrecollective recall. Journal of Memory and Language, 63, 425–445. http://dx.doi.org/10.1016/ j.jml.2010.05.002.

Buchsbaum, B. R., Ye, D., & D’Esposito, M. (2011). Recency effects in the inferior parietal lobe during verbal recognition memory. Frontiers in Human Neuroscience, 5, 59. http://dx.doi.org/10.3389/fnhum.2011.00059. Cabeza, R., Ciaramelli, E., & Moscovitch, M. (2012). Cognitive contributions of the ventral parietal cortex: An integrative theoretical account. Trends in Cognitive Sciences, 16, 338–352. http://dx.doi.org/10.1016/j.tics.2012.04.008. Cabeza, R., Mangels, J., Nyberg, L., Habib, R., Houle, S., McIntosh, A. R., et al. (1997). Brain regions differentially involved in remembering what and when: A PET study. Neuron, 19, 863–870. http://dx.doi.org/10.1016/S0896-6273(00)80967-8. Ciaramelli, E., Grady, C., Levine, B., Ween, J., & Moscovitch, M. (2010). Top-down and bottom-up attention to memory are dissociated in posterior parietal cortex: Neuroimaging and neuropsychological evidence. The Journal of Neuroscience, 30, 4943–4956. http://dx.doi.org/10.1523/JNEUROSCI.1209-09.2010. Clark, R. E., & Squire, L. R. (1998). Classical conditioning and brain systems: The role of awareness. Science, 280, 77–81. http://dx.doi.org/10.1126/ science.280.5360.77. Cruse, D., & Wilding, E. L. (2011). Temporally and functionally dissociable retrieval processing operations revealed by event-related potentials. Neuropsychologia, 49, 1751–1760. http://dx.doi.org/10.1016/j.neuropsychologia.2011.02.053. Dobbins, I. G., Rice, H. J., Wagner, A. D., & Schacter, D. L. (2003). Memory orientation and success: Separable neurocognitive components underlying episodic recognition. Neuropsychologia, 41, 318–333. Donaldson, D. I., & Rugg, M. D. (1998). Recognition memory for new associations: Electrophysiological evidence for the role of recollection. Neuropsychologia, 36, 377–395. http://dx.doi.org/10.1016/S0028-3932(97)00143-7. Donaldson, D. I., & Rugg, M. D. (1999). Event-related potential studies of associative recognition and recall: Electrophysiological evidence for context dependent retrieval processes. Cognitive Brain Research, 8, 1–16. http://dx.doi.org/10.1016/ S0926-6410(98)00051-2. Donaldson, D. I., Wheeler, M. E., & Petersen, S. E. (2010). Remember the source: Dissociating frontal and parietal contributions to episodic memory. Journal of Cognitive Neuroscience, 22, 377–391. http://dx.doi.org/10.1162/ jocn.2009.21242. Dudukovic, N. M., & Wagner, A. D. (2007). Goal-dependent modulation of declarative memory: Neural correlates of temporal recency decisions and novelty detection. Neuropsychologia, 45, 2608–2620. http://dx.doi.org/10.1016/ j.neuropsychologia.2007.02.025. Ecker, U. K. H., Zimmer, H. D., Groh-Bordin, C., & Mecklinger, A. (2007). Context effects on familiarity are familiarity effects of context – An electrophysiological study. International Journal of Psychophysiology, 64, 146–156. http://dx.doi.org/ 10.1016/j.ijpsycho.2007.01.005. Eichenbaum, H., Yonelinas, A. R., & Ranganath, C. (2007). The medial temporal lobe and recognition memory. Annual Review of Neuroscience, 30, 123–152. http:// dx.doi.org/10.1146/annurev.neuro.30.051606.094328. Friedman, D., & Johnson, R. Jr., (2000). Event-related potential (ERP) studies of memory encoding and retrieval: A selective review. Microscopy Research and Technique, 51, 6–28. http://dx.doi.org/10.1002/1097-0029(20001001)51:1 <6::AID-JEMT2>3.0.CO;2-R. Fujii, T., Suzuki, M., Okuda, J., Ohtake, H., Tanji, K., Yamaguchi, K. & Yamadori, A. (2004). Neural correlates of context memory with real-world events. NeuroImage, 21, 1596–1603. doi: 10.1016/j.neuroimage.2004.01.005. Greenhouse, S. W., & Geisser, S. (1959). On methods in the analysis of profile data. Psychometrika, 2, 95–112. http://dx.doi.org/10.1007/BF02289823. Grove, K. L., & Wilding, E. L. (2009). Retrieval processes supporting judgments of recency. Journal of Cognitive Neuroscience, 21, 461–473. http://dx.doi.org/ 10.1162/jocn.2009.21040. Grunwald, T., Pezer, N., Münte, T. F., Kurthen, M., Lehnertz, K., Van Roost, D. & Elger, C. E. (2003). Dissecting out conscious and unconscious memory (sub)processes within the human medial temporal lobe. Neuroimage, 20, S1:S139–45. doi: 10.1016/j. neuroimage .2003.09.004. Guerin, S. A., & Miller, M. B. (2011). Parietal cortex tracks the amount of information retrieved even when it is not the basis of a memory decision. NeuroImage, 55, 801–807. http://dx.doi.org/10.1016/j.neuroimage.2010.11.066. Habib, R., & Nyberg, L. (2008). Neural correlates of availability and accessibility in memory. Cerebral Cortex, 18, 1720–1726. http://dx.doi.org/10.1093/cercor/ bhm201. Hales, J. B., & Brewer, J. B. (2010). Activity in the hippocampus and neocortical working memory regions predicts successful associative memory for temporally discontiguous events. Neuropsychologia, 48, 3351–3359. http:// dx.doi.org/10.1016/j.neuropsychologia.2010.07.025. Hales, J. B., Israel, S. L., Swann, N. C., & Brewer, J. B. (2009). Dissociation of frontal and medial temporal lobe activity in maintenance and binding of sequentially presented paired associates. Journal of Cognitive Neuroscience, 21, 1244–1254. http://dx.doi.org/10.1162/jocn.2009.21096. Haramati, S., Soroker, N., Dudai, Y., & Levy, D. A. (2008). The posterior parietal cortex in recognition memory: A neuropsychological study. Neuropsychologia, 46, 1756–1766. http://dx.doi.org/10.1016/j.neuropsychologia.2007.11.015. Hayes, S. M., Ryan, L., Schnyer, D. M., & Nadel, L. (2004). An fMRI study of episodic memory: Retrieval of object, spatial, and temporal information. Behavioral Neuroscience, 118, 885–896. http://dx.doi.org/10.1037/0735-7044.118. 5.885. Herzmann, G., Jin, M., Cordes, D., & Curran, T. (2012). A within-subject ERP and fMRI investigation of orientation-specific recognition memory for pictures. Cognitive Neuroscience, 3, 174–192. http://dx.doi.org/10.1080/17588928.2012.669364. Iidaka, T., Matsumoto, A., Nogawa, J., Yamamoto, Y., & Sadato, N. (2006). Frontoparietal network involved in successful retrieval from episodic

R. Tibon, D.A. Levy / Brain and Cognition 84 (2014) 1–13 memory. Spatial and temporal analyses using fMRI and ERP. Cerebral Cortex, 16, 1349–1360. http://dx.doi.org/10.1093/cercor/bhl040. Jäger, T., Mecklinger, A., & Kipp, K. H. (2006). Intra- and inter-item associations doubly dissociate the electrophysiological correlates of familiarity and recollection. Neuron, 52, 535–545. http://dx.doi.org/10.1016/ j.neuron.2006.09.013. Jenkins, L. J., & Ranganath, C. (2010). Prefrontal and medial temporal lobe activity at encoding predicts temporal context memory. The Journal of Neuroscience, 30, 15558–15565. http://dx.doi.org/10.1523/JNEUROSCI.1337-10.2010. Kim, H. (2011). Differential neural activity in the recognition of old versus new events: An activation likelihood estimation meta-analysis. Human Brain Mapping, 34, 814–836. http://dx.doi.org/10.1002/hbm.21474. Kimura, H. M., Hirose, S., Kunimatsu, A., Chikazoe, J., Jimura, K., Watanabe, T. & Konishi, S. (2010). Differential temporo-parietal cortical networks that support relational and item-based recency judgments. NeuroImage, 49, 3474–3480. doi: 10.1016/j.neuroimage.2009.11.003. Lehn, H., Steffenach, H.-A., van Strien, N. M., Veltman, D. J., Witter, M. P., & Håberg, A. K. (2009). A specific role of the human hippocampus in recall of temporal sequences. The Journal of Neuroscience, 29, 3475–3484. http://dx.doi.org/ 10.1523/JNEUROSCI.5370-08.2009. Levy, D. A. (2012). Towards an understanding of parietal mnemonic processes: Some conceptual guideposts. Frontiers in Integrative Neuroscience, 6, 41. http:// dx.doi.org/10.3389/fnint.2012.00041. MacDonald, C. J., Lepage, K. Q., Eden, U. T., & Eichenbaum, H. (2011). Hippocampal ‘‘time cells’’ bridge the gap in memory for discontiguous events. Neuron, 71, 737–749. http://dx.doi.org/10.1016/j.neuron.2011.07.012. Makeig, S., Westerfield, M., Jung, T.-P., Covington, J., Townsend, J., Sejnowski, T. J., et al. (1999). Functionally independent components of the late positive eventrelated potential during visual spatial attention. The Journal of Neuroscience, 19, 2665–2680. Mayes, A., Montaldi, D., & Migo, E. (2007). Associative memory and the medial temporal lobes. Trends in Cognitive Sciences, 11, 126–135. http://dx.doi.org/ 10.1016/j.tics.2006.12.003. Mecklinger, A. (2000). Interfacing mind and brain: A neurocognitive model of recognition memory. Psychophysiology, 37, 565–582. http://dx.doi.org/10.1111/ 1469-8986.3750565. Moscovitch, M. (1992). Memory and working-with-memory: A component process model based on modules and central systems. Journal of Cognitive Neuroscience, 4, 257–267. http://dx.doi.org/10.1162/jocn.1992.4.3.257. Naya, Y., & Suzuki, W. A. (2011). Integrating what and when across the primate medial temporal lobe. Science, 333, 773–776. http://dx.doi.org/10.1126/ science.1206773. Nyberg, L., McIntosh, A. R., Cabeza, R., Habib, R., Houle, S., & Tulving, E. (1996). General and specific brain regions involved in encoding and retrieval of events: What, where, and when. Proceedings of the National Academy of Sciences, 93, 11280–11285. http://dx.doi.org/10.1073/pnas.93.20.11280. O’Connor, A. R., Han, S., & Dobbins, I. G. (2010). The inferior parietal lobule and recognition memory: Expectancy violation or successful retrieval? The Journal of Neuroscience, 30, 2924–2934. http://dx.doi.org/10.1523/JNEUROSCI.422509.2010. Okada, K., Vilberg, K. L., & Rugg, M. D. (2012). Comparison of the neural correlates of retrieval success in tests of cued recall and recognition memory. Human Brain Mapping, 33, 523–533. http://dx.doi.org/10.1002/hbm.21229. Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97–113. http://dx.doi.org/10.1016/00283932(71)90067-4. Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J.-M. (2011). FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 1(1–1), 9. http://dx.doi.org/ 10.1155/2011/156869. Opitz, B. (2010). Context-dependent repetition effects on recognition memory. Brain and Cognition, 73, 110–118. http://dx.doi.org/10.1016/j.bandc.2010.04.003. Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., & the R Core Team (2007). nlme: Linear and nonlinear mixed effects models. R package version 3.1-86. Rekkas, P. V., Westerveld, M., Skudlarski, P., Zumer, J., Pugh, K., Spencer, D. D., et al. (2005). Neural correlates of temporal-order judgments versus those of spatiallocation: Deactivation of hippocampus may facilitate spatial performance. Brain and Cognition, 59, 103–113. http://dx.doi.org/10.1016/j.bandc.2005.05.013.

13

Rugg, M. D., & Curran, T. (2007). Event-related potentials and recognition memory. Trends in Cognitive Sciences, 11, 251–257. http://dx.doi.org/10.1016/ j.tics.2007.04.004. Rugg, M., & Wilding, E. L. (2000). Retrieval processing and episodic memory. Trends in Cognitive Sciences, 4, 108–115. http://dx.doi.org/10.1016/S13646613(00)01445-5. Shimamura, A. P. (2011). Episodic retrieval and the cortical binding of relational activity. Cognitive, Affective & Behavioral Neuroscience, 11, 277–291. http:// dx.doi.org/10.3758/s13415-011-0031-4. Solomon, P. R., Vander Schaaf, E. R., Thompson, R. F., & Weisz, D. J. (1986). Hippocampus and trace conditioning of the rabbit’s classically conditioned nictitating membrane response. Behavioral Neuroscience, 100, 729–744. http:// dx.doi.org/10.1037//0735-7044.100.5.729. St. Jacques, P., Rubin, D. C., LaBar, K. S., & Cabeza, R. (2008). The short and long of it: Neural correlates of temporal-order memory for autobiographical events. Journal of Cognitive Neuroscience, 20, 1327–1341. http://dx.doi.org/10.1162/ jocn.2008.20091. Staresina, B. P., & Davachi, L. (2009). Mind the gap: Binding experiences across space and time in the human hippocampus. Neuron, 63, 267–276. http://dx.doi.org/ 10.1016/j.neuron.2009.06.024. Staresina, B. P., Fell, J., Do Lam, A. T. A., Axmacher, N., & Henson, R. N. (2012). Memory signals are temporally dissociated in and across human hippocampus and perirhinal cortex. Nature Neuroscience, 15, 1167–1173. http://dx.doi.org/ 10.1038/nn.3154. Suzuki, M., Fujii, T., Tsukiura, T., Okuda, J., Umetsu, A., Nagasaka, T. & Yamadori, A. (2002). Neural basis of temporal context memory: A functional MRI study. NeuroImage, 17, 1790–1796. doi: 10.1006/nimg.2002.1303. Tibon, R., & Levy, D. A. (2013). The time course of episodic associative retrieval: Electrophysiological correlates of cued recall of unimodal and crossmodal pairassociate learning. Cognitive, Affective, & Behavioral Neuroscience. Published online: 6 August, 2013. doi: 10.3758/s13415-013-0199-x. Tubridy, S., & Davachi, L. (2011). Medial temporal lobe contributions to episodic sequence encoding. Cerebral Cortex, 21, 272–280. http://dx.doi.org/10.1093/ cercor/bhq092. Tulving, E., & Markowitsch, H. J. (1998). Episodic and declarative memory: Role of the hippocampus. Hippocampus, 8, 198–204. http://dx.doi.org/10.1002/ (SICI)1098-1063(1998) 8:3<198::AID-HIPO2>3.0.CO;2-G. Vilberg, K. L., & Rugg, M. D. (2007). Dissociation of the neural correlates of recognition memory according to familiarity, recollection, and amount of recollected information. Neuropsychologia, 45, 2216–2225. http://dx.doi.org/ 10.1016/j.neuropsychologia.2007.02.027. Vilberg, K. L., & Rugg, M. D. (2008). Memory retrieval and the parietal cortex: A review of evidence from a dual-process perspective. Neuropsychologia, 46, 1787–1799. http://dx.doi.org/10.1016/j.neuropsychologia.2008.01.004. Vilberg, K. L., & Rugg, M. D. (2009). Functional significance of retrieval-related activity in lateral parietal cortex: Evidence from fMRI and ERPs. Human Brain Mapping, 30, 1490–1501. http://dx.doi.org/10.1002/hbm.20618. Wagner, A. D., Shannon, B. J., Kahn, I., & Buckner, R. L. (2005). Parietal lobe contributions to episodic memory retrieval. Trends in Cognitive Sciences, 9, 445–453. http://dx.doi.org/10.1016/j.tics.2005.07.001. Wallenstein, G. V., Eichenbaum, H., & Hasselmo, M. E. (1998). The hippocampus as an associator of discontiguous events. Trends in Neurosciences, 21, 317–323. http://dx.doi.org/10.1016/S0166-2236(97)01220-4. Wilding, E. L., & Rugg, M. D. (1996). An event-related potential study of recognition memory with and without retrieval of source. Brain, 119, 889–905. http:// dx.doi.org/10.1093/brain/119.3.889. Wilding, E. L., & Ranganath, C. (2011). Electrophysiological correlates of episodic memory processes. In S. J. Luck & E. Kappenman (Eds.), The oxford handbook of ERP components (pp. 373–396). Oxford: Oxford University Press. doi:10.1093/ oxfordhb/9780195374148.013.0187. Woodruff, C. C., Hayama, H. R., & Rugg, M. D. (2006). Electrophysiological dissociation of the neural correlates of recollection and familiarity. Brain Research, 1100, 125–135. http://dx.doi.org/10.1016/j.brainres.2006.05. 019. Yu, S. S., & Rugg, M. D. (2010). Dissociation of the electrophysiological correlates of familiarity strength and item repetition. Brain Research, 1320, 74–84. http:// dx.doi.org/10.1016/j.brainres.2009.12.071.