Contralateral delay activity during temporal order memory

Accepted Manuscript Contralateral delay activity during temporal order memory Ulrich Pomper, Thomas Ditye, Ulrich Ansorge PII:

S0028-3932(18)30562-1

DOI:

https://doi.org/10.1016/j.neuropsychologia.2019.03.012

Reference:

NSY 7049

To appear in:

Neuropsychologia

Received Date: 12 September 2018 Revised Date:

15 March 2019

Accepted Date: 22 March 2019

Please cite this article as: Pomper, U., Ditye, T., Ansorge, U., Contralateral delay activity during temporal order memory, Neuropsychologia (2019), doi: https://doi.org/10.1016/j.neuropsychologia.2019.03.012. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

Contralateral delay activity during temporal order memory

Ulrich Pomper* (a), Thomas Ditye (a, b), Ulrich Ansorge (a)

RI PT

(a) Department of Basic Psychological Research and Research Methods Faculty of Psychology University of Vienna

SC

Liebiggasse 5

(b) Faculty of Psychology Sigmund Freud University Freudplatz 1

EP

TE D

1020 Vienna, Austria

M AN U

1010 Vienna, Austria

AC C

*corresponding author: Ulrich Pomper

[email protected]

Keywords: visual working memory, temporal order, contralateral delay activity, CDA, EEG Running head: Contralateral delay activity during order memory 1 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT

Abstract In everyday life, we constantly need to remember the temporal sequence of visual events over short periods of time, for example, when making sense of others’ actions or watching a movie.

RI PT

While there is increasing knowledge available on neural mechanisms underlying visual working memory (VWM) regarding the identity and spatial location of objects, less is known about how the brain encodes and retains information on temporal sequences. Here, we whether

the

contralateral-delay

activity

(CDA),

SC

investigate

a

well-studied

electroencephalographic (EEG) component associated with VWM of object identity, also

M AN U

reflects the encoding and retention of temporal order. In two independent experiments, we presented participants with a sequence of four or six images, followed by a 1 s retention period. Participants judged temporal order by indicating whether a subsequently presented probe image was originally displayed during the first or the second half of the sequence. As a

TE D

main novel result, we report the emergence of a contralateral negativity already following the presentation of the first item of the sequence, which increases over the course of a trial with every presented item, up to a limit of four items. We further observed no differences in the

EP

CDA during the temporal-order task compared to one obtained during a task concerning the

AC C

spatial location of the presented items. Since the characteristics of the CDA appear to be highly similar between different encoded feature dimensions and increases as additional items are being encoded, we suggest this component might be a general characteristic of various types of VWM.

2 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT

1. Introduction Visual working memory (VWM) is an important cognitive function, which helps to maintain information over short periods of time for manipulation or later access (Baddeley, 2003; Postle, 2015). It is a crucial prerequisite for many higher-level cognitive processes, such as

RI PT

reading, learning by observation, long-term memory, problem solving, reasoning, the integration and processing of information across sensory modalities, and the coherence of experience across interruptions of visual input (e.g., Fukuda, Vogel, Mayr, & Awh, 2010;

SC

Rensink, 2000; Shipstead, Redick, Hicks, & Engle, 2012; Unsworth, Fukuda, Awh, & Vogel, 2014).

M AN U

In line with a modular account of working memory in general (Baddeley, 2003; Baddeley & Hitch, 1974) previous work has suggested distinct specialized subsystems involved in VWM processes. A wealth of data from behavioural studies supports the independent storage and manipulation of spatial and object-based feature dimensions (Delogu, Nijboer, & Postma,

TE D

2012a, 2012b; Gmeindl, Walsh, & Courtney, 2011; Postle, 1997; Tresch, Sinnamon, & Seamon, 1993; but see also Morey, 2018 for a recent critical review). In addition to information about objects or items per se, VWM commonly also needs to

EP

maintain information about the temporal or serial order of events (Manohar, Pertzov, &

AC C

Husain, 2017; Marshuetz, 2005; Marshuetz & Smith, 2006). Past research has suggested that this function is likely to be supported via a diverse set of cognitive mechanisms, such as grouping of information (Burgess & Hitch, 1999), relative inter-item association coding (e.g., A before B, B before C, etc., e.g., Sternberg, 1967) as well as absolute magnitude coding (A: first, B: second, C: third, etc., e.g., Glenberg & Swanson, 1986), depending on the required precision and the task at hand. Imaging, electrophysiology, as well as lesion studies have typically found a wide network of brain areas active during order maintenance, including the 3 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT prefrontal cortex (Averbeck & Lee, 2007; Ninokura et al., 2003, 2004), the parietal lobes (Marshuetz, Smith, Jonides, DeGutis, & Chenevert, 2000; Zhang et al., 2003), and hippocampus (Heusser, Poeppel, Ezzyat, & Davachi, 2016; Roberts, Libby, Inhoff, & Ranganath, 2018).

RI PT

In humans, an established electrophysiological marker of VWM is the contralateral delay activity (CDA; Luria, Balaban, Awh, & Vogel, 2016; Vogel & Machizawa, 2004), a slowwave evoked component in the electroencephalogram (EEG). In a typical protocol (e.g.,

SC

Vogel & Machizawa, 2004), each trial starts with a central cue indicating whether participants should memorize upcoming items presented on the left or on the right side of a

M AN U

visual screen. After the cue, a number of coloured squares are presented briefly, followed by a retention period of 1 s. Finally, a probe display is shown and participants are asked to report whether the items on the cued side of the probe display are identical to those on the previous encoding display. During the retention period, the CDA emerges as a negative potential,

TE D

which is larger contralateral compared to ipsilateral of the memorized hemifield over posterior-parietal electrodes.

Research on the functional significance of the CDA has shown that its amplitude scales with

EP

increases in VWM load up to a plateau at 3 to 4 items (Ikkai, McCollough, & Vogel, 2010;

AC C

Vogel & Machizawa, 2004). This is in line with behavioural VWM capacity limits (Cowan, 2001; Luck & Vogel, 2013), although the limitation might be subject to the nature and complexity of the items (Brady, Störmer, & Alvarez, 2016; but see Balaban & Luria, 2015). Further, CDA amplitude is positively correlated with individual memory capacity (Ikkai et al., 2010; Vogel & Machizawa, 2004) and can be modified via WM training (Li, He, Wang, Hu, & Guo, 2017). In general, the CDA is most commonly interpreted as neural index of WM storage (Adam, Robison, & Vogel, 2018; Feldmann-Wüstefeld, Vogel, & Awh, 2018; Vogel 4 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT & Machizawa, 2004), although some recent accounts suggest it might also reflect spatial selective attention (Berggren & Eimer, 2016; see Discussion section). So far, CDA research has primarily focused on the process of maintaining object characteristics, such as form or colour, following the presentation of a single visual display (e.g., Adam, Vogel, & Awh,

RI PT

2017; Vogel & Machizawa, 2004, but see, e.g., Berggren & Eimer, 2016, for a recent example using two subsequent visual displays rather than one). Currently, it is unknown whether the neural processes underlying the CDA are also active during the maintenance of

SC

temporal order information in VWM. A recent series of studies have suggested that an oscillatory mechanism based on the coupling between fast gamma-band and slower theta-

M AN U

band oscillations might underlie temporal order memory (Heusser et al., 2016; Lisman, 2009; Lisman & Jensen, 2013). If the CDA is involved in the maintenance of temporal order as well, this would indicate its reflection of a more general process, independent of the current VWM task or the retained visual feature dimension. Thus, in the present study, we set out to

TE D

investigate whether the CDA also operates when participants have to remember the temporal order of sequentially presented objects. In two independent experiments, we used both existing (pictures of objects) and non-existing (pictures of amorphous figures; i.e., fribbles)

EP

visual stimuli, as well as conducted a direct comparison between a temporal order and a

AC C

spatial VWM task.

Our results demonstrate the presence of a CDA throughout the encoding and retention periods of sequentially presented items, both when participants memorized the temporal order and when they memorized the spatial location of items.

2. Experiment 1 2.1. Methods 5 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT 2.1.1. Participants Seventeen healthy university students participated in the experiment in the course of a handson university seminar. Our sample size was based on previous reports of the CDA, which commonly incorporated between 12 and 18 participants (e.g., Balaban & Luria 2015;

RI PT

Berggren & Eimer, 2016; Drew et al., 2012; Ikkai et al., 2010; Katus & Müller, 2016; Vogel & Machizawa, 2004). The data of five participants had to be excluded due to extensive muscle artefacts and/ or slow EEG drifts. The remaining twelve participants (seven female;

SC

mean age: 22.5 years, range: 18 to 29) were right-handed, had normal or corrected to normal vision, and were naive to the purpose of the experiment. All gave written informed consent

M AN U

and received course credit for their participation. The study was conducted in accordance with the standards of the Declaration of Helsinki.

2.1.2. Apparatus

TE D

Stimuli were presented on a 19-inch CRT monitor with a resolution of 1,024 by 768 pixels and a refresh rate of 85 Hz. Participants sat inside a dimly lit room 58 cm away from the screen, with their heads supported by a chin rest. Eye movements were monitored and

EP

recorded during the entire experiment to ensure central fixation throughout each trial (Eyelink

AC C

1000+; SR Research, Ontario, Canada; desktop mount). The experiment was controlled by MATLAB (2013, The MathWorks, Natick, MA) using the Psychophysics Toolbox (Brainard, 1997) with the Eyelink extension (Cornelissen, Peters, & Palmer, 2002) on a PC running Windows 7.

2.1.3. Stimuli

6 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT We employed a large stimulus set, in order to minimize the number of items used on multiple trials and, thus, for example, influences of incongruencies between past and present position of an item in a sequence. The items to memorize were manually grey-scaled images of objects from 25 different categories1. Each category included 16 images with a size of 300 ×

RI PT

300 pixels (e.g., chairs, see Figure 1A, left). All images were acquired via an online search for images of the respective object category. Images were selected to include only a single object on a uniform white background, similar in size across all images, and well

SC

representative of the respective category. During the experiment, a scrambled version of each image was generated by applying a grid with a square side length of 25 × 25 pixels to the

M AN U

images and then randomly shuffling the resulting squares (Figure 1A, right).

--- insert Figure 1 about here ---

TE D

2.1.4. Task and Procedures

The participants’ task was to memorize the temporal order of a number of items presented sequentially. Figure 1B illustrates the experimental design. Each trial started with the

EP

presentation of a central fixation cross. Participants were instructed to fixate the cross

AC C

throughout each trial, and minimize eye movements and blinks. After 500 ms of correct fixation (automatically controlled via eye-tracking), a series of image pairs was presented sequentially. These were the encoding displays. Each pair consisted of an original object image on one side of the fixation cross, and the scrambled version of the same image presented on the opposite side. Within each trial, the images were drawn from the same category. Images were presented on uniform grey background (CIE XYZ, 53/0/0), vertically centred on the screen, and located at 6.2° of visual angle to the left and to the right of 7 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT fixation. Each image pair was presented for a duration of 800 ms, followed by an interstimulus interval of 125 ms. In 50% of the trials, four image pairs were presented in sequence (low-load condition) and in the other 50% six pairs were presented (high-load condition). The variable load (low vs. high) was randomly balanced across trials. Further, in 50% of the trials

RI PT

the object images of all pairs were presented on the same side, either left or right, and the scrambled distractors on the opposite side (lateralized condition). Left and right lateralized presentations were counter-balanced across trials, too. In the other 50% of trials, half of the

SC

images within each trial were presented on the left and the other half on the right (balanced condition). Here, the location of original images and scrambled versions was selected

M AN U

randomly within each trial. This balanced condition is essentially a randomly shuffled baseline. The factor of lateralization (lateralized vs. balanced) was randomized across trials. Since the CDA is defined as an increase in negativity contralateral to the visual hemifield of the items to memorize (Luria et al., 2016; Vogel & Machizawa, 2004), lateralization of the

TE D

items is a key requirement for the emergence of a CDA, employed in all previous studies. We hypothesized that if the CDA is a general marker of VWM processes, independent of the retained feature dimension, it would be present during the lateralized condition contralateral

EP

to the images, but not during the balanced condition, and that it would be larger for the

AC C

higher memory load of the lateralized condition than for the lower memory load of the lateralized condition.

The last image pair in each trial was followed by a retention period of 1 s duration. During this period, a fixation cross was presented at the centre of the screen and rectangular place holders (i.e., frames) were presented in the same positions as the previously shown memory items. Finally, during the probe interval, a randomly selected stimulus pair from the previous sequence was presented. The participants’ task was to provide a speeded response indicating 8 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT whether the probe item was originally presented in the first or the second half of the encoded sequence. Set sizes of four (low-load) and six (high-load) items were used since the tasks required an even number of images greater than 2 in order to keep task difficulty in a sensible range across conditions. In 50% of trials, the test item was presented on the same side, left or

RI PT

right, as during encoding (congruent condition). In the other half, the positions of the test item and its scrambled version were switched (incongruent condition). This factor of congruency was randomized across trials. We expected that spatial congruency between

SC

target and probe would be relevant for the balanced, but not for the lateralized condition. That is, during the balanced condition, a spatial match between target and probe might help to

M AN U

narrow down the potential target among the memorized items. However, this would not necessarily be the case during the lateralized condition, in which all stimuli were presented from the same side. Responses were made via the buttons '2' and '8' of the number block of a keyboard, using the left and right index fingers (mappings counter-balanced between

TE D

participants). The test item remained on the screen until a response was given. Subsequently, an inter-trial interval of 1,500 ms followed, during which only the central fixation cross was presented.

EP

In total, Experiment 1 comprised eight conditions (2 loads × 2 lateralizations × 2 congruency

AC C

levels), with 30 trials per condition, resulting in a total of 240 trials. Opportunities to rest were given after blocks of 40 trials. Prior to the start of the experiment, an additional practice of 10 trials familiarized participants with the task. Practice trials included a visual feedback on the accuracy of the responses and were repeated if necessary. This was not the case for the data-collection trials proper.

2.1.5. Electroencephalography recording and data pre-processing 9 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT EEG was recorded at 1,000 Hz using a full-band DC-EEG system (neuroConn GmbH, Ilmenau, Germany), with active electrodes (Brain Products, actiCAP system) mounted in a cap (EASY-CAP GmbH, Herrsching, Germany) at 64 positions of the extended 10/10 system. Electrode impedances were kept below 10 kΩ (Kappenman & Luck, 2010). The

RI PT

ground electrode at AFZ served as an online reference. Offline, data were bandpass filtered (finite impulse response filter) between 0.01 and 40 Hz, downsampled to 250 Hz, and rereferenced to the average of both mastoids. Trials containing muscle and technical artifacts

SC

were rejected via visual inspection. On average, 10.5% of trials were rejected. Electrodes with extremely high- and/or low-frequency artifacts throughout the recording (M = 2.4, SD =

M AN U

1.1) were linearly interpolated using a model of the amplitude topography at the unit sphere surface based on all non-artifactual electrodes (Perrin, Pernier, Bertrand, & Echallier, 1989). To reduce artifacts such as eye-blinks, horizontal eye movements, or electrocardiographic activity, an independent component analysis approach was applied (extended Runica; Lee,

TE D

Girolami, & Sejnowski, 1999). Components representing artifacts were removed from the EEG data by back-projecting all but these components (M = 3.7, SD = 0.8). This approach allows to selectively remove activity related to eye-movements, while retaining most trials

EP

containing eye artifacts. EEG data were then epoched from −500 ms before to 7 s after the

AC C

onset of the encoding period and baseline corrected using a −200 ms to 0 ms pre-stimulus interval. All offline data processing was done using MATLAB (The MathWorks, Natick, MA), EEGLAB (http://www.sccn.ucsd.edu/eeglab; Delorme & Makeig, 2004), and FieldTrip (http://www.ru.nl/fcdonders/fieldtrip; Oostenveld, Fries, Maris, & Schoffelen, 2011).

2.1.6. Analysis of behavioural data

10 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Prior to the statistical analysis, outlier trials with response times (RTs) deviating more than 2 SDs from the mean were excluded per participant and condition. Mean accuracy (percent correct) and RT measures (for correct trials only) were computed separately for each condition. For statistical analysis, accuracies and correct RT data were each subjected to a

RI PT

repeated-measurements analysis of variance (ANOVA), including the within-subject variables load (low vs. high), lateralization (lateralized vs. balanced), and congruency

SC

(congruent vs. incongruent).

M AN U

2.1.7. EEG data analysis

Event-related potentials (ERPs) were computed time-locked to the onset of the encoding sequence, separately for each combination of conditions. For displaying purposes, the contralateral delay activity (CDA) was calculated as follows (Luria et al., 2016; Vogel &

TE D

Machizawa, 2004): For trials in which the memory items were presented on the left side of the screen, ERPs at a left (ipsilateral) parietal site (electrode PO7) were subtracted from ERPs at a right (contralateral) parietal site (electrode PO8). Likewise, for trials in which the

EP

memory items were presented on the right side of the screen, ERPs at a right (ipsilateral)

AC C

parietal site (electrode PO8) were subtracted from ERPs at a left (contralateral) parietal site (electrode PO7). Importantly, in the balanced condition half of the memory items within each trial were presented on the left- and half on right side of the display. In order to still calculate an equivalent of the CDA for this condition, each trial was randomly labelled as either a left or right stimulation trial. CDAs were then averaged across both left- and right lateralized stimulation trials and across both congruent and incongruent conditions2, but separately for the balanced and lateralized and low- and high load conditions. For displaying purposes, 11 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT ERPs were further low-pass filtered at 20 Hz. For statistical analysis, ERPs prior to the subtraction of the contra- minus ipsilateral activity were used. Mean amplitudes at channels PO7 and PO8 were extracted during a time window from 300 ms following the onset of the first memory item until the end of the retention period. Then, a three-way repeated-

RI PT

measurements ANOVA was calculated, using the variables load (high vs. low), lateralization (lateralized vs. balanced), and hemisphere (ipsilateral vs. contralateral to the presentation side of the memory items). Note that we included the variable hemisphere in the ANOVA

SC

rather than using the (statistically identical) difference between ipsi- minus contralateral activity, in order to test for the presence of a potential CDA component (i.e., a significant

M AN U

main effect of hemisphere). Only ERPs from correct trials were used in the analyses. With congruent and incongruent trials pooled together for the CDA analysis, each condition

2.2. Results

TE D

contained an average of 37.9 trials.

2.2.1. Behavioural results

EP

Results can be seen in Figure 2. For both RTs and accuracy data, a three-way ANOVA was calculated, using the variables load (low vs. high), lateralization (lateralized vs. balanced),

AC C

and congruency (congruent vs. incongruent). For mean correct RTs, we found a significant main effect of lateralization, F(1, 11) = 6.02, p = .032, ηp2 = .35, due to faster RTs in the lateralized (M = 1,303 ms, SD = 403 ms) compared to the balanced condition (M = 1,350 ms, SD = 450 ms). There was a trend towards an interaction between load and congruency, F(1, 11) = 3.99, p = .071, ηp2 = .27, due to faster RTs in the low- compared to the high-load

12 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT condition for congruent, but not for incongruent trials. No other main effect or interaction reached significance (largest nonsignificant p = .106). For accuracy data, the ANOVA revealed a significant main effect of load, F(1, 11) = 8.73, p = .013, ηp2 = 0.44, due to a higher number of correct responses in the low- (M = 77%, SD =

RI PT

12.2%) compared to the high-load condition (M = 71.7%, SD = 9.1%). No other main effect or interaction reached significance (largest nonsignificant p = .09).

2.2.2. Event-related potentials

M AN U

SC

--- insert Figure 2 about here ---

Figures 3 and 4 illustrate the results from the ERP analysis. Overall, the lateralized condition (all memory items presented on one side), exhibited stronger negative amplitudes for

TE D

electrodes contralateral compared to ipsilateral of the memory items. As a result, the difference between the two activities (i.e., the contralateral delay activity, CDA) is clearly visible between the time-point of the first memory item presentation, and the onset of the

EP

probe item following the retention interval. Importantly, no such difference (i.e., CDA) is

AC C

visible in the balanced condition, during which memory items were presented randomly on both sides of the visual display. For statistical analysis, we computed a three-way repeated-measurements ANOVA using the variables load (high vs. low), lateralization (lateralized vs. balanced), and hemisphere (ipsilateral vs. contralateral to the presentation side of the memory items). We found a significant main effect of hemisphere, F(1, 11) = 24.65, p = .0004, ηp2 = .69, due to smaller mean amplitudes contralateral (M = 0.44 µV, SD = 2.6 µV) than ipsilateral (M = 2.29 µV, SD 13 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT = 2.11 µV) to the side where encoded items were presented. Next, we found a significant main effect of lateralization, F(1, 11) = 5.63 p = .037, ηp2 = .34, as a result of larger ERP amplitudes in the lateralized (M = 2.01 µV, SD = 2.77 µV) compared to the balanced condition (M = 0.65 µV, SD = 2.01 µV). Finally, we also found an interaction between the

RI PT

variables hemisphere and lateralization, F(1, 11) = 10.8, p = .007, ηp2 = .047. Post-hoc t-tests indicated the presence of a CDA component in the lateralized condition, t(11) = −4.31, p = .0012, d = 1.08, and a lack thereof in the balanced condition, p = .67. No other main effect

SC

or interaction reached significance (smallest nonsignificant p = .326).3

As the results for the balanced condition were based on an initial random labelling of trials as

M AN U

either left or right stimulation trials, we performed a resampling procedure to obtain a distribution of potential chance effects. Specifically, we repeated the randomization as well as the subsequent statistical analysis 500 times. Less than 5% of the obtained main effects, interactions, and post-hoc tests were different in direction and/ or significance from our

TE D

original findings. Further, the mean statistical results following this procedure were consistent with the original analysis. We found a main effect of hemisphere, F(1, 11) = 15.95 p = .005, ηp2 = .53 , a main effect of lateralization, F(1, 11) = 7.88 p = .002, ηp2 = .34, an

EP

interaction between the variables hemisphere and lateralization, F(1, 11) = 11.97, p = .005,

AC C

ηp2 = .59, and no difference between contra- and ipsilateral ERP amplitudes in the post-hoc test for the balanced condition t(11) = −0.04, p = .505, d = .12. Since, contrary to previous studies, we found no effect of load on overall CDA amplitudes (F(1, 11) = 0.08, p = .078, ηp2 = .008, achieved power = 0.09), we conducted an additional analysis to test whether the CDA amplitude might change over time in response to each presented image pair, and up to a capacity limit of around four items (Cowan, 2001; Ikkai et al., 2010; Luck & Vogel, 2013; Vogel & Machizawa, 2004). A similar approach has 14 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT previously been employed by Ikkai et al. (2010), who calculated the CDA separately for two consecutive memory arrays, and found an increase in mean amplitude between the first and the second array. Here, we calculated the average CDA amplitude separately for the first four memory displays, as the mean activity contra minus ipsilateral to the memory item within a

RI PT

time-window of 300 – 925 ms following the onset of each encoding display (e.g., Vogel & Machizawa, 2004). We then entered these values into a repeated measures three-way ANOVA, with the variables load (low vs. high), lateralization (lateralized vs. balanced), and

SC

time (first, second, third, fourth encoding display). Figure 4B, bottom row, illustrates the results of this comparison. We found a significant main effect of lateralization, F(1, 11) =

M AN U

14.63, p = .003, ηp2 = .60, a significant main effect of time, F(1, 11) = 8.92, p = .001, ηp2 = .386, as well as a significant interaction between lateralization and time, F(1, 11) = 4.13, p = .014, ηp2 = .298. To further investigate this interaction we conducted separate one-way ANOVAs including the variable time, for the balanced and lateralized condition, with data

TE D

pooled across both load conditions. Here, we found no effect for the balanced condition (p = .37, ηp2 = .070), and a significant effect for the lateralized condition F(1, 11) = 13.0 p < .001, ηp2 = .542. Finally, we ran follow-up t-tests between each successive pair of memory

EP

displays (i.e., first vs. second, second vs. third, third vs. fourth) for the lateralized condition

AC C

only, pooled across the variable load. This yielded a significant increase in lateralization between the first and second display, t(11) = 3.94, p = .002, d = .28, and between the third and fourth display, t(11) = 2.33, p = .04, d = .15. The difference between second and third display was nonsignificant (p = .13, d = .15).

--- insert Figure 3 about here ---

15 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT --- insert Figure 4 about here ---

2.3. Discussion Experiment 1 As hypothesized, we found a significant CDA in the lateralized condition, but not in the

RI PT

balanced control condition. Compared to previous studies, item presentation in our experiment was sequential rather than simultaneously, and the task was to memorize the temporal order rather than object characteristics or spatial information. Thus, our finding

SC

indicates that CDA is a more general aspect of VWM processes, elicited by the maintenance of various input features such as object form or colour (Ikkai et al., 2010; Luria et al., 2016;

M AN U

Vogel & Machizawa, 2004), but also temporal order.

Interestingly, however, in contrast to our expectations, we observed no main effect of load on the CDA. This was the case although behavioural accuracy indicated that the high-load condition was significantly more difficult than the low-load condition. To help interpret this

TE D

finding, we additionally calculated Bayes Factor (BF, according to Rouder, Speckman, Sun, Morey, & Iverson, 2009, using a scale factor of r = .7) indicating the evidence for or against an effect. We found a BF of 0.298, supporting the present lack of a load effect. Since four

EP

items are the average capacity limit for VWM tasks reported previously (Balaban & Luria,

AC C

2015; Ikkai et al., 2010; Luck & Vogel, 2013), it is possible that our set sizes of four and six items were not ideally suited to produce load effects on CDA amplitudes across the entire trial duration. As explained in the Methods section, these set-sizes were chosen with regard to the present temporal-order memory task, which required at least four items as well as an even number of presentation displays. It might be possible that load manipulation affects the CDA differently in a temporal than in a spatial task. Another potential explanation of this nullfinding is that trial numbers in the present experiment were too small to produce reliable load 16 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT effects. Importantly, however, despite the lack of a classic load effect, we found that CDA amplitude in the lateralized condition increased with the number of items encoded into WM (Figure 4B, bottom row). This indicates that load affected CDA amplitudes within conditions, as more items were encoded into WM over time.

RI PT

Further, it is interesting to see that our congruency manipulation – whether or not an image in the retrieval display was at the same position as in the encoding display – had no influence on behavioural memory performance. A number of studies have reported that retrieval is

SC

facilitated if spatial locations are congruent (Sun & Gordon, 2009, 2010) or if participants are allowed to look at the location of an encoded item rather than being instructed to look

M AN U

somewhere else (e.g., Johansson & Johansson, 2014). A potential reason for this difference is that we only used two locations per display, whereas prior demonstrations of congruence effects on memory retrieval used more than two positions. It is also conceivable that an overt shift of gaze, as used in previous studies, has a stronger effect on WM retrieval than the

TE D

covert shift of attention in our current study (Scholz, Klichowicz, & Krems, 2018). Interestingly however, we found that presenting all the memory items consistently from one side (lateralized condition) resulted in faster RTs compared to presenting them on both sides

EP

in a random fashion (balanced condition). This indicates that some aspect of spatial

AC C

congruency (albeit between different encoded stimuli rather than between encoding and retrieval displays) during VWM operations does facilitate memory performance. As a potential complication in this experiment, it is possible that some participants might have adopted a strategy of only memorizing the first half of the stimulus sequence (‘first half’ strategy). By just memorizing the first half of the encoding sequence, recognizing the probe item would mean it was presented in the first half, whereas not recognizing it would mean it was presented in the second half. Since the probe was always taken from the initial item 17 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT sequence, such a strategy would reduce the memory load and lower the task difficulty. Additionally, participants might have exploited the familiarity with the items and used silent verbal rehearsal during the encoding and retention phase in order to improve their working

RI PT

memory performance. These potential complications were addressed in Experiment 2.

3. Experiment 2

SC

To rule out the potential confounds present in Experiment 1, we conducted a second experiment. Specifically, the following changes to the original design were made: (1) To

M AN U

minimize the potential for verbal rehearsal, we used semantically meaningless, amorphic ‘fribble’ stimuli (Barry, Griffith, De Rossi, & Hermans, 2014; see Figure 1A, middle) instead of real-word objects as memory items. (2) To rule out the above mentioned ‘first half’ strategy, 50% of all trials in Experiment 2 used novel items as probe stimulus, which were

TE D

not part of the previous encoding sequence (variable target: present before vs. novel). (3) In order to directly compare the ERP activity related to memorizing the temporal order of items versus memorizing their spatial location, Experiment 2 featured two types of tasks. In the

EP

temporal task, which was identical to the one used in Experiment 1, participants had to

AC C

indicate whether the probe stimulus was presented during the first or second half of the encoding sequence. During the spatial task, participants had to indicate whether the probe stimulus was presented on the same or different side during the encoding sequence.

3.1. Methods 3.1.1. Participants

18 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Twenty-four healthy university students participated in this experiment in the course of a practical postgraduate seminar. Six participants had to be excluded due to extensive muscle artefacts and/ or slow EEG drifts. The remaining 18 participants (13 female; mean age: 24.1 years, range: 21 to 28) were all right-handed, had normal or corrected to normal vision, and

RI PT

were naive to the purpose of the experiment. All gave written informed consent and received course credit for their participation. The study was conducted in accordance with the

SC

standards of the Declaration of Helsinki.

3.1.2. Stimuli

M AN U

The items to memorize were grey-scaled, amorphic ‘fribble’ objects with a size of 400 × 300 pixels (Barry et al., 2014; see Figure 1A). These objects were designed to be similar to real world objects, but easy to generate and control in terms of their properties and their similarity within and between categories. The images were taken from an online object database (The

TE D

CNBC Wiki, retrieved from: http://wiki.cnbc.cmu.edu/Novel_Objects). For each trial, fribbles were randomly selected from within three categories, each containing 12 examples. During the experiment, a scrambled version of each fribble was generated online by applying

EP

a grid with a square side length of 25 × 25 pixels to the images and then randomly shuffling

AC C

the resulting squares.

3.1.4. Apparatus

The setup and apparatus were identical to Experiment 1.

3.1.5. Task and Procedure

19 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Figure 1C illustrates the design of Experiment 2. Each trial started with the presentation of a central fixation cross. After 500 ms of correct fixation (as controlled via eye-tracking), a series of image pairs was presented sequentially. Each pair consisted of an original fribble image on one side of the fixation cross, and a scrambled version of the same image presented

RI PT

on the opposite side. Images were presented on uniform grey background (CIE XYZ, 53/0/0), vertically centred on the screen, and located at 6.2° visual angle to the left and to the right of fixation. As in Experiment 1, each image pair was presented for a duration of 800 ms,

SC

followed by an inter-stimulus interval of 125 ms. In 50% of trials, four image pairs were presented (low-load condition) and in the other 50% six pairs were presented (high-load

M AN U

condition). The variable load (low vs. high) was randomly balanced across trials. Further, in 50% of trials the to-be memorized fribbles were presented on the left side, and in 50% on the right side of the fixation cross. Within each trial, the fribbles were always presented on the same side (either left or right) of the fixation cross (identical to the lateralized condition of

TE D

Experiment 1). Trials with left-and right lateralized presentations were counterbalanced and randomized across the experiment.

The last image pair was followed by a retention period of 1 s duration. During this period, a

EP

fixation cross was presented at the centre of the screen and rectangular place holders (i.e.,

AC C

frames) were presented in the same positions as the previously shown memory items. Following the retention period, a probe stimulus pair was presented. In 50% of trials the probe stimulus was one of the previously presented memory items (present before condition), and in 50% of trials it was a novel item taken from the same category (novel condition). This variable of target was randomized across the experiment. Further, in 50% of trials, the test item was presented on the same side, left or right, as during encoding phase (congruent condition). In the other half the positions of the test item and its scrambled version was 20 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT switched (incongruent condition). The variable of congruency was randomized across trials. Here, our intention was to disentangle temporal versus spatial visual working memory and demonstrate that different cognitive operations and strategies are underlying the performance during the two tasks. Specifically, we expected that the spatial congruency of the target

RI PT

would only affect the spatial task, but not necessarily the temporal task. Experiment 2 consisted of two different tasks, which were presented in a block-wise fashion. The order of blocks was counterbalanced across participants. In the temporal task, which was similar to

SC

Experiment 1 but also contained novel items in the recognition displays, participants had to indicate whether the probe stimulus was presented during the first or second half of the

M AN U

sequence, or whether it was not contained in the encoding displays (i.e., novel). In the spatial task, participants had to indicate whether the probe stimulus was presented on the left or right side during the encoding sequence, or whether it was novel. In both tasks, participants were asked to provide a speeded response via the buttons '2' and '8' of the number block of a

TE D

keyboard, using the left and right index fingers (mapping counter-balanced between participants). If participants identified the probe stimulus to be novel, they were asked to press the button ‘5’ on the keyboard’s number block. The probe item remained on the screen

EP

until a response was given.

AC C

In total, Experiment 2 comprised 16 conditions (2 loads × 2 presence/absence of encoded item × 2 congruency × 2 tasks), with 16 trials per condition, resulting in a total of 256 trials. Opportunities to rest were given after blocks of 64 trials. Prior to the start of each blocked task, participants practiced the respective task for 10 trials. During practice, trials included a visual feedback on the accuracy of the responses and were repeated if necessary.

3.1.6. EEG recording and data pre-processing 21 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT EEG recording and data pre-processing was performed identical to Experiment 1. On average, 9.2% of trials were rejected. A mean of 3.1 electrodes (SD = 2.8) was linearly interpolated, and a mean of 3.2 independent components (SD = 1.4) representing eye-

RI PT

movements was removed from the data.

3.1.7. Analysis of behavioural data

Prior to the statistical analysis, outlier trials with RTs deviating more than 2 SDs from the

SC

mean were excluded per participant and condition. Mean accuracy (percent correct) and RT measures (for correct trials only) were computed separately for each combination of

M AN U

conditions. For statistical analysis, accuracy data and RTs were each subjected to a repeatedmeasurements ANOVA, including the within-subject variables task (spatial vs. temporal), load (low vs. high), target (previously present vs. novel), and congruency (congruent vs.

3.1.8. EEG data analysis

TE D

incongruent).

The calculation of ERPs and the CDA for each condition was identical to Experiment 1.

EP

For statistical analysis, a repeated-measurements ANOVA was calculated, containing the

AC C

variables task (spatial vs. temporal), load (low vs. high), and hemisphere (ipsilateral vs. contralateral to the presentation side of the memory items). Data were pooled across the variables target and congruency, resulting in an average of 39.9 trials per condition.

3.2. Results 3.2.1. Behavioural results

22 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Figure 5 illustrates the behavioural results from Experiment 2. For mean correct RTs, we found a significant main effect of target, F(1, 17) = 11.0, p = .004, ηp2 = .41, due to faster RTs in the present before (M = 1,372 ms, SD = 381 ms) compared to the novel (M = 1,731 ms, SD = 687 ms) condition. Further, we found a significant interaction between the variables task

RI PT

and congruency, F(1, 17) = 8.9, p = .009, ηp2 = .36. Follow-up t-tests (with data aggregated over the variables target and load) indicated slower RTs in the incongruent compared to the congruent condition during the spatial task, t(17) = −2.59, p < .02, d = .08, but not during the

SC

temporal task (p = .068). No other main effect or interaction reached significance (largest nonsignificant p = .181).

M AN U

For accuracy data, the ANOVA revealed a significant main effect of task, F(1, 17) = 6.67, p = .019, ηp2 = .28, due to a higher number of correct responses in the spatial- (M = 71.5%, SD = 19.7%) compared to the temporal (M = 64.1%, SD = 21%) task. Further, we found a significant main effect of load, F(1, 17) = 13.68, p = .002, ηp2 = .45, due to higher accuracy in

TE D

the low- (M = 71.4%, SD = 19%) compared to the high-load condition (M = 64.2%, SD = 21.7%). Finally, we found a significant interaction between the variables task and target, F(1, 17) = 5.71, p = .029, ηp2 = .25. Follow-up t-tests (with data aggregated over the variables

EP

congruence and load) indicated less accurate responses for the present before compared to the

AC C

novel targets during the temporal task, t(17) = −2.55, p = .021, d = .8, but not during the spatial task (p = .838). No other main effect or interaction reached significance (largest nonsignificant p = .143).

--- insert Figure 5 about here ---

3.2.2. Event-related potentials 23 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Figures 6 and 7 illustrate the ERP results from Experiment 2. Amplitudes contralateral to the location of the memory items were consistently smaller compared to amplitudes ipsilateral. This resulted in a clear CDA for both task types (temporal and spatial) and both load

conditions are visible.

RI PT

conditions (low and high). However, no apparent differences between task types and load

Statistically, we found a significant main effect of hemisphere, F(1, 17) = 48.29, p = .000002, ηp2 = .74, due to smaller mean amplitudes at contralateral (M = −0.31 µV, SD = 3.61)

SC

compared to ipsilateral sites (M = 3.12 µV, SD = 4.22).

Similarly to Experiment 1, we found no overall effect of load (F(1, 17) = 0.05, p = .082, ηp2

M AN U

= .003, achieved power = 0.09) as well as no other main effects or interactions (largest nonsignificant p = .090).4 Analogous to Experiment 1, we investigated whether the CDA amplitude might change over time in response to each presented image pair. We calculated the average CDA amplitude separately for the first four memory displays, as the mean

TE D

amplitude at electrode sites contra- minus ipsilateral to the memory item, within a timewindow of 300 – 925 ms following the onset of each encoding display. These values were entered into a repeated measures three-way ANOVA, with the variables load (low vs. high),

EP

task (spatial vs. temporal), and time (first, second, third, fourth encoding display). Figure 7B, bottom row, illustrates the results of this comparison. We found a significant main effect of

AC C

time F(1, 17) = 43.70, p = .000001, ηp2 = .72, but no other effects (largest nonsignificant p = .10). As a follow-up, we conducted separate t-tests between each successive pair of memory displays (i.e., first vs second, second vs. third, third vs fourth) with data pooled across the variables load and task. This yielded a significant increase in amplitude between the first and second display, t(17) = 5.15, p = .00008, d = 0.27, between the second and third

24 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT display, t(17) = 4.34, p < .0005, d = 1.89, as well as between the third and fourth display, t(17) = 4.20, p < .0006, d = 1.53.

--- insert Figure 7 about here ---

SC

3.2.3. Discussion Experiment 2

RI PT

--- insert Figure 6 about here ---

In Experiment 2, we were able to replicate our main finding from Experiment 1, namely that

M AN U

memorizing the temporal order of sequentially presented visual items elicits a classic CDA component. Moreover, we directly compared the CDA resulting from a temporal order VWM task to that from a spatial VWM task. Importantly, the visual stimulation was identical for both conditions, and only the task instructions differed. Overall, participants provided

TE D

significantly more correct responses for the spatial- compared to the temporal task, indicating a difference in difficulty. Yet, our EEG results show highly similar CDAs for both tasks. As in Experiment 1, this null-finding might be the consequence of a too small trial number.

EP

Alternatively, however, the lack of difference between conditions could indicate that this

AC C

component reflects a very basic VWM process, common to the encoding and retention of various types of visual information. We calculated BF to compare evidence for or against the presence of a difference between CDA in spatial versus temporal VWM tasks. We found a BF of 0.247, suggesting no difference between the two conditions. Further, we replicated our finding of higher hit-rates for the low- compared to the high-load, as well as ruled out the potential confounds of verbal rehearsal and the ‘first half’ strategy.

25 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT While we did not observe an effect of load on the overall CDA amplitude (BF = 0.249), we again found increases in CDA amplitude within trials for each load condition (Figure 7B, bottom row). In other words, CDA amplitude increased with the additional presentation of encoding displays, up to four displays. Thus, we suggest that CDA amplitude in the present

RI PT

study is still affected by load, but within trials rather than between high- and low load conditions, possibly because a plateau was reached already with four items.

For accuracy data, the significant interaction between the variables task and target suggests

SC

that specifically during the temporal task, more correct responses were given when probes were novel compared to when they had been shown before. This makes sense, since two (in

M AN U

the low-load condition) or three (in the high-load condition) different temporal positions had to be classified as belonging to either the first or the second half of the sequence for a correct temporal order judgment, as compared to memorizing only two spatial positions (whether an item was presented on the left or the right) for the correct spatial judgment. As a further

TE D

interesting difference between the temporal and the spatial task, we observed an interaction in RTs between the factors task and congruency. In extending previous work on spatial congruency between memory encoding and retrieval (Sun & Gordon, 2009, 2010), this

EP

indicates that spatial congruency between the memory items and the probe is beneficial for a

AC C

spatial-, but not for a temporal order VWM task. Finally, we observed overall faster RTs to probe stimuli that were originally presented during the preceding memory sequence compared to novel ones. From this priming effect of already seen images, we conclude that the introduction of novel stimuli prevented participants from adopting a ‘first half’ strategy, and that participants indeed actively encoded the presented stimuli in both the temporal and the spatial task conditions.

26 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT In summary, Experiment 2 allowed us to replicate our findings from Experiment 1, while ruling out potential complications, and importantly, directly comparing the CDA during a

RI PT

temporal- compared to a spatial VWM task.

4. General Discussion

In the present study, we investigated whether the contralateral delay activity (CDA), a well-

SC

known marker for object-based or spatial visual working memory (VWM), would also reflect the encoding and retention of temporal-order information following sequentially presented

M AN U

visual items. As a main novel result, we observed a classic CDA component during such a temporal task, thereby extending the role of the CDA in its reflection of various VWM processes.

Previous studies (Adam et al., 2018; Luria et al., 2016; Vogel & Machizawa, 2004) have

TE D

reported the emergence of a CDA during the retention period of WM tasks, starting at around 300 ms after the offset of a single visual display. Using a variation of the previously used experimental design, our data show a CDA appearing after the first of several (four or six)

EP

encoding displays. This CDA then continues to track the encoding of the remaining items and

AC C

is sustained throughout the retention interval (Figures 4 and 7).3,4 In line with previous interpretations of the CDA, we suggest that in our study this component represents the sequential encoding and retention of each individual display. Further supporting this notion, we found increases in CDA amplitude in response to individual memory items over the course of a trial and up to around four items. Thus, while we did not find an overall difference in CDA amplitude between load conditions, we conclude that load still affected CDA amplitudes, but was more evident across time. Load effects on the CDA amplitude 27 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT might also be more pronounced if all relevant visual information needs to be encoded simultaneously via a single display. In other words, the present sequential processing of the stimuli could be a potential explanation for the lack of an overall load effect in the present data. Additionally, the fact that our set sizes were already at the average VWM capacity

RI PT

limits and beyond, load effects in the mean CDA across the entire sequence might be difficult to observe. More importantly, however, since low- and high-load trials were randomly intermixed, participants likely invested the same amount of encoding effort into both

SC

conditions, up to the end of the fourth memory display (after which the encoding period was either over, or two additional items were presented). It is compelling to see that amplitude,

M AN U

latency, and topography of our observed CDA responses did not differ between a task related to temporal versus spatial VWM (see Figure 7), and are highly similar in appearance to previously reported CDA components (Berggren & Eimer, 2016; Gao et al., 2011; Ikkai et al., 2010; Vogel & Machizawa, 2004). This was the case even though we found behavioural

TE D

differences between both tasks. A potential reason for this null-finding could be a low signalto-noise ratio, due to the comparably small number of trials entered into the ERP analysis. However, since our CDA amplitude did track increases in memory load over time, we

EP

suggest, that signal-to-noise ratio was adequate to capture potential differences between

AC C

conditions. Consequently, we suggest that the CDA observed here might reflect a general aspect of visual information encoding and maintenance, largely independent of the feature dimensions relevant for the current task. Along the same line, a recent meta-analysis has raised doubts about the common assumption of distinct WM subsystems for various WM contents, and suggests that the implementation of WM across different tasks of features might be more homogenous than previously thought (Morey, 2018).

28 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Our interpretation also relates to a currently ongoing debate regarding the underlying functional mechanisms of the CDA. Specifically, a recent study has raised doubts whether CDA actually reflects VWM load or rather the current focus of spatial attention (Berggren & Eimer, 2016). Berggren and Eimer’s (2016) participants were instructed to memorize items

RI PT

on two consecutive encoding displays rather than one, with the items being either located on the same side or on different sides in the first and second display. When items on the first and second display were presented on different sides, the amplitude of the CDA in response to the

SC

second display only reflected the number of items in the second display, regardless of how many items from the first display were already held in WM. This led the authors to suggest

M AN U

that CDA is more reflective of the current focus of spatial attention, rather than the total amount of items in VWM. Their criticism applies to virtually all previous observations of CDA, since the lateralized nature of CDA requires that only items in one hemifield (i.e., left or right) are being attended, encoded, and retained via VWM. Additionally, a CDA-like

TE D

component has been observed outside of working memory tasks during multiple-object tracking (Drew, Horowitz, Wolfe, & Vogel, 2012; Drew & Vogel, 2008), in which it reflects the number of lateralized visual items being tracked. However, most recent studies (Adam et

EP

al., 2018; Feldmann-Wüstefeld, Vogel, & Awh, 2018) on this issue offer further support for a

AC C

VWM account. For instance, Adam et al. (2018) compared CDA amplitudes between single trials resulting in a successful versus an unsuccessful VWM performance. They found that CDA amplitude differed between those trials, whereas established measures of visual attention, such as lateralized oscillatory alpha band activity (Pomper & Chait, 2017; Sauseng et al., 2005) or amplitude of the visually evoked P1 component did not. Furthermore, Feldmann-Wüstefeld and colleagues (2018) directly challenged the interpretation by Berggren and Eimer discussed above, by showing that a CDA reflecting all items currently 29 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT held in WM still emerges in the second of two consecutively presented displays, but only if the spatial locations of items within the first and second display are non-overlapping, and allow a mental combination of items into a single display. Our data support the WM account of CDA, since we found CDA amplitude to increase stepwise with each presented item, while

RI PT

spatial attention was (in all but the balanced condition in Experiment 1) constantly focussed on one hemifield, and, thus, should not have changed over the course of a trial (or should potentially even have decreased, as it would already be at the position where the next item in

SC

a sequence was presented and, hence, needed not to be shifted to this side again; this would be particularly true of Experiment 2 in which all items were always shown on one side). In

M AN U

the present data, we found no difference in CDA amplitude between a task in which the spatial versus a task in which the temporal information was important. While we are very cautious to interpret this null finding, we would have expected a stronger CDA amplitude modulation in the spatial compared to the temporal task, if CDA would indeed reflect spatial

TE D

attention rather than WM storage. Taken together, while the exact neural mechanisms reflected in CDA are still under debate, the majority of studies so far favour a VWM storagebased over a spatial attention-based account.

EP

Interestingly, while the neural responses in our present study are highly similar between the

AC C

temporal and the spatial task of Experiment 2, as well as the lateralized condition of Experiment 1, we found a number of intriguing differences in the behavioural results. First, in both experiments, accuracy was consistently higher for the low- compared to the high-load task, indicating that our load manipulation was effective, even though we did not find this effect reflected in the overall CDA amplitude. Accuracy was also significantly higher for the spatial compared to the temporal task in Experiment 2, again without an observable difference in the CDA. These results are unexpected, given the previously reported link 30 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT between individual performance and CDA amplitude (Adam et al., 2018; Ikkai et al., 2010; Vogel & Machizawa, 2004). Apart from the differences in the relevant visual feature dimension (here: temporal order and spatial location; in previous studies: object identity), the most likely reason for this inconsistency is the fact that items in the present study were

RI PT

presented sequentially over several seconds, whereas displays were shown for just around 100 ms in previous studies. Presumably, the present means of stimulus presentation allowed for more time and resources to be dedicated to the memory encoding, thus, changing the

SC

observable neural correlates of this process.

As a further behavioural finding, spatial congruency between the encoding and probe

M AN U

displays did speed up RTs, but only for the spatial (Experiment 2) and not the temporal task (Experiments 1 and 2). Thus, our study shows that whether or not contextual spatial encoding-retrieval congruency affects VWM performance might depend largely on the task and relevant feature dimension, with spatial congruency likely more relevant for a spatial

TE D

compared to a temporal VWM task. Additionally, it supports our notion that different cognitive operations are underlying the performance during temporal versus spatial visual working memory, despite the similarities between the respective CDAs. Finally, the observed

EP

interaction in accuracy between the variables task and target suggests that, specifically for the

AC C

temporal task, more correct responses were only given when probes were novel compared to when they had been shown before. This makes sense, since the correct classification of two to three different temporal positions per each order judgment is more demanding than the classification of only one position per each spatial judgment. In conclusion, our results from two independent experiments demonstrate the presence of the CDA in VWM for temporal order, related to sequentially presented items. While it is currently unclear whether the CDA in this context is more related to memory load or spatial 31 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT attention, our current finding of CDA amplitudes increasing over time with the number of items presented supports the former (i.e., memory load) account. Since the characteristics of CDA appear to be highly similar between different encoded feature dimension, we suggest it

RI PT

might reflect a basic, task-independent aspect of VWM operation.

5. Author contributions

TD and UA designed the experiment. TD collected the data. UP and TD analysed the data.

M AN U

6. Competing financial interests

SC

UP, TD and UA wrote the manuscript.

AC C

EP

TE D

The authors declare no competing financial interests.

7. References 32 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Adam, K. C. S., Robison, M. K., & Vogel, E. K. (2018). Contralateral delay activity tracks fluctuations in working memory performance. Journal of Cognitive Neuroscience, 1–12. https://doi.org/10.1162/jocn Adam, K. C. S., Vogel, E. K., & Awh, E. (2017). Clear evidence for item limits in visual

RI PT

working memory. Cognitive Psychology, 97, 79–97. https://doi.org/10.1016/j.cogpsych.2017.07.001

Averbeck, B., & Lee, D. (2007). Prefrontal neural correlates of memory for sequences.

SC

Journal of Neuroscience, 27(9), 2204–2211. http://dx.doi.org/10.1523/JNEUROSCI.4483-06.2007.

M AN U

Baddeley, A. (2003). Working memory: Looking back and looking forward. Nature Reviews Neuroscience 4(10), 829-839. doi:10.1038/nrn1201

Baddeley, A., & Hitch, G. (1974). Working memory. Recent Advances in Learning and Motivation, 8, 47-89.

TE D

Balaban, H., & Luria, R. (2015). The number of objects determines visual working memory capacity allocation for complex items. NeuroImage, 119, 54–62. https://doi.org/10.1016/j.neuroimage.2015.06.051

EP

Barry, T. J., Griffith, J. W., De Rossi, S., & Hermans, D. (2014). Meet the fribbles: Novel

AC C

stimuli for use within behavioural research. Frontiers in Psychology, 5: 103. https://doi.org/10.3389/fpsyg.2014.00103 Berggren, N., & Eimer, M. (2016). Does contralateral delay activity reflect working memory storage or the current focus of spatial attention within visual working memory? Journal of Cognitive Neuroscience, 28(12), 2003-2020. https://doi.org/10.1162/jocn_a_01019 Brady, T. F., Störmer, V. S., & Alvarez, G. A. (2016). Working memory is not fixedcapacity: More active storage capacity for real-world objects than for simple stimuli. 33 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Proceedings of the National Academy of Sciences of the United States of America, 113(27), 7459–7464. https://doi.org/10.1073/pnas.1520027113 Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10(4), 433-436. https://doi.org/10.1163/156856897X00357

RI PT

Burgess, N., & Hitch, G. J. (1999). Memory for serial order: A network model of the phonological loop and its timing. Psychological Review, 106(3), 551-581. http://dx.doi.org/10.1037/0033-295X.106.3.551

SC

Cornelissen, F. W., Peters, E. M., & Palmer, J. (2002). The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox. Behavior Research Methods,

M AN U

Instruments, & Computers, 34(4), 613–617. https://doi.org/10.3758/BF03195489 Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-114. https://doi.org/10.1017/S0140525X01003922

TE D

Delogu, F., Nijboer, T. C. W., & Postma, A. (2012a). Binding “when” and “where” impairs temporal, but not spatial recall in auditory and visual working memory. Frontiers in Psychology, 3:62. https://doi.org/10.3389/fpsyg.2012.00062

EP

Delogu, F., Nijboer, T. C. W., & Postma, A. (2012b). Encoding location and serial order in

AC C

auditory working memory: Evidence for separable processes. Cognitive Processing, 13(3), 267-276. https://doi.org/10.1007/s10339-012-0442-3 Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of singletrial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009 Drew, T., Horowitz, T. S., Wolfe, J. M., & Vogel, E. K. (2012). Neural measures of dynamic changes in attentive tracking load. Journal of Cognitive Neuroscience, 24(2), 440–450. 34 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT doi: 10.1162/jocn_a_00107 Drew, T., & Vogel, E. K. (2008). Neural measures of individual differences in selecting and tracking multiple moving objects. Journal of Neuroscience, 28(16), 4183-4191 https://doi.org/10.1523/JNEUROSCI.0556-08.2008

RI PT

Feldmann-Wüstefeld, T., Vogel, E. K., & Awh, E. (2018). Contralateral delay activity

indexes working memory storage, not the current focus of spatial attention. Journal of Cognitive Neuroscience, 25, 1-11. https://doi.org/10.1162/jocn

SC

Fukuda, K., Vogel, E., Mayr, U., & Awh, E. (2010). Quantity, not quality: The relationship between fluid intelligence and working memory capacity. Psychonomic Bulletin &

M AN U

Review, 17(5), 673–679. https://doi.org/10.3758/17.5.673

Gao, Z., Xu, X., Chen, Z., Yin, J., Shen, M., & Shui, R. (2011). Contralateral delay activity tracks object identity information in visual short term memory. Brain Research, 1406, 30–42. https://doi.org/10.1016/j.brainres.2011.06.049

TE D

Glenberg, A. M., & Swanson, N. G. (1986). A temporal distinctiveness theory of recency and modality effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12(1), 3-15. http://dx.doi.org/10.1037/0278-7393.12.1.3

EP

Gmeindl, L., Walsh, M., & Courtney, S. M. (2011). Binding serial order to representations in

AC C

working memory: A spatial/verbal dissociation. Memory and Cognition, 39(1), 37–46. https://doi.org/10.3758/s13421-010-0012-9 Heusser, A. C., Poeppel, D., Ezzyat, Y., & Davachi, L. (2016). Episodic sequence memory is supported by a theta – gamma phase code. Nature Neuroscience, 19, 1374–1380. https://doi.org/10.1038/nn.4374 Ikkai, A., McCollough, A. W., & Vogel, E. K. (2010). Contralateral delay activity provides a neural measure of the number of representations in visual working memory. Journal of 35 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Neurophysiology, 103(4), 1963–1968. https://doi.org/10.1152/jn.00978.2009 Johansson, R., & Johansson, M. (2014). Look here, eye movements play a functional role in memory retrieval. Psychological Science, 25(1), 236-42. https://doi.org/10.1177/0956797613498260

RI PT

Kappenman, E. S., & Luck, S. J. (2010). The effects of electrode impedance on data quality and statistical significance in ERP recordings. Psychophysiology, 47(5), 888–904. https://doi.org/10.1111/j.1469-8986.2010.01009.x

SC

Katus, T., & Müller, M. M. (2016). Working memory delay period activity marks a domainunspecific attention mechanism. Neuroimage, 128, 149-157.

M AN U

https://doi.org/10.1016/j.neuroimage.2015.12.051

Lee, T.-W., Girolami, M., & Sejnowski, T. J. (1999). Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation, 11(2), 417-41. https://doi.org/10.1162/089976699300016719

TE D

Li, C., He, X., Wang, Y., Hu, Z., & Guo, C. (2017). Visual working memory capacity can be increased by training on distractor filtering efficiency. Frontiers in Psychology, 8:196. https://doi.org/10.3389/fpsyg.2017.00196

EP

Lisman, J. (2009). Working memory: The importance of theta and gamma oscillations.

AC C

Current Biology, 20(11), R490–R492. https://doi.org/10.1016/j.cub.2010.04.011 Lisman, J. E., & Jensen, O. (2013). The theta-gamma neural code. Neuron, 77(6), 1002– 1016. https://doi.org/10.1016/j.neuron.2013.03.007 Luck, S. J., & Vogel, E. K. (2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Science, 17(8), 391– 400. https://doi.org/10.1016/j.tics.2013.06.006. Luria, R., Balaban, H., Awh, E., & Vogel, E. K. (2016). The contralateral delay activity as a 36 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT neural measure of visual working memory. Neuroscience and Biobehavioral Reviews, 62, 100–108. doi: 10.1016/j.neubiorev.2016.01.003. Manohar, S. G., Pertzov, Y., & Husain, M. (2017). Short-term memory for spatial, sequential and duration information. Current Opinion in Behavioral Sciences, 17, 20–26.

RI PT

https://doi.org/10.1016/j.cobeha.2017.05.023

Marshuetz, C. (2005). Order information in working memory: An integrative review of evidence from brain and behavior. Psychological Bulletin. 131(3):323-39.

SC

https://doi.org/10.1037/0033-2909.131.3.323

Marshuetz, C., & Smith, E. E. (2006). Working memory for order information: Multiple

M AN U

cognitive and neural mechanisms. Neuroscience. 139(1):195-200 https://doi.org/10.1016/j.neuroscience.2005.08.024

Marshuetz, C., Smith, E. E., Jonides, J, DeGutis, J., & Chenevert, T. L. (2000). Order information in working memory: fMRI evidence for parietal and prefrontal

TE D

mechanisms. Journal of Cognitive Neuroscience, 12, 130–144. 10.1162/08989290051137459

Morey, C. C. (2018). The case against specialized visual-spatial short-term memory.

EP

Psychological Bulletin, 144(8), 849-883. https://doi.org/10.1037/bul0000155

AC C

Ninokura, Y., Mushiake, H., & Tanji, J. (2003). Representation of the temporal order of visual objects in the primate lateral prefrontal cortex. Journal of Neurophysiology,

89(5), 2868-2873. http://dx.doi.org/10.1152/jn.00647.2002

Ninokura, Y., Mushiake, H., & Tanji, J. (2004). Integration of temporal order and object information in the monkey lateral prefrontal cortex. Journal of Neurophysiology, 91(1), 555–560. http://dx.doi.org/10.1152/jn.00694.2003. Oostenveld, R., Fries, P., Maris, E., & Schoffelen, J. M. (2011). FieldTrip: Open source 37 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience, 2011:156869. doi: 10.1155/2011/156869 Perrin, F., Pernier, J., Bertrand, O., & Echallier, J. F. (1989). Spherical splines for scalp potential and current density mapping. Electroencephalography and Clinical

RI PT

Neurophysiology, 72(2), 184–187.

Pomper, U., & Chait, M. (2017). The impact of visual gaze direction on auditory object

tracking. Scientific Reports, 7(1):4640. https://doi.org/10.1038/s41598-017-04475-1

SC

Postle, B. R. (1997). Spatial, but not object, delayed response is impaired in early Parkinson’s disease. Neuropsychology, 11(2), 171-179. https://doi.org/10.1037/0894-4105.11.2.171

M AN U

Postle, B. R. (2015). The cognitive neuroscience of visual short-term memory. Current Opinion in Behavioral Sciences, 1, 40-46. https://doi.org/10.1016/j.cobeha.2014.08.004 Rensink, R. A. (2000). Seeing, sensing, and scrutinizing. Vision Research, 40(10-12), 146987. https://doi.org/10.1016/S0042-6989(00)00003-1

TE D

Roberts, B. M., Libby, L.A., Inhoff, C., & Ranganath, C. (2018). Brain activity related to working memory for temporal order and object information. Behavioural Brain Research, 354, 55-63. http://dx.doi.org/10.1016/j.bbr.2017.05.068

EP

Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests

AC C

for accepting and rejecting the null hypothesis. Psychonomic Bulletin and Review, 16(2), 225-237. https://doi.org/10.3758/PBR.16.2.225 Sauseng, P., Klimesch, W., Stadler, W., Schabus, M., Doppelmayr, M., Hanslmayr, S., … Birbaumer, N. (2005). A shift of visual spatial attention is selectively associated with human EEG alpha activity. European Journal of Neuroscience, 22(11), 2917-26. https://doi.org/10.1111/j.1460-9568.2005.04482.x Scholz, A., Klichowicz, A., & Krems, J. F. (2018). Covert shifts of attention can account for 38 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT the functional role of “eye movements to nothing.” Memory and Cognition, 46(2), 230243. https://doi.org/10.3758/s13421-017-0760-x Shipstead, Z., Redick, T. S., Hicks, K. L., & Engle, R. W. (2012). The scope and control of

https://doi.org/10.1080/09658211.2012.691519

RI PT

attention as separate aspects of working memory. Memory, 20(6), 608-628.

Sternberg, S. (1967). Retrieval of contextual information from human memory. Psychonomic Science, 8, 55–56. https://doi.org/10.3758/BF03330664

SC

Sun, H.-M., & Gordon, R. D. (2009). The effect of spatial and nonspatial contextual information on visual object memory. Visual Cognition, 17(8), 1259-1270.

M AN U

https://doi.org/10.1080/13506280802469510

Sun, H. M., & Gordon, R. D. (2010). The influence of location and visual features on visual object memory. Memory and Cognition, 38(8), 1049-57. https://doi.org/10.3758/MC.38.8.1049

TE D

Tresch, M. C., Sinnamon, H. M., & Seamon, J. G. (1993). Double dissociation of spatial and object visual memory: Evidence from selective interference in intact human subjects. Neuropsychologia, 31(3), 211-9. https://doi.org/10.1016/0028-3932(93)90085-E

EP

Unsworth, N., Fukuda, K., Awh, E., & Vogel, E. K. (2014). Working memory and fluid

AC C

intelligence: Capacity, attention control, and secondary memory retrieval. Cognitive Psychology, 71, 1-26. https://doi.org/10.1016/j.cogpsych.2014.01.003 Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts individual differences in visual working memory capacity. Nature, 428(6984), 748–751. https://doi.org/10.1038/nature02447 Zhang, D. R., Li, Z. H., Chen, X. C., Wang, Z. X., Zhang, X. C., Meng, X. M., …, Hu, X. P. (2003). Functional comparison of primacy, middle and recency retrieval in human 39 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT auditory short-term memory: An event-related fMRI study. Cognitive Brain Research,

8. Footnotes 1

RI PT

16, 91–98. DOI: 10.1016/S0926-6410(02)00223-9

Bicycles, cakes, cameras, cars, chairs, guitars, accordions, helicopters, keys, leaves,

sun glasses, toys, vases, watches, wine glasses.

Note that congruence versus incongruence was regarding the spatial location of the item

M AN U

2

SC

mushrooms, pastries, pipes, sandwiches, scissors, safes, shelves, ships, shoes, stuffed toys,

between encoding and retrieval displays. Thus, this variable has no effect on the CDA component, which is observed in the time-window between encoding and retrieval.

As explained in the Methods section, CDA amplitudes for statistical comparisons were

TE D

3

averaged from 300 ms following the onset of the first memory item until the end of the retention interval. We chose this analysis time-window, since VWM constantly needs to be

EP

updated throughout the encoding interval, and preceding items need to be retained during the

AC C

presentation of subsequent items. Thus, any neural process related to the retention of VWM should be present throughout our selected time-window. However, to investigate the presence of a retention-related CDA in absence of encoding processes, we reran our statistical analyses for activity averaged over the 1-s retention interval only. Doing so revealed essentially the same effects as when using the original longer time-window. We found a significant main effect of hemisphere, F(1, 11) = 17.2, p = .002, ηp2 = .54, due to smaller mean amplitudes contralateral to the side where encoded items were presented. Next, we found a significant 40 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT main effect of lateralization, F(1, 11) = 6.37, p = .028, ηp2 = .39, as a result of larger ERP amplitudes in the lateralized compared to the balanced condition. Lastly, the interaction between the variables hemisphere and lateralization observed in the original analysis just failed significance, F(1, 11) = 3.04, p = .071, ηp2 = .27, but was in the same direction as

RI PT

before. No other main effect or interaction reached significance (largest nonsignificant p = .22).

As for Experiment 1, we re-ran our statistical analyses using CDA amplitudes averaged

SC

4

over the retention interval only. We found the same significant main effects and interactions

M AN U

as when using the original longer time-window. We found a significant main effect of hemisphere, F(1, 17) = 61.99, p = .0000005, ηp2 = .78, due to smaller mean amplitudes at contralateral compared to ipsilateral sites. Additionally, we found an interaction between the variables task and load, F(1, 17) = 8.92, p = .008, ηp2 = .34 . This interaction indicated that

TE D

CDA in the low-load condition was larger for the temporal compared to the spatial task, while in the high-load condition it was larger for the spatial compared to the temporal task.

AC C

EP

No other main effect or interaction reached significance (largest nonsignificant p = .149).

9. Figures

41 Contralateral delay activity during order memory

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

Figure 1: Stimuli and experimental design. (A) Example of an object stimulus (chair category) from Experiment 1 (left), a fribble stimulus from Experiment 2 (middle), and a scrambled image (right). (B) Trial design from Experiment 1. During an encoding period, participants were presented with either six (high-load condition, left timeline) or four (lowload condition, right timeline) encoding displays. Images within each trial were taken from the same category. Each display was presented for 800 ms, with an inter-stimulus interval (ISI) of 125 ms. Following a retention period of 1 s, a probe display was presented, 42 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT containing one of the previously presented memory items. Participants had to indicate whether the item was originally presented in the first or second half of the encoding period. In the example shown, the left timeline shows a trial from the balanced condition (memory items presented on both left and right sides of the display), whereas the right timeline shows a

RI PT

trial from the lateralized condition (memory items presented on one side of the display throughout the trial). (C) Trial design from Experiment 2. As in Experiment 1, participants were presented with either six (high-load condition, left timeline) or four (low-load condition,

SC

right timeline) encoding displays during an initial encoding period. Each display was presented for 800 ms, with an inter-stimulus interval of 125 ms. Following a retention period

M AN U

of 1 s, a probe display was presented. The probe stimulus was either one of the previously presented memory items, or a novel one. During the temporal task, participants had to indicate whether the probe item was originally presented in the first or second half of the encoding period (i.e., the same task as in Experiment 1). During the spatial task, participants

TE D

had to indicate whether the probe item was originally presented on the left or right side of the display. In the example shown, the left timeline shows a trial from the congruent condition (probe item presented on the same side as the original encoded item), whereas the right

EP

timeline shows a trial from the incongruent condition (probe item presented on the opposite

AC C

side of the original encoded item). Arrows depict the flow of time.

43 Contralateral delay activity during order memory

SC

RI PT

ACCEPTED MANUSCRIPT

M AN U

Figure 2: Behavioural results Experiment 1. Left barplot: accuracy, right barplot: response times. Data are shown separately for the low-load (blue colour) and high-load (red colour) conditions, for trials with spatially congruent (Con) and incongruent (Inc) probe locations, as

AC C

EP

TE D

well as for blocks with stimulus location being lateralized or balanced.

44 Contralateral delay activity during order memory

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

Figure 3: Event-related potentials from Experiment 1. ERPs from contra- and ipsilateral electrodes (PO7/PO8) are shown separately for the low-load (left column) and high-load (right column) conditions, as well as for the lateralized (top row) and balanced (bottom row) conditions. ERPs from an electrode contralateral to the presentation side of encoded items are shown in red, whereas ERPs from an ipsilateral electrode are shown in blue. In balanced conditions, contra- and ipsilateral were defined randomly. The difference between the two

45 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT (i.e., the contralateral delay activity) is shown in black. Grey bars indicate the retention time-

AC C

EP

TE D

M AN U

SC

RI PT

interval.

46 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Figure 4: Contra-lateral delay activity from Experiment 1. (A): Traces of the contralateral-delay activity (CDA) from the lateralized (red hue) and balanced (blue hue) conditions, and both the low-load (low colour intensity) and high-load (high colour intensity) conditions. (B) Top: Topographic representation of the CDA, averaged from 300 ms after

RI PT

beginning of the encoding period, until the end of the retention period. Topographies are calculated by subtracting activity from trials with memory items presented on the left from trials with memory items presented on the right. Bottom: Changes in CDA amplitude in

SC

response to increasing numbers of memory items. Mean CDA amplitudes were calculated separately during presentation of each of the encoding displays, and are shown for low- and

AC C

EP

TE D

M AN U

high load, as well as lateralized and balanced conditions.

47 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Figure 5: Behavioural results Experiment 2. Left barplot: accuracy, right barplot: response times. Data are shown separately for the spatial and temporal task, the low-load (blue colour) and high-load (red colour) load conditions, for trials with spatially congruent (Con) and incongruent (Inc) probe locations, as well as for trials with previously present and novel

AC C

EP

TE D

M AN U

SC

RI PT

targets.

Figure 6: Event-related Potentials from Experiment 2. ERPs from contra- and ipsilateral electrodes (PO7/PO8) are shown separately for the lowload (left column) and high-load (right column) conditions, as well as for the spatial (top 48 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT row) and temporal (bottom row) task. ERPs from an electrode contralateral to the presentation side of memory items are shown in red, whereas ERPs from an ipsilateral electrode are shown in blue. The difference between the two (i.e., the contralateral delay

AC C

EP

TE D

M AN U

SC

RI PT

activity) is shown in black. Grey bars indicate the retention time-interval.

49 Contralateral delay activity during order memory

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

Figure 7: Contra-lateral delay activity from Experiment 2. (A): Traces of the contralateral-delay activity (CDA) from the spatial (red hue) and temporal (blue hue) task, 50 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT and both the low-load (low colour intensity) and high-load (high colour intensity) conditions. (B) Top Topographic representation of the CDA, averaged from 300 ms after beginning of the encoding period, until the end of the retention period. Topographies are calculated by subtracting activity from trials with memory items presented on the left from trials with

RI PT

memory items presented on the right. Bottom: Changes in CDA amplitude in response to increasing numbers of memory items. Mean CDA amplitudes were calculated separately during presentation of each of the encoding displays, and are shown for low- and high load,

AC C

EP

TE D

M AN U

SC

as well as spatial and temporal conditions.

51 Contralateral delay activity during order memory

ACCEPTED MANUSCRIPT Contralateral delay activity is an EEG wave linked to visual-spatial working memory We show that this component can also be observed during temporal order working memory Further, its amplitude increases over time along with the number of retained items

AC C

EP

TE D

M AN U

SC

RI PT

We suggest it is a general marker of visual working memory, independent of spatial attention