Choice-predictive activity in parietal cortex during source memory decisions

Accepted Manuscript Choice-predictive activity in parietal cortex during source memory decisions Roberto Guidotti, Annalisa Tosoni, Mauro Gianni Perrucci, Carlo Sestieri PII:

S1053-8119(19)30077-1

DOI:

https://doi.org/10.1016/j.neuroimage.2019.01.071

Reference:

YNIMG 15596

To appear in:

NeuroImage

Received Date: 27 September 2018 Revised Date:

16 January 2019

Accepted Date: 28 January 2019

Please cite this article as: Guidotti, R., Tosoni, A., Perrucci, M.G., Sestieri, C., Choice-predictive activity in parietal cortex during source memory decisions, NeuroImage (2019), doi: https://doi.org/10.1016/ j.neuroimage.2019.01.071. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT Choice-predictive activity in parietal cortex during source memory decisions Roberto Guidotti1, Annalisa Tosoni1, Mauro Gianni Perrucci1 & Carlo Sestieri1 1

SC

AC C

EP

TE D

M AN U

Corresponding author: Carlo Sestieri ITAB Institute for Advanced Biomedical Technologies Department of Neuroscience, Imaging and Clinical Sciences G. d’Annunzio University of Chieti-Pescara 66100 Chieti, Italy Phone: +39 0871 3556951 E-mail: [email protected]

RI PT

ITAB Institute for Advanced Biomedical Technologies, Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti-Pescara, 66100 Chieti, Italy

1

ACCEPTED MANUSCRIPT

SC

RI PT

Abstract Neurobiological research has classically focused on perceptual decision-making, although many real-life decisions are based on information that is not currently available but stored in long-term memory. Previous studies have suggested that the lateral parietal cortex encodes decision-related signals during item recognition judgments. In the present fMRI study, we employed a parametric manipulation of evidence for source memory judgments and tested several hypotheses concerning memory decision signals in parietal cortex. As expected, the mean BOLD signal in several parietal regions was modulated by decision evidence. An analysis of the locally distributed pattern of activity, moreover, identified a parietal cluster showing significant choice-predictive activity even at the lowest level of decision evidence, with decoding accuracy that increased as a function of evidence. Decoding patterns were consistent across subjects as shown by a leave-onesubject-out classification analysis. Finally, we found that the pattern of choice-predictive activity in parietal lobe was temporally correlated with that observed in medial temporal regions traditionally associated with long-term memory functions. The present findings are consistent with a general role of lateral parietal regions located around the intraparietal sulcus in representing a decision variable for memory-based decisions.

M AN U

Keywords episodic memory, source memory, decision-making, parietal lobe, fMRI, MVPA

AC C

EP

TE D

Funding This study was supported by a grant from University G. d’Annunzio (ex 60%) to C.S.

2

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

1 Introduction Research on the neural mechanisms supporting decision-making in humans and primates has primarily focused on perceptual decisions. However, many real-life decisions are also based on information stored in long-term memory (Sestieri et al., 2017; Shadlen and Kiani, 2013). Choosing a particular restaurant based on our previous experience is an example of how episodic memory informs decision-making. The interest on the neural signals underlying memory-based decisions originated from the observation that activity in lateral parietal cortex appears to reflect the subjective, rather than the objective, memory status (seen vs. unseen) of an item (Kahn et al., 2004; Wheeler and Buckner, 2003), as if activity in this region were determining the outcome of the memory judgment. In line with influential neurobiological models of perceptual decisions, the "mnemonic accumulator" hypothesis (Wagner et al., 2005) have proposed that parietal activity during memory retrieval reflects the neural implementation of a diffusion process, in which decision evidence is accumulated over time from a starting point to a decision bound (Ratcliff, 1978; Smith and Ratcliff, 2004). Consistent with the hypothesis, we have recently shown that the activity in a parietal area follows the parametric manipulation of evidence for old decisions during old/new recognition judgments (Sestieri et al., 2014). According to a recent model (Sestieri et al., 2017), this area belongs to a series of regions located along the intraparietal sulcus that contribute to memory-based decision-making and manipulation of retrieved information, and differs from more ventral and posterior parietal regions associated with the representation of retrieved information. Despite the mounting evidence for the involvement of the lateral parietal cortex in item recognition decisions, there are still several outstanding issues concerning the neural bases of memory-based decision-making. A first question concerns the role of lateral parietal regions in representing a decision variable during other forms of memory decisions, such as those involving the evaluation of specific aspects of the encoding context (Mitchell and Johnson, 2009). The neuropsychological literature has classically associated deficits of source memory with lesions of medial temporal (Mayes et al., 2007) and frontal (Janowsky et al., 1989) cortices, but neuroimaging studies have frequently reported the involvement of the parietal lobe in source monitoring tasks (reviewed in (Mitchell and Johnson, 2009)). In the present study we tested whether all, or a subset, of these regions track decision evidence, defined as the amount of information favoring one of two possible alternatives, also for source memory decisions. Secondly, despite the manipulation of stimulus difficulty is often used to identify decision-related activity in perceptual studies (Heekeren et al., 2008), the effect of decision evidence in source memory tasks could be also interpreted on the basis of other accounts, e.g. differential amount of retrieved information (Guerin and Miller, 2011). Thus, the decision-making account must be supported by an additional index of decision-related activity that is not solely based on the amount of decision evidence, such as the demonstration of choice-predictive locally distributed signals (Hebart et al., 2012; Rissman and Wagner, 2012). Specifically, regions that represent a decision variable should exhibit choice-predictive activity also in the most difficult condition, which is characterized by many source misattributions, and scale with the level of evidence for the source memory decision, reflecting the increasing difference in the local spatial distribution of activity for the two outcomes. In this respect, our approach differs considerably from previous studies that demonstrated successful decoding of memory content in parietal areas (Bonnici et al., 2016; Kuhl and Chun, 2014; Xiao et al., 2017). In particular, these studies required the recall of specific information associated with a particular cue, with no forced choice between two alternatives, and focused on the reactivation of the encoding pattern during recall, likely emphasizing the contribution of regions associated with the representation of retrieved information. 3

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

Thirdly, recent findings suggest that the spatial distribution of retrieval-related activity in highlevel cortical areas is similar across subjects (Chen et al., 2017; Kragel and Polyn, 2016; Richter et al., 2016; Rissman et al., 2010), but so far there is no evidence that choice-predictive signals tracking decision evidence show a similar spatial organization across subjects. Significant acrosssubject decoding would not only demonstrate the spatial consistency of choice-predictive voxels but also imply a similar representation of memory choices across individuals. Finally, according to the mnemonic accumulator hypothesis, regions involved in the representation of memory decision variables should read/integrate mnemonic information from regions of the medial temporal lobe (MTL) traditionally associated with memory retrieval (Squire, 1992). This communication should be supported by a significant inter-regional interaction. However, evidence for such MTL-parietal functional coupling during memory decision-making is scarce (Cabeza et al., 2011; Geib et al., 2017; Kuhl et al., 2013). To address these issues, we used univariate and multivariate analyses of fMRI activity during a source memory task that involved the manipulation of the amount of evidence for deciding whether an object image had been previously encoded together with face or place images. This was obtained through the manipulation of study repetitions, i.e. the number of stimulus exposures in the encoding phase, based on the assumption that the number of study repetitions provides graded evidence for a source memory decision (in analogy with the effect of sensory evidence, i.e., motion coherence, for motion direction discrimination in perceptual decisions (Gold and Shadlen, 2007).

EP

TE D

2 Methods. 2.1 Subjects. Participants gave informed consent in accordance with guidelines set by the Human Studies Committee of G. D’Annunzio Chieti University (protocol #1007, approved on March 18th, 2016). Nineteen healthy right-handed subjects (10 males, mean age 25 ± 3 years) participated in the study. One subject was excluded from the analysis because his behavioral performance was at chance level. The study included a behavioral encoding session in the morning (duration: approx. 1 h) followed by an fMRI retrieval session in the afternoon (duration: approx. 2 h). The two sessions were separated by a 4 h interval. Subject received a monetary reimbursement (50 euro) for their participation and competed for an extra bonus (50 euro) that was assigned to the subject showing the best performance.

AC C

2.2 Stimuli. Stimuli consisted of 256×256 pixel color photographs depicting objects, human faces and scenes selected from existing databases and, occasionally, downloaded from the web. Images of objects (N=558, 480 images for the main experiment, 78 images for practice and instructions) were mainly selected from the database used by Brady and colleagues (Brady et al., 2008). Face objects [N=276 (240+36)] were selected from the Color FERET database (Phillips et al., 1998). Place images [N=276 (240+36)] were mainly selected from the SUN397 database (Xiao et al., 2010) and a similar (Konkle et al., 2010) database. 2.3 Apparatus. The behavioral encoding session was performed in a testing room. Images were presented on a 17' LCD computer monitor (1024×768 pixels, 60 Hz refresh rate) at a distance of 60 cm. Participants responded using a Lumina RB-830 Response Pad. In the fMRI retrieval session, visual stimuli were projected on a screen located at the head of the magnet bore via a LCD projector and viewed through a mirror attached to the head coil. Subjects wore MRI-compatible earphones and 4

ACCEPTED MANUSCRIPT responded using a Cedrus Lumina LU400 fiber optic Response Pad. Stimuli were presented using EPrime 2.0 software (Psychology Software Tools).

TE D

M AN U

SC

RI PT

2.4 Paradigm. 2.4.1 Encoding Session. The experimental design of the encoding session is illustrated in figure 1A. Participants were asked to associate a picture of an object with one of two simultaneously presented images of faces (object-face condition) or places (object-place conditions). The two conditions were presented in 10 alternating blocks, each including 96 trials, starting with an object-face block. Each trial started with the presentation of an object at the top-center of the screen for 2s. The image remained on the screen for other 1.5s while two face (or place) images were presented on the bottom-left and bottom-right of the screen. Subjects were asked to choose one of the two face (or place) images they believe was more strongly associated with the object, by pressing the corresponding key on a response pad (right index or middle finger). They were told that the judgment was entirely subjective. A fixed 0.2s ITI preceded the following trial. The primary manipulation consisted in the frequency of object presentations, a procedure known to increase the memory strength of the repeated item (Criss et al., 2013; Ratcliff, 1978; Sestieri et al., 2014). The rationale was that repeating the association between an object and a given class of stimuli (face or place) at encoding should produce a corresponding increase of the amount of evidence for a subsequent source memory judgment concerning the same association. Accordingly, objects could be presented once (1x), twice (2x) or three times (3x), always in the same condition. Face and place images changed across repetitions. The same objects could not be presented multiple times in the same block and each block included 16 1x trials, 32 2x trials and 48 3x trials. The experimental session was preceded by a practice session including 72 trials. To avoid the use of individual encoding strategies, subjects were not informed in advance about the nature of the following session.

AC C

EP

2.4.2 Retrieval Session. The retrieval session was administered inside the MR scanner. Participants were asked to indicate whether each presented object had been associated with a face or a place image during the previous encoding session (Fig. 1B). Subjects performed two versions of the task in different, alternating blocks. In the delay paradigm, subjects were asked to wait until a go signal to report their memory decisions whereas in a speeded paradigm they were asked to report the decision as soon as it was formed. In the present manuscript, the speeded version of the task only provides data for behavioral analyses but not for fMRI analyses, which are beyond the scope of the article. The assignment of an object to a particular encoding (face or place) or retrieval (delayed, speeded) condition was counterbalanced across subjects, while each object could be randomly assigned to a pair of faces or places in the corresponding condition. Each trial of the delay paradigm started with the presentation of a picture of an object at the center of the screen for 1.2s, followed by a delay of 7.3s duration, in which a gray central fixation cross was presented. The cross then turned red for 1s, instructing subjects to report the decision by pressing the corresponding key with their right hand. The go signal was followed by a variable ITI (2.4-5.8s) that preceded the next trial. This procedure allowed us to isolate decision-related from motor-related activity (Sestieri et al., 2014; Tosoni et al., 2008). The paradigm differs from traditional source memory tasks (e.g. (Hutchinson et al., 2014; Kahn et al., 2004)) in that no lures were presented at retrieval, intrinsically informing subjects that all trials were old and forcing them to always provide a source memory decision. Also in this respect, the paradigm follows an analogy with perceptual decision paradigms, where decisions usually take the form of stimulus detection or discrimination, but not both. The rationale for using this paradigm was based on the 5

ACCEPTED MANUSCRIPT intention to avoid consecutive or three-choice decisions that would complicate the interpretation of BOLD activity specifically associated with recognition and source attribution. The association between the memory decision (face, place) and the response key (index, middle finger) was provided at the beginning of the experiment and was counterbalanced across subjects. Participants performed 240 trials (40 for each experimental condition), divided in 8 runs, and received a feedback of their performance at the end of each run.

TE D

M AN U

SC

RI PT

2.5. Behavioral analyses. To test whether the manipulation of study repetitions effectively induced the expected modulation of evidence accumulation, which corresponds to the drift rate in the diffusion model framework, we fitted the Ratcliff diffusion model (Ratcliff and McKoon, 2008) to behavioral data using the Diffusion Model Analysis Toolbox (Vandekerckhove and Tuerlinckx, 2008). The model was applied to data obtained in the speeded version of the task, which, unlike the delayed version, also provided a measure of reaction times. For simplicity, we collapsed over source type and focused on the effect of stimulus repetition. As in previous studies (Philiastides et al., 2011; Sestieri et al., 2014) we used a two-step approach. Firstly, we selected a unique model parameterization that best fitted the data by comparing the fit of several model parameterizations using data from all subjects (meta-subject step). Sixteen model parameterizations were compared: all the possible combination of increasing size of four parameters [drift rate (v), boundary separation (a), starting point (z), and non-decision time (Ter)], a model in which all parameters were fixed, and one in which all parameters were allowed to freely vary across conditions. Model selection was performed using the Bayesian Information Criterion (BIC) to account for model complexity. Once the best model parameterization was determined at the group level, it was used to fit each individual subject data (individual-subject step). Finally, the presence of a significant modulation of the drift rate across experimental conditions was assessed through a one-way repeated measures ANOVA with evidence level as factor.

AC C

EP

2.6 fMRI methods. 2.6.1 Image acquisition. Functional T2*-weighted images were collected on a Philips Achieva 3T scanner, using a gradientecho EPI sequence to measure the BOLD contrast over the whole brain (TR= 1.71s, TE= 25ms, 34 slices acquired in ascending interleaved order, voxel size= 3.59×3.59×3.59 mm, 64×64 matrix, flip angle=80°). Each fMRI runs included 246 volumes and lasted for approx. 7 minutes. Structural images were collected using a sagittal 3D FFE T1-weighted sequence (TR= 8.14ms, TE= 3.7ms, flip angle= 8°, voxel size 1×1×1 mm) and a T2-weighted sequence (TR=3 s, TE=80ms, flip angle= 90°, voxel size=0.98×1×1 mm, 39 slices). Preprocessing and univariate data analysis were performed using in-house software developed at Washington University (St. Louis, MO, USA). 2.6.2 Preprocessing. BOLD images were motion-corrected within and between runs, corrected for across-slice timing differences, resampled into 3 mm isotropic voxels, and warped into 711–2C space, a standardized atlas space (Talairach and Tournoux, 1988; Van Essen, 2005). Preprocessing included a wholebrain normalization correcting for changes in overall image intensity between BOLD runs. No temporal filtering was applied. 2.6.3 General Linear Model. Hemodynamic responses were estimated using a standard HRF-assumed GLM approach. The model separated the decision from the execution phase and included 12 decision regressors 6

ACCEPTED MANUSCRIPT starting at the onset of the image [Source (face, place); Evidence (1x, 2x, 3x); Accuracy (correct, incorrect)], and 1 execution regressor starting at the onset of the go-signal. The assumed response for each process was generated by convolving a rectangle function representing the duration of the process (1.2 s for the decision phase corresponding to image duration, 1 s for execution phase corresponding to go-signal duration) with a standard hemodynamic response function (Boynton et al., 1996).

M AN U

SC

RI PT

2.6.4 Second Level Analysis. To identify brain regions modulated by the amount of evidence for source memory decisions, a group analysis was conducted using a random-effect ANOVA on the estimated hemodynamic responses for the decision phase. The ANOVA was performed on correct trials only and included source (face, place) and evidence (1x, 2x, 3x) as factors. We used only correct trials to exclude the potential confounding effect of retrieval success, as the inclusion of incorrect trials would result in a different ratio of correct vs. incorrect trials between evidence conditions. Data were spatially smoothed before entering the analysis using a Gaussian filter with a FWHM of 6 mm. The ANOVA was corrected for non-independence of the time points by adjusting the degrees of freedom and for multiple comparisons using joint z-score/cluster size thresholds corresponding to z= 3.0 and a cluster size of 13 face-contiguous voxels (corresponding to p<0.05, corrected). The group-averaged percent signal change for the six conditions of interest was extracted for display purposes in spherical ROIs of 9mm radius, formed using an algorithm that identified peaks in the corrected zmap and consolidated foci closer than 12 mm by coordinate averaging.

TE D

2.7 Multivariate Pattern Analysis. Multivariate Pattern Analysis (Haynes, 2015; Haynes and Rees, 2006) was performed to test whether the fine-grained locally distributed pattern of activity predicted subject's decisions and whether choice-predictive activity was further modulated by decision evidence. Since we were interested in predicting choices regardless of accuracy, the analysis was performed on all trials. Analyses were carried out using nilearn (Abraham et al., 2014), pymvpa (Hanke et al., 2009) and scipy (Oliphant, 2007).

AC C

EP

2.7.1 Feature extraction We extracted beta patterns for each experimental trial using a GLM that included one regressor for each trial, starting at the onset of the image, plus a common regressor for all the trials, starting at the onset of the go-signal. The assumed response was generated using the same procedure described in the previous General Linear Model section. 2.7.2 Within-subject searchlight analysis We ran a searchlight analysis (Kriegeskorte et al., 2006) using beta maps as the input data and subject’s response (face, place) as the label. Beta maps were first normalized using z-score in order to avoid the bias represented by the overall level of activity in a region in the classification analysis (voxel-wise normalization). This procedure ensured that the classification model discriminated differences in the spatial distribution of activity rather than in the mean signal (Davis et al., 2014; Hebart and Baker, 2017). Moreover voxel activity was detrended and then normalized across trials using z-score normalization (trial-wise normalization). We scrolled a sphere of 3 voxel radius (up to 123 voxels/sphere) in all the voxels of the brain. For each sphere, a Linear Support Vector Machine (SVM) with regularization parameter C=1 was trained and tested in a k-fold leave-one-out crossvalidation with k=5, assigning the testing accuracy to each sphere center (Etzel et al., 2013). Separate searchlight analyses were conducted for each evidence level in each subject, resulting in 7

ACCEPTED MANUSCRIPT 3×18 accuracy maps. For each searchlight analysis we balanced the dataset within each crossvalidation fold (see balancing strategy section).

RI PT

2.7.3 Across-subject searchlight analysis. The across-subject searchlight analysis was a variation in the cross-validation schema of the within-subject analysis. A linear SVM (C=1) was trained for each sphere using a leave-one-subjectout cross-validation. Specifically, data from n-1 subjects were used to train the model, and the left out subject was used for testing, resulting in an accuracy map per subject. Since the set of subjects used for training was independent from the subject used for testing, the analysis allowed us to decode patterns of fMRI activity that were shared across-subject (Esterman et al., 2010; Kaplan and Meyer, 2012). Beta maps were z-scored and voxels were detrended and normalized across trials and subjects. In this case, a within-subject balancing of the dataset was performed (see balancing strategy section).

TE D

M AN U

SC

2.7.4 Balancing strategy. Since the multivariate analyses classified subject's decision, rather than the actual source type, the two classes could be associated with a different number of trials. Difference in numerosity can result in a bias in classification accuracy since the classifiers will minimize the costs of misclassification by classifying all samples into the majority class (Dubey et al., 2014). This issue can be solved using a resampling approach such as removing examples from the bigger class (downsampling) or creating new examples in the lower class (upsampling). We opted for a random downsampling approach as it has been shown to have the best overall performance (Dubey et al., 2014). This strategy randomly removes examples from the bigger class to make it as numerous as the smaller class (Kotsiantis, 2006). Table 1 provides the information about the average number of used/removed examples. The same balancing procedure was used for both the within- and acrosssubject analyses.

AC C

EP

2.7.5 Statistical analyses. We identified clusters where classification accuracy was: i) above chance in at least one of the evidence levels and ii) modulated by amount of decision evidence. These statistical analyses were carried out using two Linear Mixed Effects (LME) models (Chen et al., 2013). A first model performed an omnibus contrast while a second model tested for the linear effect of a covariate represented by the evidence level (1x, 2x, 3x). The LME framework is a powerful model to correctly estimate the variance–covariance structures for both random effects and residuals, especially for within-subject covariates (Chen et al., 2013). The statistical maps were corrected using FDR correction (α=0.01) and cluster level thresholding (Cox et al., 2017). For both the withinand across-subject analyses, the map obtained with the omnibus contrast was thresholded at χ2=15.71 (corresponding to p<0.001, FDR-corrected) whereas the map of the linear effect of evidence was thresholded at z=2.97 (corresponding to p<0.005, FDR-corrected). The maps were further thresholded to select only significant clusters of voxels: 30 face-contiguous voxels cluster size for the omnibus test and 20 face-contiguous voxels cluster size for the linear effect of evidence thresholds were used. The rationale for using two different thresholds was that voxels that showed choice-predictive activity in at least one condition were already identified through a strict test, so a more lenient test could be used to further isolate those voxels showing also a linear relationship with the level of evidence, without increasing the risk of false positives. Regions of interest were extracted from the map of the intersection between these two maps. 2.8 Functional connectivity analyses. 8

ACCEPTED MANUSCRIPT We conducted univariate and a multivariate connectivity analyses to test the presence of a significant pattern of covariation between activity in parietal and medial-temporal regions. As no medial-temporal region was found in the univariate map of the main effect of evidence (Figure 2A, Inline Supplementary Table 2A), both analyses used regions of interest formed on the intersection map of the across-subject searchlight analysis.

SC

RI PT

2.8.1 Beta series correlations. We first used a beta series correlation approach (Rissman et al., 2004) to test whether a significant pattern of co-activation between this set of regions was observed when looking at the single-trial estimates of BOLD activity averaged across voxels. Beta images corresponding to each trial were obtained from the same single-trial GLM that was used as input for the multivariate analyses (see paragraph 2.7.1). The timecourse of single-trial betas was extracted from the same set of ROIs obtained with the intersection map of the across-subject multivariate analysis, averaging across voxels. We then computed a z fisher-transformed pairwise correlation matrix and performed a one sample t-test to identify statistically significant correlations (p<0.005, Bonferroni correction).

AC C

EP

TE D

M AN U

2.8.2 Multivariate Functional Connectivity. We further investigated whether a significant covariation existed between multivariate patterns of activity in choice-predictive regions. Different methods have been proposed to study multivariate connectivity (Anzellotti and Coutanche, 2018). For example, a previous study on memory reactivation, which used independent training (encoding) and testing (retrieval) sessions, computed the correlation of the output of the classification model within each ROI (Kuhl and Chun, 2012). Since the present study does not include independent training and test phases, we opted for an alternative method that quantifies the covariation of multivariate pattern trajectories in different regions. Our basic assumption was that a correlation of the transition speed between brain representations in two regions can be considered a measure of their interdependence, or functional connectivity. Notably, since the hyperspaces of two different ROIs have different dimensions and are unregistered, we could not directly compare the actual trajectories. The multivariate pattern extracted from each region of interest was a vector ∈ where d is the number of voxels within the ROI and β is the index of the considered trial going from 1 to n. We then calculated the Euclidean distance between x at β and x at β+1, which can be defined as the velocity at which the multivariate pattern moves at each step (trial) from point x(β) to x(β+1) within the hyperspace. This can be defined as: =

−

+1

The velocity timecourse = … was obtained by calculating the v for each of the n trials for each ROI. Next, we built a connectivity matrix for each subject computing pairwise z fisher-transformed partial correlations of the velocity timecourse for each ROI. The velocity timecourse of the entire brain mask was used as a control variable in a partial correlation analysis to remove the contribution of a common velocity timecourse shared by the whole brain voxels. Finally, we performed a one sample t-test to assess the statistical significance of the correlation, correcting for the number of pairwise tests (p<0.005, Bonferroni correction). Some limitations of this approach need to be considered for a proper interpretation of the results. Firstly, the method does not directly compare the trajectories and therefore cannot distinguish, for example, whether brain representations are in phase or in anti-phase between two regions. 9

ACCEPTED MANUSCRIPT Moreover, the method cannot be selectively applied to specific conditions or trials, meaning that the obtained measure of connectivity generalizes across all the trials of the memory task. Despite these limitations, we believe this method provides a meaningful index of regional interdependence, i.e. functional connectivity.

TE D

M AN U

SC

RI PT

3 Results 3.1 Modulation of decision evidence: effect of encoding strength on source memory accuracy. We first examined whether the manipulation of evidence for source memory decisions specifically affected the drift rate of a drift diffusion model applied to the behavioral data of the speeded version of the task. According to the BIC, the best parameterization at the meta-subject level was a model in which the drift rate was allowed to freely vary across conditions while the other parameters remained constant (Inline Supplementary Table 1). Consistently, a one-way ANOVA conducted on the estimates of the drift rates from individual subject fitting for the three conditions [0.03±0.01 (mean±se), 0.17±0.04 and 0.96±0.23], revealed a significant effect of study repetition (F(2,34)=5.60; p<0.01). Thus, in accordance with our predictions, the current manipulation specifically affected the process of evidence accumulation during decision formation. We then assessed the effect of study repetition on source memory accuracy of the delayed paradigm, which was used for the fMRI analyses. A 2-way repeated-measure ANOVA conducted on source memory accuracy, with memory source (face, place) and decision evidence (1x, 2x, 3x repetitions) as factors, revealed a significant main effect of evidence [F(2,34)= 72.96; p< 0.001] (Fig. 1C). Duncan post-hoc tests confirmed the presence of a significant difference between the three levels of evidence (all comparisons: p< 0.001). No significant effect of source [F(1,17) = 0.52; p= 0.48] nor an interaction effect [F(2,34)= 1.99; p= 0.15] were observed. One-way ANOVAs with decision evidence (1x, 2x, 3x repetitions) as factor on measures from signal detection theory confirmed that the frequency manipulation produced a linear increase of sensitivity [F(2,34)= 65.05, p<0.0001] (Fig. 1D) without affecting the criterion [F(2,34)= 1.32, p= 0.28] of source memory judgments. Thus, as predicted, the manipulation of decision evidence affected the accuracy of source memory decisions regardless of the particular outcome.

AC C

EP

3.2 Brain regions tracking the amount of evidence for source-memory decisions. As a first test to identify decision-related activity during source memory judgments, we looked for brain regions showing an effect of decision evidence prior to motor response, using a traditional univariate analysis (Heekeren et al., 2004; Ho et al., 2009; Kayser et al., 2010; Liu and Pleskac, 2011). We expected that parietal regions located along the intraparietal sulcus would have exhibited a symmetric evidence modulation, characterized by higher activity for higher evidence, regardless of source type. Figure 2A shows the brain regions that exhibited a significant main effect of evidence during the decision delay, identified through a voxelwise 2-way ANOVA with source type and evidence as factors. These regions were located bilaterally in the prefrontal cortex, insula, and medial and lateral posterior parietal cortices (see Inline Supplementary Table 2A). The majority of these regions showed no significant effect of source type (Fig. 2B, see also Inline Supplementary Table 2B), and, at the chosen threshold, no region could be identified exhibiting a 2-way interaction between source type and decision evidence, suggesting that the amount of decision evidence was encoded independently from the specific source, at least in terms of average BOLD activity within a region. The bar plots representing the averaged activity for the six experimental conditions in parietal and frontal ROIs (Fig. 2C), selected from the map of the main effect of evidence, show that activity tracked the amount of evidence in an approximately symmetric way. This pattern was observed regardless of the sign of the activation 10

ACCEPTED MANUSCRIPT compared to the baseline (activity from all the regions is plotted in Fig. 2D). To summarize, the univariate analysis showed that the level of BOLD activity in a set of brain regions, including the lateral parietal cortex along the intraparietal sulcus, tracked the amount of evidence for the decision in a symmetrical fashion during source memory judgments, regardless of source type.

AC C

EP

TE D

M AN U

SC

RI PT

3.3 Multivariate mapping of choice-predictive activity during source memory decisions. Next, we employed a searchlight MVPA approach to identify regions where BOLD activity predicted the subjects' choices (face, place) irrespective of decision accuracy, by exploiting information in local brain activity patterns across the whole brain. Specifically, we computed the decoding accuracy separately for each subject and level of evidence and searched for regions in which decoding accuracy showed a linear increase matching the increase in decision evidence, based on the rationale that decision-related activity should reflect the amount of decision evidence. Figure 3A illustrates regions displaying a significant choice-predictive activity for at least one level of evidence (linear mixed effect model (Chen et al., 2013), omnibus test). Of note, the current approach (see methods) ensures that the classifier output did not simply reflect activity of regions showing a main effect of source type in the univariate analysis. The multivariate analysis showed that decisions could be predicted from local activity in a vast portion of cortex (see Inline Supplementary Table 3A), including more dorsal parietal regions previously associated with memory decision-making (Sestieri et al., 2017) and more ventral parietal regions associated with the representation of retrieved information (Bonnici et al., 2016; Kuhl and Chun, 2014; Sestieri et al., 2017; Xiao et al., 2017). However, only in a restricted set of regions (Fig. 3B), including clusters located on medial and lateral parietal, medial temporal and inferior frontal cortex (see Inline Supplementary Table 3B), the accuracy of the decoder showed a linear effect of evidence, paralleling the effect on behavioral performance. The intersection of the two maps and the values of decoding accuracy for the three levels of evidence in the identified clusters are presented in Figure 3C and 3D, respectively (see also Table 2). Table 2 shows the results of a regional analysis that confirmed that the majority of these clusters exhibited significant choice-predictive activity also in the most difficult condition (1x), consistent with a role in representing a decision variable. Finally, an additional regional analysis indicated that only a restricted number of regions that were modulated by decision evidence in the univariate analysis also showed choice-predictive activity (Inline Supplementary Table 2A). To summarize, a small set of cortical regions, including a cluster located along the intraparietal sulcus, displayed choicepredictive activity, which was present also in the most difficult condition and tracked the amount of decision evidence, consistent with a role in memory-based decision-making. 3.4 Shared representation of choice-predictive activity across subjects. Next, we tested whether different subjects also showed a similar representation of choicepredictive activity during memory-based decisions, by performing a leave-one-subject-out version of the searchlight classification analysis (Richter et al., 2016). As in the previous analysis, we searched for regions showing both significant choice-predictive activity in at least one level of evidence and a linear effect of decision evidence. Significant across-subject decoding using local information was identified in a network of brain regions similar to the one shown in the previous within-subject analysis, although the regions were now mainly confined to parietal and medial temporal cortices (Fig. 4A, see Inline Supplementary Table 3A). No choice-predictive activity was identified in the somatomotor cortex, possibly reflecting the fact that the across-subjects analysis canceled out BOLD activity related to motor intentions. In the map corresponding to the linear effect of evidence, moreover, we identified two sets of regions with an opposite (positive, negative) relationship with the amount of decision evidence (Fig. 4B, see Inline Supplementary 11

ACCEPTED MANUSCRIPT

RI PT

Table 4B). Interestingly, the intersection between the results of the two analyses (Fig. 4C) indicated that, while across-subject decoding was still observed in left lateral parietal and medial temporal cortices, no significant cross-decoding was observed in medial parietal cortex (Table 3). The lateral parietal cluster appears located slightly more ventrally than the cluster obtained with the within-subject analysis. Nonetheless, a regional analysis confirmed the presence of significant across-subjects decoding also in the latter cluster (omnibus test: F(3,34)=3.72; p<0.05, linear effect of evidence: t(35)=3.28; p<0.01). In addition, a region of the right ventrolateral prefrontal cortex showed an inverse relationship characterized by highest decoding performance for low evidence trials. To summarize, the analysis demonstrated that different subjects had a similar spatial representation of choice-predictive activity modulated by decision evidence in a limited number of cortical regions, including the lateral parietal cortex.

AC C

EP

TE D

M AN U

SC

3.5 Multivariate connectivity analyses. Finally, we tested whether high-level regions involved in processing information for memorybased decisions were functionally connected with neural structures causally involved in memory retrieval. In animals and humans, a key role in the encoding and retrieval of episodic memories is played by regions of the MTL (Eichenbaum et al., 2007; Squire, 1992). To address the issue, we first used a standard beta series correlation approach to test whether a significant pattern of coactivation could be found between the regions identified in the previous analysis (Table 3). However, no significant correlation between the mean activities of any of the regions pair was observed at the chosen threshold. Therefore, we developed a novel analysis strategy to obtain an index of the temporal covariation of the multivariate pattern of activity measured in the same set of regions. Specifically, the analysis computed a measure of the inter-regional correlation of the velocity at which the multivariate pattern moved in the hyperspace described by the voxels within each region (Fig. 5A). Indeed, we found a significant correlation (Fig. 5B-C) between the activity in the left parahippocampal gyrus and both the left angular gyrus [t(18)=4.63; p<0.005, Bonferroni corrected] and the right cerebellum [t(18)=4.97; p<0.005], whereas a negative correlation was observed between the left angular gyrus and right middle frontal gyrus [t(18)=-6.52; p<0.005]. Notably, a significant correlation between lateral parietal and medial temporal ROIs was also observed when using an alternative multivariate connectivity approach (see Supplementary Figure 1 and Supplementary Methods) similar to the one employed by Kuhl and Chun {Kuhl, 2012 #647}. While these findings are consistent with the hypothesis that left lateral parietal regions communicate with medial temporal regions during memory based-decisions, they further indicate the presence of a negative relationship between regions showing opposite modulations of decision evidence. 4 Discussion. Using an experimental design that involved a parametric variation of decision evidence during source memory judgments, we demonstrated that a restricted set of brain regions, including the lateral parietal cortex around the intraparietal sulcus, exhibits several properties that are consistent with a role in representing a decision variable during source memory judgments. Firstly, these regions showed a robust modulation of the mean signal in response to the manipulation of decision evidence. Moreover, multivariate analyses indicated that these regions showed choicepredictive activity that was: i. significant even at the lowest level of decision evidence, ii. modulated by the amount of evidence, iii. shared across subjects, and iv. correlated with choicepredictive activity in medial-temporal regions. 4.1 The neural representation of memory decision variables. 12

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

In a previous study, we have identified a region of the lateral parietal cortex whose activity tracked the amount of evidence for item recognition (old/new) decisions (Sestieri et al., 2014). This finding is compatible with a "mnemonic accumulator" hypothesis (Wagner et al., 2005), which interprets retrieval-related activity in lateral parietal cortex as a potential neural implementation of a diffusion process for the accumulation of evidence during memory decisions (Ratcliff, 1978; Smith and Ratcliff, 2004). The present study examined the role of lateral parietal regions located at/near the intraparietal sulcus in more complex decisions that cannot be performed solely on the basis of familiarity with the probe stimulus (Johnson et al., 1993). Importantly, we found that the activity of these regions did not simply track the amount of decision evidence but also carried information about subjects' choices (Hebart et al., 2012; Rissman and Wagner, 2012). Therefore, these parietal signals appear to encode a decision variable that can be used to predict the subjects' choices independent of the objective memory status. Importantly, behavioral analyses of the speeded version of the task showed that the manipulation of study repetitions employed in the current study specifically affected the drift rate of a diffusion model, which corresponds to the process of evidence accumulation. On this basis, a tentative conclusion is that the observed BOLD modulations reflect an accumulation mechanism. However, the intrinsic temporal limitation of the fMRI technique does not allow to make strong claims about accumulation or ramp-like activity at the neural level. We therefore point out that the issue must be investigated with alternative methods (e.g. electroand magneto-encephalographic) working at a faster time scale and capable of tracking instantaneous neural activity. Furthermore, as noted by previous authors, the presence of choice-predictive activity does not necessarily mean that the identified regions determine the decision outcome (Hebart et al., 2012; Nienborg and Cumming, 2009). It is also possible that the different components of the identified set of areas have partially distinct roles in the decision process, from the actual retrieval of mnemonic information associated to a particular probe stimulus (e.g. medial temporal cortex) to the evaluation of evidence for the two possible outcomes (e.g. parietal cortex) to the actual implementation of the chosen plan (e.g. insula/frontal cortex). Finally, the involvement of areas located around the intraparietal sulcus during memory retrieval appears to underlie multiple computations, including the mere sense of familiarity (Vilberg and Rugg, 2008; Wagner et al., 2005), the evaluation of evidence for memory decisions ((Sestieri et al., 2014) and present results), but also the online manipulation of retrieved information in more complex memory judgments (Sestieri et al., 2011; Sestieri et al., 2017). Providing a coherent picture of this multifaceted involvement is a matter for future investigations. 4.2. Decision-making versus alternative accounts of retrieval-related activity. We believe that the critical factor that supports a decision-making account in the present study is the combination of several properties of the BOLD signals observed in the parietal cortex. For example, since decision evidence was manipulated by varying the frequency of study repetitions, the amount of evidence was closely intertwined with the amount of information actually retrieved from memory (e.g. number of faces/places associated with a specific object). As a matter of fact, any modulation of the signal in response to varying decision evidence, both in univariate and multivariate analysis, could be explained by assuming a sensitivity to other variables, such as quantity of recollected information, capture of bottom-up attention, sense of oldness, taskrelevance or decision confidence. However, all these accounts hardly explain the presence of choice-predictive activity per se, as observed within each level of evidence. Indeed, the lack of a significant effect of source type in the behavioral analysis suggests that the two source types were 13

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

not associated with a difference in any of the aforementioned variables that could explain choicepredictive activity. On the other hand, the presence of choice-predictive activity, considered alone, could be alternatively explained by sensitivity to retrieval content. Previous studies on memory reactivation effects have indeed demonstrated that retrieval-related activity in lateral parietal regions can be used to predict not only the recalled stimulus category, but also the individual stimuli (Bonnici et al., 2016; Kuhl and Chun, 2014; Xiao et al., 2017). However, these studies differ from the present work in several crucial features. Instead of asking to recall information associated to a certain stimulus and focus on memory reactivation effects, here we forced subjects to decide between two alternatives and tested whether choice-predictive activity varied as a function of decision evidence, emphasizing the decisional aspect of the task. A further argument favoring the decisionmaking account is the presence of significant decoding in the most difficult condition, where subjects made several source misattributions, which suggests that BOLD activity followed the current choice. In general, we believe that the present pattern of results is better explained in terms of a decision-making mechanism that combines/integrates signals related to amount and type of retrieved information for the purpose of a decision, i.e. using these signals to convert memory-related information into a behavioral response.

AC C

EP

TE D

4.3. Anatomical specificity of memory decision signals. Several anatomo-functional models developed in the last decade share the idea that distinct regions of the lateral parietal cortex play different roles during memory retrieval (Cabeza et al., 2008; Sestieri et al., 2017; Vilberg and Rugg, 2008). In particular, we have recently traced a basic distinction between more posterior-ventral regions, included in the so called Default Mode network (DMN) (Raichle et al., 2001) and associated with the representation of retrieved information, and more dorsal and anterior regions associated with the manipulation of retrieved information and decisional aspects of the memory task. Although source monitoring likely recruits both sets of regions, we expected to identify decision-related signals specifically in the second set. Figure 6 shows how the present results fit with previous relevant studies from our laboratory (Sestieri et al., 2011; Sestieri et al., 2010; Sestieri et al., 2014) and suggests several considerations. Firstly, the overlap (yellow) between regions showing an effect of decision evidence in the univariate (green) and the multivariate (red) analysis was not large, but was nonetheless observed within regions previously associated with memory-based decision-making and manipulation of retrieved information (black border), in line with the working hypothesis. Secondly, each of these effects showed minimal spatial overlap not only with regions previously associated with the representation of retrieved events (white border) but also with regions that differentiated between source types in the univariate analysis (blue), supporting the idea that these signals were not simply related to the representation of the retrieved content. Thirdly, the two effects also showed minimal overlap with regions that tracked decision evidence during old/new judgments (cyan, blue borders). However, based on the available data, it is not possible to conclude whether this reflects a true anatomical segregation between the neural bases of item recognition and source memory decisions or an effect of other variables, e.g. different subjects, analyses or stimulus categories. 4.4 Shared representation of choice-predictive activity across subjects. Previous studies have already shown that memory-related neural signatures can be consistent across individuals (Richter et al., 2016; Rissman et al., 2010), also during the recall of words (Kragel and Polyn, 2016) and complex events (Chen et al., 2017) in high-level cortical areas. One key finding of the present study was the demonstration of a shared spatial representation of choice14

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

predictive signals that modulate with decision evidence in parietal associative regions. Regarding this issue, it is important to point out that the results of the across-subjects decoding go beyond the demonstration of a mere consistency in face vs. scene voxels across subjects (which can be inferred from the within-subject analysis) but further imply some degree of inter-subject consistency of face-related and place-related mnemonic patterns. The results of the across-subject analysis also allow to rule out an explanation of the effect of decision evidence on choice-predictive activity in terms of motor intentions. The issue arises from the use of a delay paradigm, where easier (i.e. faster) decisions allow more time to prepare a specific motor response (i.e. index vs. middle finger). This confound might explain, for example, the presence of choice-predictive activity within somatomotor regions in the within-subject analysis, which was insensitive to the particular choice/key press association. However, a similar confound does not apply to the across-subject analysis, as a significant decoding in a subject with a particular choice/key press association would be canceled out by the decoding in a subject with the opposite association, resulting in the absence of significant across-subject decoding. Nonetheless, the precise assessment of the role of motor intentions in modulating decision signals requires a comparison with paradigms in which subjects are informed about the response mapping only at the onset of the go signal (Gherman and Philiastides, 2015; Hebart et al., 2012).

AC C

EP

TE D

4.5 Lateral parietal-medial temporal connectivity during memory-based decisions. According to the mnemonic accumulator hypothesis (Wagner et al., 2005), parietal regions are expected to accumulate decision evidence through interaction with the MTL, such as sensorymotor regions integrate the activity of motion-sensitive areas during motion-discrimination decisions (Donner et al., 2009; Shadlen et al., 2008; Shadlen and Newsome, 2001). Although the presence of inter-regional correlation does not directly imply integration, it nonetheless represents an important prerequisite for an integration account. Currently, there is only weak evidence for a functional coupling between lateral-parietal and medial-temporal regions during memory-based decisions in the literature (Cabeza et al., 2011; Geib et al., 2017; Kuhl et al., 2013). Here, we provide support for the hypothesis by demonstrating a significant correlation between the multivariate pattern variation in lateral parietal and medial temporal regions. Specifically, the present method provided a measure of the synchronism in the change of brain representations (i.e. multivariate patterns) across ROIs. Although we acknowledge that this measure of connectivity is agnostic to whether the variation in the BOLD response relates to a specific outcome variable (e.g. source and/or evidence effects), we note that a significant correlation between lateral parietal and medial temporal regions was also observed when using a distance measure that weights the contribution of voxels that are relevant to decision evidence {Kuhl, 2012 #647}. Furthermore, a traditional beta series correlation method (Rissman et al., 2004) did not reveal any significant correlation between lateral parietal and medial-temporal regions. Unfortunately, it is not possible to discern whether this apparent discrepancy reflects the higher sensitivity of the multivariate approach or more basic methodological factors, such as sub-optimal ROI definition (given the lack of a functionally-defined MTL region from the univariate analysis). Finally, techniques with higher temporal resolution and methods for assessing effective connectivity and the temporal dynamics of brain representation (King and Dehaene, 2014) are needed to investigate the integrative nature, and the direction, of the putative information transfer. Competing interests We have no financial and non-financial competing interests to declare. 15

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

Acknowledgments We thank Dina Di Pietro for her assistance on data collection of the pilot experiment.

16

ACCEPTED MANUSCRIPT Tables Table 1.

Table 2. x y

Region

voxels

Decoding in 1x condition ** **

SC

z

RI PT

used discarded mean std mean std 1x 32.7 5.5 12.7 11.6 2x 35.7 3.1 6.7 5.7 3x 35.6 3.6 7.0 6.7 Average and standard deviation of the number of trials that were used/discarded in the multivariate analyses.

Table 3. x y

z

TE D

M AN U

-47 -43 36 Left Inferior Parietal Lobule (IPL) 124 0 -64 32 Right Precuneus (PreCu) 103 -37 13 -2 Left Insula 95 -21 -43 -5 Left Parahippocampal Gyrus (paraHG) 39 ** -11 -69 43 Left Precuneus (PreCu) 30 -31 -23 66 Left Precentral Gyrus (preCG) 22 * Talairach coordinates of the center of mass corresponding to the clusters displayed in Fig. 3C. Labels were assigned using the AFNI tool whereami (AFNI software, (Cox, 1996)). The last column indicates the statistical significance of ROI-based decoding analyses for the 1x condition. The asterisks indicate the p-value associated with the t-test versus chance level (accuracy=0.5). One asterisk indicates p<0.05, two asterisks indicate p<0.01.

Regions

voxels

Decoding in 1x condition **

AC C

EP

-54 -59 37 Left Inferior Parietal Lobule (IPL) 46 12 -60 -29 Right Cerebellum 34 -35 41 -10 Left Middle Frontal Gyrus (MFG) 33 ** -24 -35 -7 Left Parahippocampal Gyrus (paraHG) 26 * 34 -1 -6 Right Insula 12 -35 -65 -12 Left Fusiform Gyrus (FG) 11 * Talairach coordinates of the center of mass corresponding to the clusters displayed in Fig. 4C. The last column indicates the statistical significance of ROI-based decoding analyses for the 1x condition. The asterisks indicate the p-value associated with the t-test versus chance level (accuracy=0.5). One asterisk indicates p<0.05, two asterisks indicate p<0.01.

17

ACCEPTED MANUSCRIPT Figure captions.

SC

RI PT

Figure 1. Experimental paradigm and behavioral results. A. In different blocks of the encoding phase, participants were presented with a picture of an object together with two face or place images and were asked to choose the face or place image they felt was more related to the object. B. In the retrieval phase conducted in the fMRI scanner, participants had to indicate whether the presented object has been previously encoded along with face or place images. All the fMRI analyses focus on activity associated with the object and delay periods, prior to the presentation of the go-signal for manually reporting the decision. C. The accuracy of the source memory judgment (left) was robustly modulated by decision evidence, which was operationalized as the number of object presentations (1x, 2x, 3x) during the encoding phase. D. The sensitivity of source memory judgments (right) linearly increased as a function of encoding repetitions. In this graph, thin colored lines represent individual subjects while the thick black line represents the mean across subjects.

EP

TE D

M AN U

Figure 2. Effect of decision evidence modulation on the mean BOLD activity. A. Voxelwise map of the main effect of decision evidence from the two-way repeated measures ANOVA conducted on retrieval related activity, superimposed over an inflated representation of both hemispheres using Caret software v5.65 (Van Essen, 2005). The circle in the middle provides a different view of the left lateral parietal cortex. The analysis was conducted on correct trials only. B. Intersection (yellow) between the maps of the main effect of evidence (red) and source type (green), indicating the limited spatial overlap of the two main effects. C. Bar plots showing retrieval-related activity in the six experimental conditions extracted from parietal and frontal regions of the map shown in 2A. Plots are intended for visualization purposes only, since data are non independent from the main analysis (Kriegeskorte et al., 2010). Higher BOLD activity was associated with stronger decision evidence and the effect was symmetric for both source types (red=face, blue=place). D. Graph summarizing the profile of activity for the six experimental conditions in all the regions generated using the map shown in 2A (thin colored lines) as well as for the average across regions (thick black line). The BOLD activity positively tracked the amount of decision evidence regardless of the overall sign of activity with respect to the baseline.

AC C

Figure 3. Effect of decision evidence modulation on choice-predictive activity. A. Map showing significant decoding of subject's choices in at least one level of evidence (omnibus test). The color scale indicates the effect size. The threshold of the map corresponded to p<0.001, corrected. B. Regions where decoding accuracy was linearly modulated by the level of decision evidence. The threshold of the map corresponded to p<0.005, corrected. C. Intersection between maps in figure 3A and 3B. D. Graph showing the decoding accuracy as a function of the three levels of evidence in the six clusters formed from the intersection map in 3C. Shades indicate one standard deviation from the average accuracy. Plots are for visualization purposes only. Figure 3. Similar representation of choice-predictive activity across individuals. A. Regions showing significant decoding of subject's choices in at least one level of evidence (p<0.001, corrected). B. Regions where decoding accuracy was linearly modulated by the level of decision evidence (p<0.005, corrected). C. Intersection between the maps in 4A and 4B. The circles in the upper panel illustrate the clusters located in lateral parietal and prefrontal cortex. The circle in the lower panel illustrates the left MTL. D. Graph showing the decoding accuracy as a function of the three levels of evidence in the six clusters formed from the intersection map in 4C. Shades indicate one standard deviation from the average accuracy. 18

ACCEPTED MANUSCRIPT

RI PT

Figure 5. Multivariate connectivity between lateral parietal and medial temporal regions. A. Illustration of the multivariate connectivity approach developed in the present study. Multivariate connectivity was computed as the correlation between the speed (vx) of variation (dist) of the multivariate patterns trajectory [X(β0)... X(βn)], measured in the regions presented in figure 4C. B. Correlation matrix showing significant positive (red) and negative (blue) multivariate connectivity. The color scale indicates the effect size of the one sample t-test against zero correlation. Only those correlation associated with a p<0.005 (Bonferroni corrected) are reported. C. Graphical illustration of the region pairs showing significant multivariate connectivity.

AC C

EP

TE D

M AN U

SC

Figure 6. Anatomical specificity of memory-decision signals. The figure illustrates the anatomical location of the main findings of the present study relative to a neuro-anatomical model of the involvement of lateral parietal region during episodic memory retrieval and our previous study on item recognition decisions. In a recent review (Sestieri et al., 2017), we distinguished between more ventral-posterior regions (delimited by white border), overlapping with the Default Mode Network (DMN) and associated with the representation of retrieved information, and more dorsal and anterior regions along the intraparietal sulcus (black border), overlapping with the so-called fronto-parietal control network and associated with memory-based decision-making and the manipulation of retrieved information. In a previous study on item recognition decisions (Sestieri et al., 2014), we further circumscribed a sub-region from this latter component, named Lat-IPS (cyan border), whose activity tracks the amount of evidence favoring old decisions, compatible with a mnemonic accumulator. The study also identified another region in the ventral AG (blue border) that showed a symmetric modulation of evidence for both old and new items. The univariate analysis of the present study identified two anatomically distinct sets of regions, those modulated by decision evidence (green) and those modulated by source type (blue). Moreover, the multivariate analysis (within-subject) identified regions where choice-predictive activity was modulated by decision evidence (red). The figure shows that regions modulated by evidence and those showing choice-predictive activity show overlap (yellow) with each other and with regions previously associated with decision-making and the manipulation of retrieved information, but not with regions associated with the representation of retrieved information or item recognition decisions.

19

ACCEPTED MANUSCRIPT References

AC C

EP

TE D

M AN U

SC

RI PT

Abraham, A., Pedregosa, F., Eickenberg, M., Gervais, P., Mueller, A., Kossaifi, J., Gramfort, A., Thirion, B., Varoquaux, G., 2014. Machine learning for neuroimaging with scikit-learn. Front Neuroinform 8, 14. Anzellotti, S., Coutanche, M.N., 2018. Beyond Functional Connectivity: Investigating Networks of Multivariate Representations. Trends Cogn Sci 22, 258-269. Bonnici, H.M., Richter, F.R., Yazar, Y., Simons, J.S., 2016. Multimodal Feature Integration in the Angular Gyrus during Episodic and Semantic Retrieval. J Neurosci 36, 5462-5471. Boynton, G.M., Engel, S.A., Glover, G.H., Heeger, D.J., 1996. Linear systems analysis of functional magnetic resonance imaging in human V1. J Neurosci 16, 4207-4221. Brady, T.F., Konkle, T., Alvarez, G.A., Oliva, A., 2008. Visual long-term memory has a massive storage capacity for object details. Proc Natl Acad Sci U S A 105, 14325-14329. Cabeza, R., Ciaramelli, E., Olson, I.R., Moscovitch, M., 2008. The parietal cortex and episodic memory: an attentional account. Nat Rev Neurosci 9, 613-625. Cabeza, R., Mazuz, Y.S., Stokes, J., Kragel, J.E., Woldorff, M.G., Ciaramelli, E., Olson, I.R., Moscovitch, M., 2011. Overlapping parietal activity in memory and perception: evidence for the attention to memory model. J Cogn Neurosci 23, 3209-3217. Chen, G., Saad, Z.S., Britton, J.C., Pine, D.S., Cox, R.W., 2013. Linear mixed-effects modeling approach to FMRI group analysis. Neuroimage 73, 176-190. Chen, J., Leong, Y.C., Honey, C.J., Yong, C.H., Norman, K.A., Hasson, U., 2017. Shared memories reveal shared structure in neural activity across individuals. Nat Neurosci 20, 115-125. Cox, R.W., 1996. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29, 162-173. Cox, R.W., Chen, G., Glen, D.R., Reynolds, R.C., Taylor, P.A., 2017. fMRI clustering and false-positive rates. Proc Natl Acad Sci U S A 114, E3370-E3371. Criss, A.H., Wheeler, M.E., McClelland, J.L., 2013. A differentiation account of recognition memory: evidence from fMRI. J Cogn Neurosci 25, 421-435. Davis, T., LaRocque, K.F., Mumford, J.A., Norman, K.A., Wagner, A.D., Poldrack, R.A., 2014. What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis. Neuroimage 97, 271-283. Donner, T.H., Siegel, M., Fries, P., Engel, A.K., 2009. Buildup of choice-predictive activity in human motor cortex during perceptual decision making. Curr Biol 19, 1581-1585. Dubey, R., Zhou, J., Wang, Y., Thompson, P.M., Ye, J., Alzheimer's Disease Neuroimaging, I., 2014. Analysis of sampling techniques for imbalanced data: An n = 648 ADNI study. Neuroimage 87, 220-241. Eichenbaum, H., Yonelinas, A.P., Ranganath, C., 2007. The medial temporal lobe and recognition memory. Annu Rev Neurosci 30, 123-152. Esterman, M., Tamber-Rosenau, B.J., Chiu, Y.C., Yantis, S., 2010. Avoiding non-independence in fMRI data analysis: leave one subject out. Neuroimage 50, 572-576. Etzel, J.A., Zacks, J.M., Braver, T.S., 2013. Searchlight analysis: promise, pitfalls, and potential. Neuroimage 78, 261-269. Geib, B.R., Stanley, M.L., Dennis, N.A., Woldorff, M.G., Cabeza, R., 2017. From hippocampus to whole-brain: The role of integrative processing in episodic memory retrieval. Hum Brain Mapp 38, 2242-2259. Gherman, S., Philiastides, M.G., 2015. Neural representations of confidence emerge from the process of decision formation during perceptual choices. Neuroimage 106, 134-143. Gold, J.I., Shadlen, M.N., 2007. The neural basis of decision making. Annu Rev Neurosci 30, 535-574. Guerin, S.A., Miller, M.B., 2011. Parietal cortex tracks the amount of information retrieved even when it is not the basis of a memory decision. Neuroimage 55, 801-807. Hanke, M., Halchenko, Y.O., Sederberg, P.B., Hanson, S.J., Haxby, J.V., Pollmann, S., 2009. PyMVPA: A python toolbox for multivariate pattern analysis of fMRI data. Neuroinformatics 7, 37-53. Haynes, J.D., 2015. A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives. Neuron 87, 257-270. 20

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

Haynes, J.D., Rees, G., 2006. Decoding mental states from brain activity in humans. Nat Rev Neurosci 7, 523-534. Hebart, M.N., Baker, C.I., 2017. Deconstructing multivariate decoding for the study of brain function. Neuroimage. Hebart, M.N., Donner, T.H., Haynes, J.D., 2012. Human visual and parietal cortex encode visual choices independent of motor plans. Neuroimage 63, 1393-1403. Heekeren, H.R., Marrett, S., Bandettini, P.A., Ungerleider, L.G., 2004. A general mechanism for perceptual decision-making in the human brain. Nature 431, 859-862. Heekeren, H.R., Marrett, S., Ungerleider, L.G., 2008. The neural systems that mediate human perceptual decision making. Nat Rev Neurosci 9, 467-479. Ho, T.C., Brown, S., Serences, J.T., 2009. Domain general mechanisms of perceptual decision making in human cortex. J Neurosci 29, 8675-8687. Hutchinson, J.B., Uncapher, M.R., Weiner, K.S., Bressler, D.W., Silver, M.A., Preston, A.R., Wagner, A.D., 2014. Functional Heterogeneity in Posterior Parietal Cortex Across Attention and Episodic Memory Retrieval. Cereb Cortex. Janowsky, J.S., Shimamura, A.P., Squire, L.R., 1989. Source memory impairment in patients with frontal lobe lesions. Neuropsychologia 27, 1043-1056. Johnson, M.K., Hashtroudi, S., Lindsay, D.S., 1993. Source monitoring. Psychol Bull 114, 3-28. Kahn, I., Davachi, L., Wagner, A.D., 2004. Functional-neuroanatomic correlates of recollection: implications for models of recognition memory. J Neurosci 24, 4172-4180. Kaplan, J.T., Meyer, K., 2012. Multivariate pattern analysis reveals common neural patterns across individuals during touch observation. Neuroimage 60, 204-212. Kayser, A.S., Buchsbaum, B.R., Erickson, D.T., D'Esposito, M., 2010. The functional anatomy of a perceptual decision in the human brain. J Neurophysiol 103, 1179-1194. King, J.R., Dehaene, S., 2014. Characterizing the dynamics of mental representations: the temporal generalization method. Trends Cogn Sci 18, 203-210. Konkle, T., Brady, T.F., Alvarez, G.A., Oliva, A., 2010. Scene memory is more detailed than you think: the role of categories in visual long-term memory. Psychol Sci 21, 1551-1556. Kotsiantis, S., Kanellopoulos, D., Pintelas, P. , 2006. Handling imbalanced datasets: A review GESTS International Transactions on Computer Science and Engineering 30, 25-36. Kragel, J.E., Polyn, S.M., 2016. Decoding Episodic Retrieval Processes: Frontoparietal and Medial Temporal Lobe Contributions to Free Recall. J Cogn Neurosci 28, 125-139. Kriegeskorte, N., Goebel, R., Bandettini, P., 2006. Information-based functional brain mapping. Proc Natl Acad Sci U S A 103, 3863-3868. Kriegeskorte, N., Lindquist, M.A., Nichols, T.E., Poldrack, R.A., Vul, E., 2010. Everything you never wanted to know about circular analysis, but were afraid to ask. J Cereb Blood Flow Metab 30, 1551-1557. Kuhl, B.A., Chun, M.M., 2012. Attending to the present when remembering the past. Neuron 75, 944-947. Kuhl, B.A., Chun, M.M., 2014. Successful remembering elicits event-specific activity patterns in lateral parietal cortex. J Neurosci 34, 8051-8060. Kuhl, B.A., Johnson, M.K., Chun, M.M., 2013. Dissociable neural mechanisms for goal-directed versus incidental memory reactivation. J Neurosci 33, 16099-16109. Liu, T., Pleskac, T.J., 2011. Neural correlates of evidence accumulation in a perceptual decision task. J Neurophysiol 106, 2383-2398. Mayes, A., Montaldi, D., Migo, E., 2007. Associative memory and the medial temporal lobes. Trends Cogn Sci 11, 126-135. Mitchell, K.J., Johnson, M.K., 2009. Source monitoring 15 years later: what have we learned from fMRI about the neural mechanisms of source memory? Psychol Bull 135, 638-677. Nienborg, H., Cumming, B.G., 2009. Decision-related activity in sensory neurons reflects more than a neuron's causal effect. Nature 459, 89-92. Oliphant, T.E., 2007. Python for Scientific Computing. Computing in Science and Engineering 9, 10-20. Philiastides, M.G., Auksztulewicz, R., Heekeren, H.R., Blankenburg, F., 2011. Causal role of dorsolateral prefrontal cortex in human perceptual decision making. Curr Biol 21, 980-983. 21

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

Phillips, P.J., Wechsler, H., Huang, J., Rauss, P.J., 1998. The FERET database and evaluation procedure for face-recognition algorithms. Image and Vision Computing 16, 295-306. Raichle, M.E., MacLeod, A.M., Snyder, A.Z., Powers, W.J., Gusnard, D.A., Shulman, G.L., 2001. A default mode of brain function. Proc Natl Acad Sci U S A 98, 676-682. Ratcliff, R., 1978. A Theory of Memory Retrieval. Psychological Review 85, 59-108. Ratcliff, R., McKoon, G., 2008. The diffusion decision model: theory and data for two-choice decision tasks. Neural Comput 20, 873-922. Richter, F.R., Chanales, A.J.H., Kuhl, B.A., 2016. Predicting the integration of overlapping memories by decoding mnemonic processing states during learning. Neuroimage 124, 323-335. Rissman, J., Gazzaley, A., D'Esposito, M., 2004. Measuring functional connectivity during distinct stages of a cognitive task. Neuroimage 23, 752-763. Rissman, J., Greely, H.T., Wagner, A.D., 2010. Detecting individual memories through the neural decoding of memory states and past experience. Proc Natl Acad Sci U S A 107, 9849-9854. Rissman, J., Wagner, A.D., 2012. Distributed representations in memory: insights from functional brain imaging. Annu Rev Psychol 63, 101-128. Sestieri, C., Corbetta, M., Romani, G.L., Shulman, G.L., 2011. Episodic memory retrieval, parietal cortex, and the default mode network: functional and topographic analyses. J Neurosci 31, 4407-4420. Sestieri, C., Shulman, G.L., Corbetta, M., 2010. Attention to Memory and the Environment: Functional Specialization and Dynamic Competition in Human Posterior Parietal Cortex J Neurosci 30, 8445-8456. Sestieri, C., Shulman, G.L., Corbetta, M., 2017. The contribution of the human posterior parietal cortex to episodic memory. Nat Rev Neurosci 18, 183-192. Sestieri, C., Tosoni, A., Mignogna, V., McAvoy, M.P., Shulman, G.L., Corbetta, M., Romani, G.L., 2014. Memory accumulation mechanisms in human cortex are independent of motor intentions. J Neurosci 34, 6993-7006. Shadlen, M.N., Kiani, R., 2013. Decision making as a window on cognition. Neuron 80, 791-806. Shadlen, M.N., Kiani, R., Hanks, T.D., Churchland, A.K., 2008. Neurobiology of Decision Making: An Intentional Framework. MIT Press, Cambridge. Shadlen, M.N., Newsome, W.T., 2001. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86, 1916-1936. Smith, P.L., Ratcliff, R., 2004. Psychology and neurobiology of simple decisions. Trends Neurosci 27, 161168. Squire, L.R., 1992. Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans. Psychol Rev 99, 195-231. Talairach, J., Tournoux, P., 1988. Co-Planar Stereotaxic Atlas of the Human Brain. Thieme Medical Publishers, Inc., New York. Tosoni, A., Galati, G., Romani, G.L., Corbetta, M., 2008. Sensory-motor mechanisms in human parietal cortex underlie arbitrary visual decisions. Nat Neurosci 11, 1446-1453. Van Essen, D.C., 2005. A Population-Average, Landmark- and Surface-based (PALS) atlas of human cerebral cortex. Neuroimage 28, 635-662. Vandekerckhove, J., Tuerlinckx, F., 2008. Diffusion model analysis with MATLAB: a DMAT primer. Behav Res Methods 40, 61-72. Vilberg, K.L., Rugg, M.D., 2008. Memory retrieval and the parietal cortex: a review of evidence from a dualprocess perspective. Neuropsychologia 46, 1787-1799. Wagner, A.D., Shannon, B.J., Kahn, I., Buckner, R.L., 2005. Parietal lobe contributions to episodic memory retrieval. Trends Cogn Sci 9, 445-453. Wheeler, M.E., Buckner, R.L., 2003. Functional dissociation among components of remembering: control, perceived oldness, and content. J Neurosci 23, 3869-3880. Xiao, J., Hays, J., Ehinger, K., Oliva, A., Torralba, A., 2010. SUN Database: Large-scale Scene Recognition from Abbey to Zoo. IEEE Computer Society, 3485-3492. Xiao, X., Dong, Q., Gao, J., Men, W., Poldrack, R.A., Xue, G., 2017. Transformed Neural Pattern Reinstatement during Episodic Memory Retrieval. J Neurosci 37, 2986-2998. 22

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT