Brain and autonomic association accompanying stochastic decision-making

NeuroImage 49 (2010) 1024–1037

Contents lists available at ScienceDirect

NeuroImage j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / y n i m g

Brain and autonomic association accompanying stochastic decision-making Hideki Ohira a,⁎, Naho Ichikawa a, Michio Nomura b, Tokiko Isowa a,c, Kenta Kimura a,d, Noriaki Kanayama a, Seisuke Fukuyama e,f, Jun Shinoda e, Jitsuhiro Yamada e a

Department of Psychology, Nagoya University, Nagoya, Japan Graduate School of Integrated Arts and Sciences, Hiroshima University, Higashi-Hiroshima, Japan Department of Nursing, Mie University, Tsu, Japan d Department of Life Sciences, The University of Tokyo, Tokyo, Japan e Kizawa Memorial Hospital, Chubu Ryogo Center, Minokamo, Japan f Department of Physiology and Neuroscience, Kanagawa Dental College, Yokosuka, Japan b c

a r t i c l e

i n f o

Article history: Received 1 December 2008 Revised 16 July 2009 Accepted 23 July 2009 Available online 30 July 2009 Keywords: Decision-making PET ERP Brain Autonomic activity

a b s t r a c t To examine the functional association between brain and autonomic activities accompanying decisionmaking, we simultaneously recorded regional cerebral blood flow using 15O-water positron emission tomography and event-related brain potentials (ERPs) time-locked to feedback of reward and punishment, as well as cardiovascular parameters, during a stochastic decision-making task. We manipulated the uncertainty of outcomes in the task; specifically, we compared a condition with high predictability of reward/punishment (contingent-reward condition) and a condition with low predictability of reward/ punishment (random-reward condition). The anterior cingulate cortex (ACC) was commonly activated in both conditions. Compared with the contingent-reward condition, the orbitofrontal and right dorsolateral prefrontal cortices and dorsal striatum were activated in the random-reward condition, where subjects had to continue to seek contingency between stimuli and reward/punishment. Activation of these brain regions correlated with a positive component of ERPs locked to feedback signals (feedback-related positivity), which showed an association with behavioral decision-making in the contingent-reward condition. Furthermore, cardiovascular responses were attenuated in the random-reward condition, where continuous attention and contingency monitoring were needed, and such attenuation of cardiovascular responses was mediated by vagal activity that was governed by the rostral ACC. These findings suggest that the prefrontal-striatal network provides a neural basis for decision-making and modulation over the peripheral autonomic activity accompanying decision-making. © 2009 Elsevier Inc. All rights reserved.

Introduction To survive, animals and humans adapt to environments by forming appropriate behavioral and physiological responses. For most encounters in our everyday lives, the precise values of available options and the precise probabilities of association between the options and outcomes are generally not known. In this sense, our world is uncertain. Behaviorally, animals and humans can make decisions based on an evaluation of the contingency between available options and outcomes in an uncertain situation. Physiologically, modulation of autonomic activity is specifically critical for survival because decision-making leads to actions associated with seeking reward or escaping harm. Especially in an uncertain situation, a strategy for saving and allocating physical energy is essential for the success of such actions. Common neural mechanisms are likely to ⁎ Corresponding author. Department of Psychology, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan. Fax: +81 52 789 2220. E-mail address: [email protected] (H. Ohira). 1053-8119/$ – see front matter © 2009 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2009.07.060

underlie both behavioral decision-making and modulation of autonomic activity. The primary aim of the present study is to elucidate the functional association between the brain and autonomic responses during decision-making, according to change in the degree of uncertainty. Recently, the brain mechanisms underlying decision-making have been intensely studied. In monkeys (Tremblay & Schultz, 1999; Tobler et al., 2005) and in humans (O'Doherty et al., 2001), the orbitofrontal cortex (OFC) is involved in coding reward and punishment. The OFC is believed to have a role in the establishment of contingency between stimuli, actions, and outcomes in a specific situation, and thus is believed to contribute to decision-making (for a review, Roberts, 2006). The anterior cingulate cortex (ACC) has been reported to be activated during both reward-related decision-making (O'Doherty et al., 2004; Delgado et al., 2005; Hampton & O'Doherty, 2007) and punishment-related avoidance learning (Kosson et al., 2006). The ACC is involved in processing conflict and assessing which action alternatives are more desirable and should be undertaken (Bush et al., 1998; Carter et al., 2000). The dorsolateral prefrontal cortex

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

(DLPFC), which is involved in maintaining and updating representations about a task, the goal of a task, and task-related information, has also been reported to contribute to decision-making (Lee and Seo, 2007; Labudda et al., 2008). Furthermore, the dorsal striatum is a main target area of dopamine neurons that are connected with the mesolimbic dopamine system (Schultz, 2002; Wise, 2002). This area is critical for evaluating prediction errors of reward, which indicate that an event is better or worse than expected (Schultz et al., 1997; Montague et al., 2004). These brain regions form the cortico-striatal circuit (Haber, 2003), and calculations in this neural network can determine decision-making in uncertain situations (Haruno and Kawato, 2006; Cohen, 2008). In humans, event-related brain potentials (ERP) have been used to investigate the process of decision-making. Feedback signals conveying reward and punishment elicit a component of ERP, called feedback-related negativity (FRN). The reinforcement learning theory (Holroyd and Coles, 2002) posits that an outcome that is worse than expected results in a transient reduction of activity in dopamine neurons, which affects the medial prefrontal cortex (MPFC) and adjacent ACC. The MPFC and ACC have been suggested as the origin of the FRN, and FRN is believed to reflect the prediction error of reward conducted in such brain regions (Holroyd and Coles, 2002; Herrmann et al., 2004). However, whether the FRN reflects the processes of decision-making on the basis of reward prediction error is still unclear. Additionally, as described above, computation about reward and punishment values of available options, about evaluation of contingency of stimulus–action–outcomes, and about decisionmaking is probably conducted by functional neural networks including the OFC, ACC, MPFC, DLPFC, and striatum. Considering this, the amplitude of the FRN can be modulated by a wide range of brain regions, even if the direct electrical origin of the FRN is the MPFC and ACC. This issue has not yet been examined. On the other hand, some studies have reported a positive component of ERP related to feedback signals that is usually observed after the FRN and may be equivalent to the response-locked positive component called Pe (as a review, Overbeek et al., 2005). Because this positive component is sometimes sensitive both to reward and to punishment (Groen et al., 2007; Eppinger et al., 2008), we call this component feedback-related positivity (FRP). Though amplitudes of FRN and FRP are sometimes associated (Dywan et al., 2008), some functional dissociation has been reported. For example, amplitudes of the FRP but not the FRN showed changes related to progress of learning processes (Eppinger et al., 2008). However, detailed functional dissociation between FRN and FRP related to decision-making awaits elucidation. The ACC and MPFC also modulate peripheral physiological responses because these regions have direct neural projections to limbic and midbrain areas that regulate autonomic and endocrine activities (Kringelbach and Rolls, 2004; Kringelbach, 2005). Human neuroimaging studies indicated that activation in the MPFC and ACC correlated with autonomic activities as seen in cardiovascular activity and skin conductance during cognitive and stress tasks (Critchley et al., 2000a, 2000b, 2003, 2005; Gianaros et al., 2004, 2005; Matthews et al., 2004; Wong et al., 2007; Lane et al., 2009). Because changes in peripheral physiological responses are important for preparing the body to support appropriate behaviors, it is reasonable to infer that the peripheral responses may be modulated by the prefrontal brain regions that are involved in decision-making. Previous studies have provided support for this inference. During a stochastic learning task, feedback signaling errors elicited prolonged deceleration of heart rate compared with feedback signaling success, and the degree of such deceleration of heart rate correlated with amplitudes of the FRN (Groen et al., 2007). Such deceleration of heart rate can be interpreted as orienting response and may be mediated by cardiac vagal activity (Jennings and Van der Molen, 2002). Indeed, amplitudes of FRP during a source-monitoring task correlated with vagal tone measured by respiratory sinus arrhythmia (Dywan et al., 2008; see also Hajcak et

1025

al., 2003; 2004). On the basis of these findings, we inferred that variations of FRN, FRP, and autonomic parameters may reflect activities of the widely shared frontal neural system that contributes to performance monitoring, learning, and decision-making. Furthermore, we hypothesized that peripheral physiological change accompanying decision-making may be mediated via the vagus nerve system, which is under top-down modulation of the ACC and MPFC. However, to date, brain activity, ERP components, physiological responses, and possibly mediating vagal activity during decisionmaking are rarely examined in a single experimental setting, making it difficult to confirm such hypotheses. Therefore, we simultaneously measured regional cerebral blood flow (rCBF) using 15O-water PET, ERP components of the FRN and FRP, and heart rate (HR), mean blood pressure (MBP), and total peripheral resistance (TPR) as autonomic indices during a typical stochastic learning task. To estimate variations in vagal activity, we analyzed the high-frequency (HF) component of heart rate variability (HRV), which is thought to relate to respiratory sinus arrhythmia and is attributable to parasympathetic influence (Sayers, 1973). In the stochastic learning task where subjects chose one option from two alternatives, we manipulated contingency between options and reward/punishment to examine the brain and body harmonization accompanying decision-making as it corresponds to the degree of uncertainty of situations. In the contingent-reward condition, one option led to monetary reward at a probability of 70% and led to monetary punishment at a probability of 30%. Another option was associated with reward and punishment at an inverted probability (30% reward and 70% punishment). Thus subjects should relatively easily establish prediction about outcomes of their decision-making. On the other hand, in the random-reward condition, the reward and punishment were delivered randomly for both stimuli. Thus the situation for subjects was substantially uncertain, and they had to continue efforts to make predictions about outcomes. Methods Subjects Sixteen male volunteers (right-handed Japanese undergraduate and graduate students; age range, 19–28 years; mean age, 21.69 years, SD = 2.25) participated in the study. All subjects were healthy and were not taking any medications. The subjects reported that they had no past history of psychiatric or neurological illness. They gave written informed consent in accordance with the Declaration of Helsinki. This study was approved by the Ethics Committee of Kizawa Memorial Hospital. Task and experimental procedure The present article reports portions of findings of a combined study measuring behavioral, PET, EEG, cardiovascular, neuroendocrine, and immune parameters during a stochastic decision-making task. Neuroendocrine and immune data and some parts of the data on behavioral, PET, and cardiovascular indices have been reported elsewhere (Ohira et al., 2009). This article specifically focuses on the association between brain (PET and EEG) and autonomic activity accompanying decision-making. Subjects performed 8 blocks of a stochastic decision-making task: 3 blocks of a contingent-reward condition, 3 blocks of a random-reward condition, and 2 blocks of a control condition for subtraction analyses of PET images. Each block lasted for 4 min, with an 11-min interval from the previous block, and contained 40 trials. For each of the trials, subjects were presented 2 abstract line drawings on the left and right sides of a fixation stimulus for 700 ms, and they were required to choose one by pressing a key within the time of the presentation of the stimuli. The stimuli were

1026

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

selected from the set of Novel Shapes, which were validated on evaluation of the level of verbalization, association, and simplicity (Endo et al., 2001). The merit of using such abstract stimuli is to prevent subjects from verbal coding and from forming easy memory strategies. Then, 3500 ms after the termination of presentation of the stimuli, a feedback signal indicating reward (gain of 100 Japanese Yen (JPY)) or punishment (loss of 100 JPY) was shown for 1000 ms (Fig. 1a). If subjects did not choose a stimulus within 700 ms, they lost on that trial. In this task, the probability of reward and punishment associated with both stimuli was manipulated. In the contingent-reward condition, one stimulus was associated with reward at a probability of 70% but with punishment at a probability of 30% (advantageous stimulus), and the other stimulus was associated with reward at a probability of 30% and with punishment at a probability of 70% (disadvantageous stimulus) (Fig. 1b). On the other hand, in the random-reward condition, both stimuli were rewarded and punished with probabilities of 50%. Thus in this condition, the advantageous stimulus was operationally defined as a stimulus that was selected at random by the experimenters (Fig. 1c). Subjects performed each condition in 3 continuous blocks, and the order of the 2 conditions was counterbalanced between the subjects. The subjects were just instructed that the task was a gamble on each trial. Additionally, they were told that the amount of money which would be paid for participation in the experiment

would be modulated precisely according to their performance in the gamble. Furthermore, the subjects' task in the control condition was identical to the task in the other 2 conditions, except that the computer made the selection on each trial and subjects had to press a key corresponding to the stimulus that was chosen by the computer. Blocks for the control condition were placed in the 1st and 5th blocks, which mean that one control condition was followed by either blocks of the contingent-reward condition or blocks of the random-reward condition. The probability of reward and punishment associated with each stimulus in the control blocks was matched to the probability in the following experimental blocks, thus contingent-reward in one control block and random-reward in the other control block. Subjects were told that reward and punishment in the control conditions also would influence the amount of money paid for participation. Just before the 2nd block, the subjects were told that a gamble would start in this block, and just before the 6th block, they were informed that a new gamble with a different “rule”, which meant a different stimuli-reward/ punishment mapping would start in this block. The subjects had a PET scan of 60 s at the middle of each block. EEG and cardiovascular indices were recorded during each block and for 5 min before each block. At the end of the experimental session, subjects were fully debriefed about the purpose and manipulation of the experiment, and they were thanked. Though subjects were told that they would

Fig. 1. (a) The figure shows the time-course of a trial. (b) In the contingent-reward condition, an advantageous stimulus (the left stimulus in the figure) was associated with reward 70% of the time and punishment 30% of the time, and a disadvantageous stimulus was associated with reward 30% of the time and punishment 70% of the time. (c) In the random-reward condition, both stimuli were associated with reward 50% of the time and punishment 50% of the time. The advantageous stimulus was defined as the stimulus that a subject selected in the first trial of the first experimental block.

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

get money according to their performance, all subjects were paid 15,000 JPY for participation. Behavioral and self-report indices In each block, we examined 2 behavioral indices of decisionmaking: response bias and acquisition of reward. Response bias was defined as the rate of selection of the advantageous stimulus. Reward acquisition was defined as the rate of rewarded trials in 40 trials in each block. Before and after each block, subjects were asked to rate and report orally how they felt stressful in a scale from 0 (not stressful at all) to 100 (extremely stressful). Measurement of physiological data EEG (ERP) EEG data were recorded from 3 scalp sites, Fz, Cz, and Pz, on the basis of the International 10-20 system. These EEG channels were referenced to the subjects' nose tips and recorded using Ag/AgCl electrodes with an MP 100 system (BIOPAC Systems Inc. Goleta, CA, USA). The sampling rate was 250 Hz, and EEGs were recorded with a 0.1 Hz high-pass filter and a 35 Hz low-pass filter. Impedance was kept below 10 kΩ. A vertical electrooculogram of the left eye was monitored at the same time. EEG analysis was performed using Matlab 7.00 (the Mathworks Inc., Sherborn, MA, USA) and EEGLAB 4.311 (Delorme and Makeig, 2004). The trials in which the deviation of signal amplitude exceeded 100 μV were regarded as containing eyeblinks or motion artifacts and removed prior to averaging. In addition, trials in which subjects' latencies for stimulus selection were longer than 700 ms were considered as response error and removed from the analyses. The feedback-locked epochs (− 200 to +800 ms relative to the feedback presentation) of the artifact-free EEG data were low-pass filtered at 15 Hz offline and averaged. The average voltage of the time window from − 200 to 0 ms relative to the feedback was used as a baseline. Feedback-related ERP waveforms were separately calculated for reward and punishment signals for the advantageous stimulus and for the disadvantageous stimulus both in the contingent-reward and random-reward conditions. After visually identifying feedback-related ERP components in the grand-average ERP waveforms, peak amplitudes of the components were measured in each subject and subjected to statistical analyses. Cardiovascular indices Cardiodynamic activity was recorded using noninvasive finger blood pressure measurements as described previously (Kimura et al., 2007; Matsunaga et al., 2009). MBP and HR were recorded using the finger cuff of a Portapres Model 2 (Finapres Medical Systems Inc., Amsterdam, the Netherlands) attached to the third finger of the dominant arm of each subject. HR was recorded by photoplethysmography with a sampling rate of 200 Hz. Analyses of HR were performed with the Beatfast software using a model flow. Furthermore, TPR as an index of vascular activity was obtained by analyzing the sampled arterial pressure waveforms with the Beatfast software. Mean values of MBP, HR, and TPR were determined for 2 min just before the task as baseline and during 4 min of the task in each block. Furthermore, the interbeat-interval (IBI) data of HR were subsequently analyzed to yield heart rate variability (HRV) in the same 2 epochs as the other cardiovascular indices: baseline and during the task. The tachogram data on IBI were re-sampled at 4 Hz to obtain equidistant time-series values. A power spectrum density was then obtained through a fast Fourier transformation. In connection with the fast Fourier transformation, the data were detrended linearly and filtered through a rectangular window. The integral of the power spectrum was examined in a high-frequency band (0.15–0.5 Hz). For statistical analyses, we examined the HF component expressed as

1027

natural logarithm values of percentages of HF power of the total power in the spectrum (Perini et al., 2000). However, we report raw values of percentages of the HF power for means and standard errors (Table 2). Neuroimaging by PET Image acquisition During each block, the distribution of rCBF was measured with a General Electric Advance NXi PET scanner operated in a highsensitivity three-dimensional mode. A venous catheter for administering the tracer was inserted in an antecubital fossa vein in the left forearm. After the subject's head was positioned in the inflatable plastic head-holder that prevented possible head movements, a 10min transmission scan using a rotating 68germanium pin source was completed. In each block, following a 370-MBq bolus injection of H15 2 O over 30 s, scanning was started and continued for 60 s. Bolus injection was started 60 s after initiation of the block. The integrated radioactivity accumulated during 60 s of scanning was used as the index of rCBF. Eight scans were acquired per subject, and the interval between successive scans was 15 min to allow for radioactive levels to return to baseline level. A Hanning filter was used to reconstruct images into 35 planes with 4.5 mm thickness and a resolution of 2 mm × 2 mm (full width half maximum). Image processing and analysis SPM 99 (Friston et al., 1995) implemented in Matlab (version 5.3, the Mathworks Inc., Sherborn, MA, USA) was used for spatial preprocessing and statistical analyses. Images were initially realigned using sinc interpolation to remove artifacts before being transformed into a standard stereotactic space. Images were corrected for whole brain global blood flow by proportional scaling and smoothed using a Gaussian kernel to a final in-plane resolution of 10 mm at full width at half maximum. Subtraction analyses of images were conducted in the following 2 stages. First, separately in the contingent-reward condition and in the random-reward condition, images in 2 blocks of the control condition were subtracted from images in the experimental blocks to reveal significant increases of rCBF in each condition (main effects of contingent-reward and random-reward). Second, direct comparisons between the contingent-reward and random-reward were conducted in the following manners: (contingent-reward—control) − (randomreward—control) and (random-reward—control) − (contingent-reward—control). Effects at each voxel were estimated using a general linear model. Voxel values for each contrast yielded a statistical parametric map of the t statistic (SPM t) and were subsequently transformed to a unit normal distribution (SPM Z). Peak voxel-value significance thresholds were set at p b .001(uncorrected), and cluster significance thresholds were set at 20 voxels. Then, to identify brain regions that showed an increase in rCBF in synchrony with behavioral, ERP, and cardiovagal activity as a mediator of cardiovascular responses, correlation maps were composed for each condition. Response bias and reward acquisition were used as behavioral indices. For ERP indices, amplitudes of observed components locked to feedback signals were used. When electrically negative components were found, absolute values of the amplitudes were used to make interpretations clear (positive correlations with rCBF represent covariance of brain activation and large ERP amplitude). As a cardiovagal index, change values of the HF component of HRV were computed by subtracting means of values at baseline from means of values during the stochastic decision-making task. All these indices were used as covariates for the correlation analyses. The OFC, ACC, MPFC, DLPFC, and dorsal striatum were primary regions of interest for the correlation analyses on the basis of the logic described in the Introduction. For all correlation analyses, the statistical threshold was set at p b .001 (uncorrected) for height, and clusters larger than 20

1028

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

contiguous voxels were reported. Those relatively conservative thresholds were determined considering inflation of type-I error rates in many correlation analyses in the present study and expansion of cluster sizes of activated voxels by image smoothing. The above thresholds kept the probability of a false positive to a minimum according to analytical conditions used in the present study (Forman et al., 1995). Statistical analyses For behavioral data, response bias and reward acquisition in each block were separately subjected to two-way (Condition [contingentreward vs. random-reward] × Block [1, 2, 3]) repeated-measures analyses of variance (ANOVAs). For self-report data, rating of subjective stress was subjected to a three-way (Condition [contingent-reward vs. random-reward] × Period (before block vs. after block) × Block [1, 2, 3]) repeated-measures ANOVA. For ERP data, the amplitudes of components were subjected to three-way (Condition [contingent-reward vs. random-reward] × Stimulus [advantageous vs. disadvantageous] × Feedback [reward vs. punishment]) repeated-measures ANOVAs. To get enough numbers of trials for each category, blocks were collapsed for analyses of ERP data. For cardiovascular data, values in the baseline of each block and values during the block were subjected to three-way (Condition [contingent-reward vs. random-reward] × Period [baseline vs. task] × Block [1, 2, 3]) repeated-measures ANOVAs. The Greenhouse–Geisser epsilon correction factor, ɛ (Jennings & Wood, 1976), was used where appropriate. In cases where significant effects were found in the ANOVAs, post hoc analyses using Tukey's test (p b .05) were conducted to examine which combinations of data points differed significantly. Results Behavioral and self-report data Means and standard errors of response bias and reward acquisition are shown in Table 1. An ANOVA revealed a significant main effect of Condition for response bias (F (1, 15) = 7.10, p b .05, η2p = .32). Neither a main effect of Block nor an interaction of Condition and Block was significant (F b 1.78). It is not surprising that reward acquisition showed similar results as response bias, specifically, a significant main effect of Condition (F (1, 15) =13.49, p b .01, η2p = .47, Table 1). These results suggested that experimental manipulation of the degree of uncertainty in the present study successfully produced differentiated patterns of decisionmaking between the contingent-reward and random-reward conditions. Throughout the experimental session, subjects reported moderate level of subjective stress (M = 40.5–50.0) probably due to the situation of the PET experiment. However no significant effect was observed in the ANOVA on subjective stress, suggesting that results of brain and autonomic responses are unlikely to be attributed to variations of emotional states such as stress or anxiety but rather should be due to characteristics of conditions of decision-making.

Table 1 Means and standard errors for behavioral indices.

Response bias (rate)

Reward acquisition(rate)

Block

Contingent-reward

Random-reward

1 2 3 1 2 3

.61 .70 .74 .56 .57 .58

.45 (.06) .50 (.08) .44 (.08) .46 (.02) .48 (.01) .49 (.01)

(.07) (.07) (.07) (.02) (.03) (.01)

Fig. 2. (a) Grand-average event-related potential waveforms time-locked to onset of feedback signal as a function of stimulus (advantageous [A] vs. disadvantageous [D]) and feedback (reward vs. punishment) in the contingent-reward condition. (b) Grandaverage ERP waveforms time-locked to onset of feedback signal as a function of stimulus (advantageous [A] vs. disadvantageous [D]) and feedback (reward vs. punishment) in the random-reward condition.

ERP data A negative peak embedded in a following larger positive peak was observed by visual inspection of the grand-average ERP waveform in each condition (Fig. 2). These components seem consistent with FRN and Pe, which have been reported in previous studies (e.g., Holroyd and Coles, 2002; Nieuwenhuis et al., 2002). Because we observed these components not only in cases of feedback of punishment (error) but also in cases of feedback of reward (correct), we call these components FRN and FRP, respectively. Furthermore, when necessary, FRN and FNP that are associated with feedback signals of reward and punishment are called rFRN, pFRN, rFRP, and pFRP, respectively. Referencing previous studies (Oliveira et al., 2007; Eppinger et al., 2008; Polezzi et al., 2008), the FRN was defined as the difference between the most negative peak within 200–400 ms from the onset of a feedback signal and the preceding positive peak. Also, FRP was defined as the most positive peak in the window between the ERN and 500 ms from onset of a feedback signal. Following Dywan et al. (2008), amplitude values at Cz were subjected to analyses (Fig. 3). For FRN, an

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

1029

Fig. 3. (a) Means and standard errors of amplitudes of feedback-related negativity (FRN) as a function of condition (contingent-reward vs. random-reward), stimulus (advantageous [A] vs. disadvantageous [D]), and feedback (reward vs. punishment). (b) Means and standard errors of amplitudes of feedback-related positivity (FRP) as a function of condition (contingent-reward vs. random-reward), stimulus (advantageous [A] vs. disadvantageous [D]), and feedback (reward vs. punishment).

ANOVA revealed a significant three-way interaction of Condition, Stimulus, and Feedback (F (1, 15) = 8.21, p b .05, η2p = .35). Further analyses indicated that the punishment feedback compared with the reward feedback for the advantageous stimulus and the reward feedback compared with the punishment feedback for the disadvantageous stimulus elicited larger negative amplitudes in the contingent-reward condition. In the random-reward condition, the punishment feedback elicited larger FRN than the reward feedback for both stimuli. Though a similar interaction (but inverted direction, contingent-reward condition, reward N punishment for advantageous stimulus and reward b punishment for disadvantageous stimulus; random-reward condition, reward N punishment for both stimuli) was visually observed for FRP, such an effect did not exceed statistical threshold (F (1, 15) = 2.59, ns. η2p = .15). Furthermore, amplitudes of FRN and FRP showed no correlation in any condition (r b .30); thus these two components seem functionally independent.

in all blocks. On the other hand, TPR showed a significant interaction of Condition and Block (F (1, 15) = 4.59, p b .05 η2p = .23), suggesting that TPR was remarkably raised in the random-reward condition in the third block. The contraction of peripheral vessels reflected by an increase of TPR seemed to boost MBP in the random-reward condition; thus in other words, cardiac activity was suppressed in the random-reward condition throughout the decision-making task. This idea is supported by a consistent main effect in HF component of HRV, which reflects cardiovagal activity (F (1, 15) = 5.63, p b .05 η2p = .27), clearly suggesting that inhibitory control by vagal nerves was more dominant in the random-reward condition compared with that in the contingent-reward condition. Brain and vagal activity mediating decision-making To elucidate the association of brain and bodily activities with decision-making, we conducted a series of regression analyses with

Cardiovascular data Results of cardiovascular data are summarized in Table 2. For MBP, main effects of Condition and Period were significant (F (1, 15) = 6.18, p b .05, η2p = .29; F (1, 15) = 37.58, p b .001 η2p = .71); additionally, a three-way interaction of Condition, Block, and Period failed to reach a significant level (F (1, 15) = 3.18, p = .066 η2p = .17). Tukey's tests revealed that MBP during the task in the contingent-reward condition was higher than that in the random-reward condition in the first and second blocks; however, MBP during the task in the random-reward condition was enhanced and became identical with that in the contingent-reward condition in the third block. The HR and TPR results can explain this complicated interaction for MBP. In principle, MBP is determined by a multiplication of cardiac activity reflected by HR and vascular activity reflected by TPR. HR showed a significant main effect of Condition (F (1, 15) = 6.92, p b .05 η2p = .32), and further tests indicated HR was continuously higher in the contingentreward condition compared with HR in the random-reward condition

Table 2 Means and standard errors of cardiovascular parameters. Block MBP (mm Hg)

HR (bpm)

TPR(AU)

HF/HRV (%)

1 2 3 1 2 3 1 2 3 1 2 3

Contingent-reward

Random-reward

Baseline

Task

Baseline

Task

79.08 80.52 79.96 65.06 65.84 65.24 .78 .78 .78 51.16 50.09 52.31

83.26 83.07 82.04 67.16 66.54 66.78 .79 .79 .78 55.53 50.09 52.05

76.12 (2.53) 76.85 (2.08) 77.62 (1.92) 63.77 (2.12) 63.50 (2.13) 63.98 (2.12) .73 (.05) .75 (.05) .76 (.06) 53.07 (3.52) 51.15 (3.55) 51.14 (5.50)

79.62 79.17 81.88 64.95 64.15 65.47 .79 .76 .82 61.70 57.54 55.32

(1.70) (1.85) (1.83) (2.02) (2.24) (2.04) (.05) (.05) (.05) (4.03) (4.63) (4.20)

(1.93) (1.46) (1.72) (2.66) (2.45) (2.43) (.06) (.05) (.05) (3.30) (4.63) (3.85)

(2.49) (2.10) (1.65) (2.22) (2.13) (2.15) (.06) (.05) (.06) (3.64) (3.86) (4.61)

MBP, mean blood pressure; HR, heart rate; TPR, total peripheral resistance; AU, arbitrary unit; HF/HRV, high-frequency component of heart rate variability.

1030

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

response bias as a dependent variable. Amplitudes of FRN and FRP, which may reflect brain processes of contingency evaluation and decision-making, and the HF power, which may reflect cardiovagal-mediated inhibitory control, were used as independent variables. All of these variables were calculated throughout all blocks of the contingent-reward condition and the random-reward condition, separately. For FRN and FRP, we calculated the rate of amplitudes of the components to reward signal for the advantageous stimulus and for the disadvantageous stimulus and the rate of amplitudes of the components to punishment signal for the advantageous stimulus and for the disadvantageous stimulus in the following ways. rFRNrate pFRNrate rFRPrate pFRPrate

= rFRNA = ðrFRNA + rFRND Þ = pFRNA = ðpFRNA + pFRND Þ : = rFRPA = ðrFRPA + rFRPD Þ = pFRPA = ð pFRPA + pFRPD Þ

Here, r and p represent reward and punishment, respectively, and A and D represent the advantageous stimulus and the disadvantageous stimulus, respectively. Thus these parameters indicate to what degree the components of ERP locked to reward and punishment were relatively dominant for the advantageous stimulus compared with the disadvantageous stimulus. We used these relative values of ERP amplitudes as independent variables, first, to avoid multicollinearity and, second, because of interactions of feedback (reward vs. punishment) and stimulus (advantageous vs. disadvantageous) for ERP amplitudes revealed by the ANOVAs described above. For the HF power, we used change values of log-transformed HF power from

Table 3 Significant increases in rCBF between experimental blocks and control blocks. Region

Side

a. Contingent-reward—control Pre-SMA Pre-SMA Cerebellum Pons ACC

R L R R R

b. Random-reward—control DLPFC OFC ACC OFC Cerebellum OFC

R R R L R L

BA 6 6

32

9 11 32 11 11

x

y

z

Z score

16 − 26 14 8 10

14 − 18 − 42 − 30 36

42 52 − 38 − 38 28

4.50 4.31 4.24 3.65 3.62

42 22 6 − 24 10 − 16

20 46 28 50 − 42 40

32 − 20 34 − 14 − 32 − 22

4.61 4.44 4.23 3.86 3.80 3.37

Coordinates are in MNI space (SPM99). R, right; L, left; BA, Brodmann's area; Pre-SMA, pre-supplementary motor area; ACC, anterior cingulate cortex; OFC, orbitofrontal cortex; DLPFC, dorsolateral prefrontal cortex; PCC, posterior cingulate cortex; x, y, z, three-dimensional coordinates used to determine a voxel referring to medial-lateral (x: positive = right), anterior-posterior (y: positive = anterior), and superior-inferior (z: positive = superior) positions; rCBF, regional cerebral blood flow.

baseline to task periods in each block to reflect phasic and transient variation of cardiovagal control. At first, we simultaneously entered all the independent variables, rFRNrate, pFRNrate, rFRPrate, pFRPrate, and HF power, to a regression equation to explain the variance of response bias in each condition. However, the regression equations were not significant (F b 1.0). Next, following Dywan et al. (2008), we constructed regression models using FRN, FRP, HF power, and response bias represented in Fig. 4a and ran analyses separately for the contingent-reward condition and the random-reward condition (Figs. 4b and c). Regression equations using FRN were not significant in either condition (F b 1.0); thus FRN may not be a strong predictor of either response bias or HF power. In analyses in the contingent-reward condition using FRP, as a first equation, the dependent variable (response bias) was regressed on HF

Table 4 Significant increases in rCBF between contingent-reward blocks and random-reward blocks.

Fig. 4. (a) Regression models predicting response bias by event-related potentials and cardiovagal activity reflected by high-frequency power of heart rate variability (HF/HRV). (b) Results of regression analyses in the contingent-reward condition. (c) Results of regression analyses in the random-reward condition. rFRN rate, rate of amplitudes of feedback-related negativity (FRN) between the reward for advantageous stimulus and the reward for disadvantageous stimulus ; pFRN rate, rate of amplitudes of FRN between the punishment for advantageous stimulus and the punishment for disadvantageous stimulus; rFRP rate, rate of amplitudes of feedback-related positivity (FRP) between the reward for advantageous stimulus and the reward for disadvantageous stimulus; pFRP rate, rate of amplitudes of FRP between the reward for advantageous stimulus and the reward for disadvantageous stimulus (see the text in detail).

Region

Side

BA

x

y

z

Z score

a. Contingent-reward—random-reward Parastriate cortex Parastriate cortex Inferior parietal cortex Inferior temporal cortex Pons

L L L L L

19 18 40 37

− 50 − 36 − 64 − 60 −8

− 76 − 90 − 32 − 62 − 38

16 2 38 − 10 − 22

5.14 4.55 3.86 3.80 3.74

b. Random-reward—contingent-reward DLPFC DLPFC Middle temporal cortex Middle temporal cortex Pre-SMA Inferior parietal cortex Putamen Pulvinar Pre-SMA Parahippocampal gyrus Superior temporal cortex Middle temporal cortex ACC Frontopolar area

R R R R R R R R R L R R L L

9 46 21 21 6 40

42 44 58 60 34 60 10 12 6 − 20 58 56 − 10 −8

22 46 −2 −8 6 − 28 − 30 8 −8 0 −8 − 50 32 50

32 18 − 18 −4 46 32 0 2 70 − 30 14 −2 18 20

4.54 4.49 4.45 4.30 4.18 4.17 3.97 3.94 3.87 3.83 3.72 3.68 3.61 3.47

6 21 22 21 32 10

Coordinates are in MNI space (SPM99). R, right; L, left; BA, Brodmann's area; DLPFC, dorsolateral prefrontal cortex; Pre-SMA, pre-supplementary motor area; ACC, anterior cingulate cortex; x, y, z, three-dimensional coordinates used to determine a voxel referring to medial-lateral (x: positive = right), anterior-posterior (y: positive = anterior), and superior-inferior (z: positive = superior) positions; rCBF, regional cerebral blood flow.

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

power, resulting in an insignificant contribution (β = −.05, ns; F (1, 13) = .08, ns). A second equation involved regressing response bias on relative FRP related to reward signal (rFRPrate) and relative FRP related to punishment signal (pFRPrate) as the independent variables, which resulted in significant associations (β = −.54 for rFRPrate, β = .50 for pFRPrate ; [F (2, 12) = 10.27, p b .01]). As a third step, HF power, as regressed on rFRPrate and pFRPrate, resulted in a significant association of pFRPrate (β = .55; [F (2, 12) = 3.50, p b . 05]). Finally, response bias was regressed on all dependent variables, rFRPrate, pFRPrate, and HF power, indicating a constant contribution of rFRPrate and pFRPrate to response bias (β = −.54 for rFRPrate, β = .53 for pFRPrate, andβ = −.05 for HF power; [F (3, 11) = 6.33, p b .01]). These results indicated that smaller FRP to reward signal after selecting the advantageous stimulus rather than the disadvantageous stimulus and larger FRP to punishment signal after selecting the advantageous stimulus rather than the disadvantageous stimulus contributed to a higher rate of selection of the advantageous stimulus (response bias). Furthermore, cardiovagal activity was affected by FRP, especially FRP related to signals of punishment; however, that did not mediate decisionmaking. The same procedure of analyses was carried out in the random-reward condition and indicated that response bias was significantly affected by rFRPrate (β = −.62, F (3, 12) = 4.40, p b .05), suggesting that a smaller FRP to reward signal, after the subject selects

1031

the advantageous stimulus operationally defined by experimenters, is associated with higher selection of the stimulus. PET data: subtraction analyses Results of subtraction analyses are summarized in Table 3 and Table 4. Both subtraction of contingent-reward blocks minus control blocks and subtraction of random-reward blocks minus control blocks revealed shared significant increases of rCBF in the dorsal ACC (BA32, x = 10, y = 36, z = 28, for the contingent-reward condition; BA32, x = 6, y = 28, z = 34, for the random-reward condition) and the cerebellum (x = 14, y = − 42, z = −38, for the contingent-reward condition; x = 10, y = − 42, z = −32, for the random-reward condition). Furthermore, unique significant main effects of the contingentreward were shown in 2 loci of the pre-supplemental motor area (Pre-SMA) and pons (Table 3, Fig. 5a). Significant activation, unique to the random-reward, was in the right DLPFC and bilateral OFC (Fig. 5b). A direct comparison of the contingent-reward blocks minus the random-reward blocks revealed significant activation in the parastriate cortex of the visual areas, inferior parietal and inferior temporal cortices, and pons (Table 4, Fig. 6a). The comparison of the random-reward blocks minus the contingent-reward blocks clarified robust activation in several loci of the right DLPFC and putamen (Fig.

Fig. 5. (a) Statistical parametric map (SPM99) showing significant increases of regional cerebral blood flow (rCBF) in the contingent-reward blocks minus the control blocks. (b) Statistical parametric map (SPM99) showing significant increases of rCBF in the random-reward blocks minus the control blocks. An uncorrected p value of .001 was used as the threshold for each subtraction analysis. A, anterior cingulate cortex; B, cerebellum; C, pre-supplemental motor area; D, pons; E, dorsolateral prefrontal cortex; F, orbitofrontal cortex.

1032

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

Fig. 6. (a) Statistical parametric map (SPM99) showing significant increases of regional cerebral blood flow (rCBF) in the contingent-reward blocks minus the random-reward blocks. (b) Statistical parametric map (SPM99) showing significant increases of rCBF in the random-reward blocks minus the contingent-reward blocks. An uncorrected p value of .001 was used as the threshold for each subtraction analysis. A, parastriate cortex; B, inferior temporal cortex; C, inferior temporal cortex; D, pons; E, dorsolateral prefrontal cortex; F, putamen.

6b). Further activation was shown in the middle and superior temporal gyri, pre-SMA, pulvinar, parahippocampal gyrus, and ACC (Table 4). PET data: correlation analyses Contingent-reward condition Amplitudes of FRN and FRP commonly showed positive correlations with increases in rCBF in the OFC (BA11, x = 26, y = 44, z = −26, Z = 5.32 for FRN; BA11, x = 24, y = 44, z = − 28, Z = 4.99 for FRP), ACC (BA32, x = 20, y = 42, z = 22, Z = 3.84 for FRN; BA32, x = 12, y = 46, z = 10, Z = 5.01 for FRP), and putamen (x = 16, y = − 2, z = 2, Z = 3.54 for FRN; x = 14, y = −2, z = 2, Z = 4.48 for FRP) in the contingentreward condition. Additionally, FRP but not FRN correlated with rCBF in several loci of the DLPFC (BA9, x = 26, y = 24, z = 32, Z = 4.90; BA46, x = 52, y = 44, z = 18, Z = 4.43) (Fig. 7). For cardiovagal activity, a significant correlation between change values of HF power and increase in rCBF was in the precuneus (BA7, x = 18, y = −64, z = 38, Z = 4.41) and expanded to posterior cingulate cortex (Fig. 8a). Behavioral parameters showed no significant correlations with rCBF.

Random-reward condition Neither FRN nor FRP significantly correlated with rCBF in the hypothesized brain regions (OFC, MPFC, ACC, DLPFC, and dorsal striatum) in the random-reward condition. Consistent with the hypothesis, the change values of HF power positively correlated with rCBF in the MPFC (BA10, x = 2, y = 52, z = 8, Z = 3.77) whose activation was expanded to the adjacent rostral ACC (BA32). Furthermore, rCBF in the DLPFC (BA46, x = 40, y = 26, z = 28, Z = 3.28) showed a positive correlation with HF power (Fig. 8b). Behavioral parameters showed no correlations with rCBF. Discussion Neural correlates of decision-making under uncertainty In the present study, the same visual stimuli and motor responses were used in both the experimental and control blocks, and thus the only difference between the experimental and control blocks was whether subjects made decisions or not (O'Doherty et al., 2004). We subtracted PET images in the same two control blocks (as a baseline) from PET images in each of two experimental conditions, manipula-

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

1033

Fig. 7. Results of correlation analyses in the contingent-reward blocks showing significant positive correlations between components of event-related potentials and regional cerebral blood flow. An uncorrected p value of .001 was used as the threshold. FRN, feedback-related negativity; FRP, feedback-related positivity; A, orbitofrontal cortex (OFC); B, anterior cingulate cortex (ACC); C, putamen; D, dorsolateral prefrontal cortex (DLPFC).

ting the degree of uncertainty (contingent-reward and randomreward), respectively. After that, we directly compared the contrast images between the contingent-reward condition and the randomreward condition. This study design enabled us to dissociate uncertainty-independent brain regions, which were commonly activated during decision-making regardless of uncertainty, from uncertainty-dependent brain regions that were sensitive to the degree of uncertainty. Activation of the ACC (BA32) and cerebellum occurred in both the contingent-reward and random-reward conditions, and the locations of activation peaks in those regions were very close between the two experimental conditions, confirming that the ACC and cerebellum are uncertainty-independent regions. The ACC is usually more sensitive to negative outcomes than to positive outcomes (Gehring and Willoughby, 2002; Nieuwenhuis et al., 2004) and probably enables the detection of behavioral conflict (Botvinick et al., 1999; van Veen et al., 2004) and errors (Carter et al., 1998; Holroyd and Coles, 2002;

Nieurwenhuis et al., 2004) to correct them quickly. The ACC was continuously activated during decision-making, even after completion of learning about contingency between stimuli and positive and negative outcomes (Sailer et al., 2007). Thus the present result suggests that the ACC may be involved in on-line monitoring of outcomes following decision-making. It should be noted that the present task is somewhat uncertain even in the contingent-reward condition and that punishment was delivered at a probability of 30% even with the most optimal choice. Thus subjects had to continue monitoring contingency between stimuli, choices, and outcomes. The cerebellum has also been reported to be involved in decision-making under uncertainty (Blackwood et al., 2004) and conflict resolution (Schweizer et al., 2007), probably by constructing internal working models of potential events in a certain context and subserving prediction of outcomes (Blackwood et al., 2004). Therefore, the ACC and cerebellum can work as a default system for decision-making with uncertainty.

1034

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

Fig. 8. Results of correlation analyses in the contingent-reward (a) and the random-reward (b) blocks showing a significant positive correlation between high-frequency power of heart rate variability and regional cerebral blood flow. An uncorrected p value of .001 was used as the threshold. A, precuneus; B, posterior cingulate cortex (PCC); C, medial prefrontal cortex (MPFC), anterior cingulate cortex (ACC); D, dorsolateral prefrontal cortex (DLPFC).

On the other hand, the OFC, right DLPFC (BA46), and putamen appear to produce uncertainty-dependent responses; that is, these regions were more activated in the random-reward condition compared with the contingent-reward condition. The OFC is thought to code the incentive value of a wide range of stimuli, from primary reward such as food and drink to abstract secondary reward such as money and game points (Krawczyk, 2002; Breiter et al., 2001; Elliott et al., 2000; O'Doherty et al., 2004). Especially, the OFC has been reported to play critical roles in prediction of rewards immediately following sensorimotor events (Tremblay & Schultz, 2000; Critchley et al., 2001) rather than on a long timescale (McClure et al., 2007), as well as in action selection on the basis of such immediate reward prediction (Rogers et al., 1999; O'Doherty et al., 2003). The DLPFC, which has reciprocal connections with the putamen (Alexander et al., 1990) and thus sometimes shows coupled activation with the putamen (Postuma and Dagher, 2006), has shown involvement in decision-making, especially when delivery of reward was delayed (McClure et al., 2004, 2007) and when the task required prediction of a future large reward and bearing immediate small loss (Tanaka et al., 2004). Such a role for the DLPFC may be supported by the function of this region, which includes working memory, attention control, and top-down control over flow of information processing (Lee and Seo, 2007). Thus this region seems to be more involved in decision-making in an uncertain situation where one has to seek rules or laws by memorizing past experiences of one's own actions and the outcomes on a relatively long timescale. Furthermore, the right side of the DLPFC has been reported to play a crucial role in the suppressive control of superficially seductive options (Fecteau et al., 2007). This function may also contribute to learning on a long

timescale. The dorsal striatum, including the putamen, is believed to identify prediction-error signals indicating an outcome is better or worse than expected (Montague et al., 2004; Schultz et al., 1997). Contrary to the ventral striatum, which responds to immediate reward, the dorsal striatum was associated with reward prediction at longer timescales (Tanaka et al., 2007). Considering the recent finding that the OFC, DLPFC, and putamen showed strong activation in an initial stage of reward-based learning but showed reduction in activation after completion of the learning (Sailer et al., 2007), these brain regions may finish their roles once representation of contingency about stimuli–actions–outcomes is established. Thus we infer that activity of these regions rapidly decreased in the contingent-reward condition of the present study because subjects could learn such contingency relatively easily. In contrast, these regions maintained their activity in the randomreward condition, where the situation was more uncertain and subjects had to continue efforts to find the contingency, resulting in the activation of these regions in a direct comparison of the randomreward condition minus the contingent-reward condition. Taken together, these results of PET imaging in the present study were consistent with previous studies and provided additional suggestions about dissociable roles of prefrontal and striatum areas in decisionmaking with uncertainty. Functional significance of feedback-related brain potentials Specifically in the contingent-reward condition where subjects could make decisions based on prediction, activity in the OFC, ACC, and putamen correlated with amplitudes of FRN and FRP commonly, whereas activation in the DLPFC selectively correlated with ampli-

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

tudes of FRP but not those of FRN. Previously, patients with lesions in the OFC (Turken and Swick, 2008), basal ganglia including the putamen (Ullsperger and von Cramon, 2006), and lateral prefrontal cortex including the DLPFC (Gehring and Knight, 2000; Ullsperger and von Cramon, 2006) have shown absence or distortion of responselocked negative error-related components (Ne) of ERPs. In addition, intracerebral ERP recording clarified Ne and Pe error-related components from sites in the ACC, OFC, and DLPFC (Brázdil et al., 2002). If the response-locked Ne and Pe reported in those previous studies were equivalent to the feedback-locked FRN and FRP observed in the present study, it would be reasonable to posit that our results suggested involvement of not only the ACC and MPFC, but also those multiple brain structures involved in the genesis of FRN and FRP. Simultaneous recording of PET images and ERPs in the present study enabled us to interpret the functional significance of feedback-related components such as FRN and FRP during decision-making. Considering the suggested dissociable roles of the DLPFC and other prefrontal regions described above, the robust correlation of DLPFC activation and FRP suggests that FRP may reflect an evaluation for outcomes of decision-making on a relatively longer timescale based on past memories and future planning, which should be borne by the DLPFC. On the other hand, an absence of correlation of DLPFC activation and FRN suggests that FRN may reflect an evaluation of outcomes on a shorter timescale based on on-line calculation of prediction errors about reward and punishment, which should be the responsibility of the OFC and ACC. This speculation seems consistent with the present results of the regression analyses that FRP but not FRN predicted behavioral performance (response bias), especially in the contingent-reward condition. More specifically, a smaller FRP to reward signal after selecting the advantageous stimuli and a larger FRP to punishment signal after selecting the advantageous stimulus were associated with a higher selection rate of the advantageous stimulus. This pattern is consistent with the prediction errors (small amplitude to predicted reward and large amplitude to unpredicted punishment). Referring previous suggestions that FRP may reflect conscious and attention processes (Nieuwenhuis et al., 2001; Overbeek et al., 2005), it is reasonable to consider that this component may reflect processes to link estimated prediction errors and explicit behavioral decision-making. FRN was also sensitive not only to unpredicted punishment for the advantageous stimulus but also to unpredicted reward for the disadvantageous stimulus; thus this component may code an absolute value of prediction error about events that are worse or better than predicted. Such a pattern of FRN variation is consistent with recent previous studies (Oliveira et al., 2007; Eppinger et al., 2008); however, FRN did not correlate with the response bias. Thus FRN reflects a step of processing for decisionmaking, but it does not link directly with decision-making itself (Polezzi et al., 2008). Furthermore, such interactions of stimuli and feedback with FRP and FRN can explain the lack of correlations between rCBF and behavioral decision-making. In spite of its great merit for simultaneous recording, neuroimaging by PET inevitably reflects only total activation of brain regions in each block. To create brain images reflecting trial-by-trial interactions of plural factors of brain activation, event-related functional magnetic resonance imaging (fMRI) is optimal. These interpretations are additionally supported by results in the random-reward condition of the present study. In this condition, FRN were consistently larger for punishment signal than for reward signal. Hypothesizing that humans place emphasis on predicting reward rather than punishment at least during the initial stages of learning (Sailer et al., 2007), it is possible that a larger prediction error would occur for punishment than reward. Then, signals of punishment would elicit a larger FRN in a situation where feedback of reward and punishment was delivered randomly and subjects had to seek the contingency of stimuli–actions–outcomes. In the random-reward

1035

condition, no contingency could be drawn. Nevertheless, regression analyses indicated that FRP related to reward was associated with decision-making even in this condition. This can be interpreted such that the FRP reflects conscious and subjective evaluations about feedback signals, as suggested by previous studies (Nieuwenhuis et al., 2001; Overbeek et al., 2005). Namely, in this situation, FRP may reflect the subjects' illusory predictions for reward, which were delivered randomly, and affected decision-making based on the prediction. In sum, considering such correspondences between activated brain regions in PET imaging and FRN and FRP, we infer that FRN and FRP may reflect processes of prediction-error calculation of reward and punishment, evaluation of stimulus–action–outcome contingency, and choice of actions that are conducted in the prefrontal-striatal network, including the OFC, ACC, DLPFC, and putamen. Modulation of autonomic activity accompanying decision-making Whereas cardiac activity reflected by HR was dominant during the task in the contingent-reward condition, in the random-reward condition, cardiac activity was relatively suppressed and vascular activity reflected by TPR was gradually enhanced. These dissociable patterns of cardiovascular responses are known as active and passive physiological stress coping (Blascovich et al., 1999; Keay and Bandler, 2001). Active coping, which is characterized by a remarkable elevation of HR and hypertension, is elicited when individuals can control the stressor and can mobilize efficient energy to cope with the stress. On the other hand, passive coping, which is characterized by a decrease of HR and increase of TPR, is elicited when individuals cannot control the stressor and experience insufficient resources (Carruthers and Taggart, 1973; McCabe and Schneiderman, 1985; Schneiderman and McCabe, 1989). Enhancement of cardiac activity in active coping means mobilization of physical energy to cope with challenging environments, and attenuation of cardiac activity in passive coping suggests prevention of energy expenditure by cutting off provided energy to ongoing behaviors and physiological responses that have become inappropriate. Using a stochastic learning task similar to the task used in the present study (Kimura et al., 2007; Ohira et al., 2009) and using a simpler mental arithmetic task (Ohira et al., 2008), we showed a typical pattern of passive stress coping in cardiovascular responses during a situation where stressor controllability was low. Thus the present autonomic response results are consistent with previous findings from our own laboratory and others, suggesting that autonomic activity can be down-regulated when the situation is uncertain, probably to secure available physical energy. Although such down-regulation of cardiac responses can be done either by reduction of sympathetic activity, enhancement of vagal activity, or both, the vagus nerve system may play a critical role in such flexible regulation (Thayer and Brosschot, 2005). Because of differences in the temporal kinetics of neuroeffectors, sympathetic effects are relatively slow compared with vagal effects (Saul et al., 1990); thus the latter ones should be more suitable for fast and delicate regulation. For adaptation to constantly changing environmental demands, patterns of organized variability, rather than static levels, are required in the central brain and peripheral physiological systems that are mediated by vagal activity (Thayer and Brosschot, 2005; Beauchaine et al., 2007). The present data confirmed this hypothesis. Corresponding to a reduction of HR, HF power was more dominant during the decision-making task in the random-reward condition, where the situation was more uncertain, than in the contingent-reward condition. Also, the observed positive correlation between HF power and rCBF in the MPFC and adjacent rostral ACC in the random-reward condition is consistent with many previous reports (Lane et al., 2001; Critchley et al., 2003; Gianaros et al., 2004; Wong et al., 2007; O'Connor et al., 2007; Lane et al., 2009). The

1036

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037

correlation between rCBF in the DLPFC and HF power in the randomreward condition is also consistent with a previous report (Napadow et al., 2008). These findings suggested that those prefrontal regions are centers of cortical modulation over cardiovagal activity. Another merit of the present study is the clarification of causal relations between processing in the prefrontal-striatal network as expressed by FRP, vagal activity reflected by HF power, and behavioral decision-making. Our regression analyses revealed that FRP affected both HF power and behavioral decision-making in a parallel manner. Specifically, a larger FRP to punishment signal after selecting the advantageous stimulus is associated with higher HF power in the contingent-reward condition. In other words, when learning progressed and predictions of outcomes were established, autonomic arousal may have been attenuated via increased vagal activity, probably because decisions became easier and there was no need for additional efforts. On the other hand, in the random-reward condition, where the situation was totally uncertain, the neural basis strongly induced exploratory modes, resulting in larger FRP to punishment signals and increased HF power, which should be linked to smaller individual differences in these indices and to a lack of correlations. Furthermore, in both conditions, HF power did not mediate effects of FRP on decision-making. These findings suggest that the prefrontal-striatal network was sensitive to uncertainty in a situation where subjects had to make decisions and, at the same time, participated in down-regulation of autonomic activity to adjust to environmental demands. Limitations and conclusion Several limitations in the present study should be discussed. First, considering the large individual differences in the behavioral performance of decision-making and physiological responses, the sample size of this study was not large enough. Probably, some correlations did not reach statistical significance because of small statistical power. Also, the present study examined only male subjects. Thus, generality of the present findings should be tested using larger samples with both sexes. Second, PET scanning required a longer time compared with other imaging techniques such as fMRI. This limitation of temporal resolution prevented us from model-based testing of dynamic brain activation accompanying decision-making (O'Doherty et al., 2007). We selected PET imaging because simultaneous recording of EEG and autonomic indices inside an MRI scanner was technically difficult, if not impossible. In future works, such efforts are promising for direct tests of brain and autonomic correlates in decision-making. Additionally, using fMRI and EEG (Holmes and Pizzagalli, 2008), functional connectivity between the prefrontal and striatal regions which are involved both in decision-making and autonomic regulation should be examined in the future. Nevertheless, we can draw a conclusion that the neural network including the OFC, ACC, DLPFC, and striatum provides a neural basis for decision-making in an uncertain situation, and at the same time, the ACC modulates peripheral autonomic activity via vagal pathways on the basis of uncertainty of the situation. Acknowledgments This work was supported by a Grant-in-Aid for Scientific Research of the Japan Society for the Promotion of Science (No. 16330136). Portions of the present study were presented at the 10th Annual Meeting of the Organization for Human Brain Mapping (Budapest, Hungary, June 2004) and at the Annual Meeting of Society for Psychophysiological Research (Santa Fe, NM, October 2004). The 7th author (SF) equally contributed to this work with the 1st author (HO). The authors thank Dr. Motohiro Kimura (Nagoya University), Dr. Takashi Nakao (Nagoya University), and

Dr. Masahiro Matsunaga (Fujita Health University) for their comments on an earlier version of this paper. References Alexander, G.E., Crutcher, M.D., DeLong, M.R., 1990. Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, “prefrontal” and “limbic” functions. Prog. Brain Res. 85, 119–146. Beauchaine, T.P., Gatzke-Kopp, L., Mead, H.K., 2007. Polyvagal theory and developmental psychopathology: emotion dysregulation and conduct problems from preschool to adolescence. Biol. Psychol. 74, 174–184. Blackwood, N., Ffytche, D., Simmons, A., Bentall, R., Murray, R., Howard, R., 2004. The cerebellum and decision making under uncertainty. Brain Res. Cogn. Brain Res. 20, 46–53. Blascovich, J., Mendes, W.B., Hunter, S.B., Salomon, K., 1999. Social “facilitation” as challenge and threat. J. Pers. Soc. Psychol. 77, 68–77. Botvinick, M., Nystrom, L.E., Fissell, K., Carter, C.S., Cohen, J.D., 1999. Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402, 179–181. Brázdil, M., Roman, R., Falkenstein, M., Daniel, P., Jurák, P., Rektor, I., 2002. Error processing— evidence from intracerebral ERP recordings. Exp. Brain Res. 146, 460–466. Breiter, H.C., Aharon, I., Kahneman, D., Dale, A., Shizgal, P., 2001. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619–639. Bush, G., Whalen, P.J., Rosen, B.R., Jenike, M.A., McInerney, S.C., Rauch, S.L., 1998. The counting Stroop: an interference task specialized for functional neuroimaging– validation study with functional MRI. 1998. Hum. Brain Mapp. 6, 270–282. Carter, C.S., Braver, T.S., Barch, D.M., Botvinick, M.M., Noll, D., Cohen, J.D., 1998. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749. Carter, C.S., Macdonald, A.M., Botvinick, M., Ross, L.L., Stenger, V.A., Noll, D., Cohen, J.D., 2000. Parsing executive processes: strategic vs. evaluative functions of the anterior cingulate cortex. Proc. Natl. Acad. Sci. U. S. A. 97, 1944–1948. Carruthers, M., Taggart, P., 1973. Vagotonicity of violence: biochemical and cardiac responses to violent films and television programmes. Br. Med. J. 3, 384–389. Cohen, M.X., 2008. Neurocomputational mechanisms of reinforcement-guided learning in humans: a review. Cogn. Affect Behav. Neurosci. 8, 113–125. Critchley, H.D., Corfield, D.R., Chandler, M.P., Mathias, C.J., Dolan, R.J., 2000a. Cerebral correlates of autonomic cardiovascular arousal: a functional neuroimaging investigation in humans. J. Physiol. 523, 259–270. Critchley, H.D., Elliott, R., Mathias, C.J., Dolan, R.J., 2000b. Neural activity relating to generation and representation of galvanic skin conductance responses: a functional magnetic resonance imaging study. J. Neurosci. 20, 3033–3040. Critchley, H.D., Mathias, C.J., Josephs, O., O Doherty, J., Zanini, S., Dewar, B.K., Cipolotti, L., Shallice, T., Dolan, R.J., 2003. Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. Brain 126, 2139–2152. Critchley, H.D., Tang, J., Glaser, D., Butterworth, B., Dolan, R.J., 2005. Anterior cingulate activity during error and autonomic response. NeuroImage 27, 885–895. Delgado, M.R., Miller, M.M., Inati, S., Phelps, E.A., 2005. An fMRI study of reward-related probability learning. Neuroimage 24, 862–873. Delorme, A., Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of singletrial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21. Dywan, J., Mathewson, K.J., Choma, B.L., Rosenfeld, B., Segalowitz, S.J., 2008. Autonomic and electrophysiological correlates of emotional intensity in older and younger adults. Psychophysiology 45, 389–397. Elliott, R., Dolan, R.J., Frith, C.D., 2000. Dissociable functions in the medial and lateral orbitofrontal cortex: evidence from human neuroimaging studies. Cereb. Cortex 10, 308–317. Endo, N., Saiki, J., Saito, H., 2001. Determinants of occurrence of negative priming for novel shapes with matching paradigm. The Japanese Journal of Psychology 72, 204–212 [in Japanese]. Eppinger, B., Kray, J., Mock, B., Mecklinger, A., 2008. Better or worse than expected? Aging, learning, and the ERN. Neuropsychologia 46, 521–539. Fecteau, S., Knoch, D., Fregni, F., Sultani, N., Boggio, P., Pascual-Leone, A., 2007. Diminishing risk-taking behavior by modulating activity in the prefrontal cortex: a direct current stimulation study. J. Neurosci. 27, 12500–12505. Forman, S.D., Cohen, J.D., Fitzgerald, M., Eddy, W.F., Mintun, M.A., Noll, D.C., 1995. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of cluster-size threshold. Magn. Reson. Med. 33, 636–647. Gehring, W.J., Knight, R.T., 2000. Prefrontal-cingulate interactions in action monitoring. Nat. Neurosci. 3, 516–520. Gehring, W.J., Willoughby, A.R., 2002. The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295, 2279–2282. Gianaros, P.J., Derbyshire, S.W., May, J.C., Siegle, G.J., Gamalo, M.A., Jennings, J.R., 2005. Anterior cingulate activity correlates with blood pressure during stress. Psychophysiology 42, 627–635. Gianaros, P.J., Van Der Veen, F.M., Jennings, J.R., 2004. Regional cerebral blood flow correlates with heart period and high-frequency heart period variability during working-memory tasks: implications for the cortical and subcortical regulation of cardiac autonomic activity. Psychophysiology 41, 521–530. Groen, Y., Wijers, A.A., Mulder, L.J., Minderaa, R.B., Althaus, M., 2007. Physiological correlates of learning by performance feedback in children: a study of EEG eventrelated potentials and evoked heart rate. Biol. Psychol. 76, 174–187. Hajcak, G., McDonald, N., Simons, R.F., 2003. To err is autonomic: error-related brain potentials, ANS activity, and post-error compensatory behavior. Psychophysiology 40, 895–903.

H. Ohira et al. / NeuroImage 49 (2010) 1024–1037 Hajcak, G., McDonald, N., Simons, R.F., 2004. Error-related psychophysiology and negative affect. Brain Cogn. 56, 189–197. Hampton, A.H., O Doherty, J.P., 2007. Decoding the neural substrates of rewardrelated decision making with functional MRI. Proc. Natl. Acad. Sci. U. S. A. 104, 1377–1382. Haber, S.N., 2003. The primate basal ganglia: parallel and integrative networks. J. Chem. Neuroanat. 26, 317–330. Haruno, M., Kawato, M., 2006. Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus– action–reward association learning. Neural Netw. 19, 1242–1254. Herrmann, M.J., Römmler, J., Ehlis, A.C., Heidrich, A., Fallgatter, A.J., 2004. Source localization (LORETA) of the error-related-negativity (ERN/Ne) and positivity (Pe). Brain Res. Cogn. Brain Res. 20, 294–299. Holroyd, C.B., Coles, M.G., 2002. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709. Holmes, A.J., Pizzagalli, D.A., 2008. Spatiotemporal dynamics of error processing dysfunctions in major depressive disorder. Arch. Gen. Psychiatry 65, 179–188. Jennings, J.R., van der Molen, M.W., 2002. Cardiac timing and the central regulation of action. Psychol. Res. 66, 337–349. Keay, K.A., Bandler, R., 2001. Parallel circuits mediating distinct emotional coping reactions to different types of stress. Neurosci. Biobehav. Rev. 25, 669–678. Kimura, K., Ohira, H., Isowa, T., Matsunaga, M., Murashima, S., 2007. Regulation of lymphocytes redistribution via autonomic nervous activity during stochastic learning. Brain Behav. Immun. 21, 921–934. Kosson, D.S., Budhani, S., Nakic, M., Chen, G., Saad, Z.S., Vythilingam, M., Pine, D.S., Blair, R.J., 2006. The role of the amygdala and rostral anterior cingulate in encoding expected outcomes during learning. Neuroimage 29, 1161–1172. Krawczyk, D.C., 2002. Contributions of the prefrontal cortex to the neural basis of human decision making. Neurosci. Biobehav. Rev. 26, 631–664. Kringelbach, M.L., 2005. The human orbitofrontal cortex: linking reward to hedonic experience. Nat. Rev. Neurosci. 6, 691–702. Kringelbach, M.L., Rolls, E.T., 2004. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog. Neurobiol. 72, 341–372. Labudda, K., Woermann, F.G., Mertens, M., Pohlmann-Eden, B., Markowitsch, H.J., Brand, M., 2008. Neural correlates of decision making with explicit information about probabilities and incentives in elderly healthy subjects. Exp. Brain Res. 187, 641–650. Lane, R.D., McRae, K., Reiman, E.M., Chen, K., Ahern, G.L., Thayer, J.F., 2009. Neural correlates of heart rate variability during emotion. Neuroimage 44, 213–222. Lane, R.D., Reiman, E.M., Ahern, G.L., Thayer, J.F., 2001. Activity in medial prefrontal cortex correlates with vagal component of heart rate variability during emotion. Brain Cogn. 47, 97–100. Lee, D., Seo, H., 2007. Mechanisms of reinforcement learning and decision making in the primate dorsolateral prefrontal cortex. Ann. NY. Acad. Sci. 1104, 108–122. Matsunaga, M., Isowa, T., Kimura, K., Miyakoshi, M., Kanayama, N., Murakami, H., Fukuyama, S., Shinoda, J., Yamada, J., Konagaya, T., Kaneko, H., Ohira, H., 2009. Association among positive mood, brain, and cardiovascular activities in an affectively positive situation. Brain Res. 1263, 93–103. Matthews, S.C., Paulus, M.P., Simmons, A.N., Nelesen, R.A., Dimsdale, J.E., 2004. Functional subdivisions within anterior cingulate cortex and their relationship to autonomic nervous system function. Neuroimage 22, 1151–1156. McCabe, P., Schneiderman, N., 1985. Psychophysiological reactions to stress. In: Schneiderman, N., Tapp, J. (Eds.), Behavioral Medicine: The biopsychology Approach. Sage Pubns, pp. 99–131. McClure, S.M., Laibson, D.I., Loewenstein, G., Cohen, J.D., 2004. Separate neural systems value immediate and delayed monetary rewards. Science 306, 503–507. McClure, S.M., Ericson, K.M., Laibson, D.I., Loewenstein, G., Cohen, J.D., 2007. Time discounting for primary rewards. J. Neurosci. 27, 5796–5804. Montague, P.R., Hyman, S.E., Cohen, J.D., 2004. Computational roles for dopamine in behavioural control. Nature 431, 760–767. Napadow, V., Dhond, R., Conti, G., Makris, N., Brown, E.N., Barbieri, R., 2008. Brain correlates of autonomic modulation: combining heart rate variability with fMRI. Neuroimage 42, 169–177. Nieuwenhuis, S., Ridderinkhof, K.R., Blom, J., Band, G.P., Kok, A., 2001. Error-related brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task. Psychophysiology 38, 752–760. Nieuwenhuis, S., Ridderinkhof, K.R., Talsma, D., Coles, M.G., Holroyd, C.B., Kok, A., van der Molen, M.W., 2002. A computational account of altered error processing in older age: dopamine and the error-related negativity. Cogn. Affect. Behav. Neurosci. 2, 19–36. Nieuwenhuis, S., Holroyd, C.B., Mol, N., Coles, M.G., 2004. Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neurosci. Biobehav. Rev. 28, 441–448. O’Connor, M.F., Gündel, H., McRae, K., Lane, R.D., 2007. Baseline vagal tone predicts BOLD response during elicitation of grief. Neuropsychopharmacology 32, 2184–2189.

1037

O’Doherty, J., Critchley, H., Deichmann, R., Dolan, R.J., 2003. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931–7939. O'Doherty, J., Kringelbach, M.L., Rolls, E.T., Hornak, J., Andrews, C., 2001. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95–102. O'Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., Dolan, R.J., 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454. O'Doherty, J.P., Hampton, A., Kim, H., 2007. Model-based fMRI and its application to reward learning and decision making. Ann. N.Y. Acad. Sci. 1104, 35–53. Oliveira, F.T., McDonald, J.J., Goodman, D., 2007. Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J. Cogn. Neurosci. 19, 1994–2004. Ohira, H., Fukuyama, S., Kimura, K., Nomura, M., Isowa, T., Ichikawa, N., Matsunaga, M., Kimura, K., Miyakoshi, M., Shinoda, J., Yamada, J., 2009. Regulation of natural killer cell redistribution by prefrontal cortex during stochastic learning. Neuroimage 47 (3), 897–907. Ohira, H., Isowa, T., Nomura, M., Ichikawa, N., Kimura, K., Miyakoshi, M., Iidaka, T., Fukuyama, S., Nakajima, T., Yamada, J., 2008. Imaging brain and immune association accompanying cognitive appraisal of an acute stressor. Neuroimage 39, 500–514. Overbeek, T.J.M., Nieuwenhuis, S., Ridderinkhof, R., 2005. Dissociable components of error processing. J. Psychophysiol. 19, 319–329. Perini, R., Milesi, S., Fisher, N.M., Pendergast, D.R., Veicsteinas, A., 2000. Heart rate variability during dynamic exercise in elderly males and females. Eur. J. Appl. Physiol. 82, 8–15. Polezzi, D., Daum, I., Rubaltelli, E., Lotto, L., Civai, C., Sartori, G., Rumiati, R., 2008. Mentalizing in economic decision-making. Behav. Brain Res. 190, 218–223. Postuma, R.B., Dagher, A., 2006. Basal ganglia functional connectivity based on a metaanalysis of 126 positron emission tomography and functional magnetic resonance imaging publications. Cereb. Cortex 16, 1508–1521. Roberts, A.C., 2006. Primate orbitofrontal cortex and adaptive behaviour. Trends Cogn. Sci. 10, 83–90. Rogers, R.D., Owen, A.M., Middleton, H.C., Williams, E.J., Pickard, J.D., Sahakian, B.J., Robbins, T.W., 1999. Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex. J. Neurosci. 19, 9029–9038. Sailer, U., Robinson, S., Fischmeister, F.P., Moser, E., Kryspin-Exner, I., Bauer, H., 2007. Imaging the changing role of feedback during learning in decision-making. Neuroimage 37, 1474–1486. Saul, J.P., Rea, R.F., Eckberg, D.L., Berger, R.D., Cohen, R.J., 1990. Heart rate and muscle sympathetic nerve variability during reflex changes of autonomic activity. Am. J. Physiol. 258, H713–H721. Sayers, B.M., 1973. Analysis of heart rate variability. Ergonomics 16, 17–32. Schneiderman, N., McCabe, P.M., 1989. Psychophysiological strategies in laboratory research. In: Schneiderman, N., Weiss, S.M., Kaufmann, P.G. (Eds.), Handbook of Research Methods in Cardiovascular Behavioral Medicine. Plenum Press, New York, pp. 349–364. Schultz, W., 2002. Getting formal with dopamine and reward. Neuron 36, 241–263. Schultz, W., Dayan, P., Montague, P.R., 1997. A neural substrate of prediction and reward. Science 275, 1593–1599. Schweizer, T.A., Oriet, C., Meiran, N., Alexander, M.P., Cusimano, M., Stuss, D.T., 2007. The cerebellum mediates conflict resolution. J. Cogn. Neurosci. 19, 1974–1982. Tanaka, S.C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., Yamawaki, S., 2004. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887–893. Tanaka, S.C., Schweighofer, N., Asahi, S., Shishida, K., Okamoto, Y., Yamawaki, S., Doya, K., 2007. Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum. PLoS ONE 2, e1333. Thayer, J.F., Brosschot, J.F., 2005. Psychosomatics and psychopathology: looking up and down from the brain. Psychoneuroendocrinology 30, 1050–1058. Tobler, P.N., Fiorillo, C.D., Schultz, W., 2005. Adaptive coding of reward value by dopamine neurons. Science 307, 1642–1645. Tremblay, L., Schultz, W., 1999. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704–708. Tremblay, L., Schultz, W., 2000. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 83, 1877–1885. Turken, A.U., Swick, D., 2008. The effect of orbitofrontal lesions on the error-related negativity. Neurosci. Lett. 441, 7–10. Ullsperger, M., von Cramon, D.Y., 2006. The role of intact frontostriatal circuits in error processing. J. Cogn. Neurosci. 18, 651–664. van Veen, V., Holroyd, C.B., Cohen, J.D., Stenger, V.A., Carter, C.S., 2004. Errors without conflict: implications for performance monitoring theories of anterior cingulate cortex. Brain Cogn. 56, 267–276. Wise, R.A., 2002. Brain reward circuitry: insights from unsensed incentives. Neuron 36, 229–240. Wong, S.W., Massé, N., Kimmerly, D.S., Menon, R.S., Shoemaker, J.K., 2007. Ventral medial prefrontal cortex and cardiovagal control in conscious humans. Neuroimage 35, 698–708.