- Email: [email protected]
A parametric relief signal in human ventrolateral prefrontal cortex
NeuroImage 44 (2009) 1163–1170
Contents lists available at ScienceDirect
NeuroImage j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / y n i m g
A parametric relief signal in human ventrolateral prefrontal cortex Juri Fujiwara a, Philippe N. Tobler b, Masato Taira c,d, Toshio Iijima a, Ken-Ichiro Tsutsui a,⁎ a
Division of Systems Neuroscience, Tohoku University Graduate School of Life Sciences, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK c ARISH, Nihon University, Tokyo, Japan d Division of Applied System Neuroscience, Department of Advanced Medical Sciences, Nihon University School of Medicine, Tokyo, Japan b
a r t i c l e
i n f o
Article history: Received 2 May 2008 Revised 12 September 2008 Accepted 29 September 2008 Available online 18 October 2008 Keywords: Counterfactual reasoning Emotion fMRI Relative coding Reward Rejoice
a b s t r a c t People experience relief whenever outcomes are better than they would have been, had an alternative course of action been chosen. Here we investigated the neuronal basis of relief with functional resonance imaging in a choice task in which the outcome of the chosen option and that of the unchosen option were revealed sequentially. We found parametric activation increases in anterior ventrolateral prefrontal cortex with increasing relief (chosen outcomes better than unchosen outcomes). Conversely, anterior ventrolateral prefrontal activation was unrelated to the opposite of relief, increasing regret (chosen outcomes worse than unchosen outcomes). Furthermore, the anterior ventrolateral prefrontal activation was unrelated to primary gains and increased with relief irrespective of whether the chosen outcome was a loss or a gain. These results suggest that the anterior ventrolateral prefrontal cortex encodes a higher-order reward signal that lies at the core of current theories of emotion. © 2008 Elsevier Inc. All rights reserved.
Emotions are elicited by external events and result in behavioral adaptation (Rolls, 2000; Critchley, 2003). Primary emotions such as joy and disappointment arise from motivationally significant outcomes such as reward and punishment. However, we experience also higher-order emotions, such as relief and regret (“rejoice”, is sometimes used synonymously with “relief”, (e.g. Chandrasekhar et al., 2008) but we use “relief” here to avoid confusion of “joy” and “rejoice”). These emotions are not entirely determined by chosen outcomes but also take into consideration the outcomes of unchosen courses of action. For example, consider an investor who decided to invest 400 dollars in one of two companies and subsequently loses 100 dollars because of a fall in the share price. Despite the disappointment of her loss, the investor might still experience higher-order relief when she comes to know that the alternative company went bankrupt and she would have lost the entire 400 dollars, had she chosen to invest in that company. Thus, higher-order emotions occur in addition and independently of primary emotions such that primary and secondary emotions can have opposing valence. Current theories of emotion emphasize the counterfactual nature of higher-order emotions: they require reasoning about what would have happened had one chosen otherwise (Bell 1982; Loomes and Sugden 1982; Gilovich and Medvec 1995; Mellers and McGraw 2001; Byrne 2002; Zeelenberg and Pieters 2007). Formally, the sign and relative difference between the factually chosen and the counter⁎ Corresponding author. Fax: +81 22 217 5048. E-mail addresses: [email protected], [email protected] (K.-I. Tsutsui). 1053-8119/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2008.09.050
factually unchosen outcome serve to define quality and strength of higher-order emotions. If the unchosen outcome is worse than the chosen outcome, one experiences a relative gain accompanied by the higher-order emotion of relief of having done the right thing (Table 1). Conversely, if the unchosen outcome is better than the chosen outcome, one experiences a relative loss accompanied by the higherorder emotion of regret of having done the wrong thing. Importantly, the formal definition of secondary emotions such as relief in terms of relative value difference allows a precise quantification and separation of secondary from primary emotions. Thus, in the example above, the investor's disappointment (−100 dollars) is smaller than her relief (+300 dollars; i.e. chosen − unchosen = −100 − (−400) = +300). In the present study we used the formal definition of higher-order emotion to quantify relief in a parametric fashion. Lesions in orbitofrontal cortex impair regret processing (Camille et al., 2004). Functional magnetic resonance imaging (fMRI) has confirmed orbitofrontal involvement in regret processing and found regret-related signals also in the dorsolateral prefrontal cortex (Coricelli et al., 2005). Thus, these data suggest that prefrontal cortex fulfils a prime role in regret processing. However, it is largely unknown whether prefrontal cortex also plays a major role in relief processing and whether relief and regret are processed by the same structures. The counterfactual nature of relief requires a higher-order comparison of obtained with unobtained outcomes. The lateral prefrontal cortex, particularly its ventral part, is an important higher-order reward structure (Sakagami and Watanabe 2007). Lateral prefrontal neurons track which specific rewards are currently available in blocks of differentially rewarded trials (Watanabe 1996;
1164
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
Table 1 Primary and secondary emotions associated with positive and negative absolute and relative value
Absolute value (chosen outcome) Relative value (chosen − unchosen outcome)
Positive
Negative
Joy Relief
Disappointment Regret
Watanabe et al., 2002). Thus, lateral prefrontal cortex neurons code the motivational reward context of current actions. The computation of a quantitative relief signals requires that unchosen rewards serve as contexts to be integrated with chosen rewards. We therefore hypothesized that the lateral prefrontal cortex should process relief. In agreement with this hypothesis, we found that a region in anterior ventrolateral prefrontal cortex specifically encoded parametric relief. Materials and methods Participants Seventeen right-handed adults (12 males and 5 females, aged 20– 29 years) with no history of neurological, psychiatric, or auditory symptoms participated in the imaging study. Written informed consent was obtained from each participant. This experiment was conducted in accordance with the Declaration of Helsinki and was approved by the Ethical Committee of the Nihon University School of Medicine. Experimental design and task Prior to the experiment, participants were instructed to compete with a computer to maximize their income in a task in which, for each trial, they had to choose one of two decks of cards. Each deck contained the same cards, which were associated with 0 yen or a gain or loss of 200, 400, or 800 yen (100 yen correspond to about 1 US dollar). Both decks also contained blank control cards. Participants had to learn the properties of the decks by trial and error, and were not given any information about the distribution and size of outcomes of the two decks. Questioning of participants after the experiment revealed that all participants except one male believed that they were competing with a computer. Data obtained from the participant who did not believe that he was competing with a computer were excluded from analyses. The sequence of events in a trial is shown in Fig 1A. At the beginning of each trial, the two decks were presented until participants made their choice, but for a maximum of 4 s. If no response occurred during that interval, the computer chose one of the decks randomly, which happened very rarely (0–1 trials for each participant). Reported results correspond only to trials in which participants responded in time. Once a choice was made, the chosen deck was highlighted (1 s). After a variable delay of 3–4 s, the outcome of the chosen deck (chosen outcome) was presented for 2 s. After a second delay of 3–4 s, the outcome of the unchosen deck (unchosen outcome) was presented for 2 s, followed by an intertrial interval of 3–4 s. The absolute value of a trial corresponded to the chosen outcome, whereas the relative value was a function of both the chosen and the unchosen outcome, and could thus only be computed at the time of the unchosen outcome according to the formula: Relative value = chosen outcome
unchosen outcome:
For example a relative value of +200 yen would arise from +400 yen (chosen outcome) and +200 yen (unchosen outcome), or from 0 yen (chosen outcome) and −200 yen (unchosen outcome), or from −200 yen (chosen outcome) and −400 yen (unchosen outcome). Importantly, the absolute and relative values were completely counterbalanced experimentally. In other words, the probability of the unchosen outcome being lower, the same as, or higher than the chosen outcome was constant (p = 1/3). Absolute and relative values
both ranged within −800 and +800 yen with a mean of 0 yen. Absolute values were 0, +/−200, +/−400, or +/−800, and relative values were 0, +/−200, +/−400, +/−600, or +/−800. The trial sequence was predetermined for each participant, and different sequence were used across participants. Each participant started the task with 3000 yen, and was told that any losses they incurred during the experiment would be subtracted from this total, whereas any gains would be added to the total. Thus, participants knew that their eventual payments at the end of the experiment would consist of a base amount of 3000 plus or minus a variable amount earned during the experiment. The actual payoff of the variable amount earned and lost during the experiment was zero in all participants. This was ensured by pre-determining the sequence of chosen and unchosen outcomes such that the actual mean payoff corresponded to the theoretical mean payoff of 0 yen after every 12 to 14 trials. However, for ethical reasons, each participant received 5000 yen (about 50 US dollars), according to the duration of their participation, approximately 3 months after scanning. Participants were not informed of this before the experiment. The experiment comprised a training phase (20 trials), a scanning phase (122 trials), and a behavioral testing phase (122 trials). Earnings of the training phase did not contribute to the total earnings of participants. In the behavioral testing phase, participants were required to indicate how happy or unhappy the chosen and unchosen outcomes of each trial made them on a scale from 1 (very happy) to 9 (very unhappy). In order to avoid cumulative wealth effects (Kahneman and Tversky 1979), accumulated earnings were not displayed and both absolute and relative gains and losses were counterbalanced such that the cumulative wealth of all participants was 0 yen after every 12 to 14 trials. Participants reported that they did not try to add up their earnings during the experiment. Image acquisition All images were acquired using a 1.5-T Siemens Symphony MRI scanner (Siemens, Erlangen, Germany) at Nihon University. Gradientrecalled echo planar imaging (EPI) was used for the fMRI sequence to obtain blood oxygen level-dependent (BOLD) contrast. A total of 976 functional scans per participant were acquired using a gradient echo EPI sequence (20 axial slices in the AC–PC plane; repetition time (TR) = 2000 ms, echo time (TE) = 50 ms, flip angle = 90°, field of view (FOV) = 192 mm, 64 × 64 matrix, voxels size= 3 × 3 × 5 mm, slice thickness= 4 mm, gap= 1 mm). A T1 anatomical scan of each participant was obtained (192 sagittal slices; TR= 2000 ms, TE = 3.93 ms, flip angle= 15°, FOV = 256× 224 mm, in-plane resolution = 1 × 1 mm, slice thickness= 1 mm). fMRI analysis Preprocessing and data analysis were performed using the SPM2 software. The first two functional scans were discarded to allow for magnetic saturation. The individual slices of a functional volume were temporally corrected for their acquisition time difference, with reference to the middle (tenth) slice. The functional images of each participant were realigned with reference to the first image to correct for head motion. The anatomical images were coregistered with the mean functional images and normalized to the MNI brain template. Functional data were then normalized using the same transformation parameters and smoothed in the spatial domain (isotropic Gaussian kernel of 8 mm at full width-half maximum). Functional data were analyzed in an event-related design. In general linear models we modeled each condition by the canonical hemodynamic response function and its temporal derivative. Participant-specific movement parameters were modeled as covariates of no interest. A high pass filter with a cutoff period of 128 s was used to remove the low-frequency noise. Global scaling was not applied.
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
1165
Fig. 1. Experimental design and behavior. (A) Behavioral task. In each trial, two decks were presented on the left and the right side of a monitor and participants chose one of the two decks as identified by different letters. Each deck contained the same cards, which were associated with 0 yen or a gain or loss of 200, 400, or 800 yen (100 yen correspond to about 1 US dollar). Participants' choice was highlighted for 1 s and after a variable delay the outcome of the chosen deck was presented for 2 s. After a delay of 3–4 s the unchosen outcome was presented for 2 s, followed by an intertrial interval of 3–4 s. The likelihood of the unchosen outcome being higher, the same or lower than the chosen outcome remained constant and trial types were randomly interleaved. (B) Average happiness rating in all participants as a function of the absolute value of the chosen outcome (error bars represent standard deviation). For each level of absolute gain or loss, ratings were pooled over different levels of relative gain or loss. The scale ranged from 1 (happy) to 9 (unhappy). Ratings were taken at the end of each trial. (C) Average happiness rating as a function of relative value (chosen outcome − unchosen outcome). For each relative value, ratings were pooled over different levels of absolute value. (D) Absence of relation between reaction time in current trial and absolute value in previous trial. (E) Faster reaction times in current trial with increasing relative value in previous trial (r = − 0.71, P = 0.03). For (C) and (D) reaction times were measured from time of deck presentation.
General linear models served to compute trial type-specific betas, reflecting the strength of covariance between the brain activation and the canonical response function for a given condition at each voxel for each participant (see Friston et al., 1995 for detailed descriptions). In a first general linear model we defined 6 event types of interest, negative, zero and positive absolute value (onset at chosen outcome) and relative value (onset at unchosen outcome; value determined as difference of chosen-unchosen outcome). This model pooled over different levels of positive or negative value, and was used to replicate basic regret-related (relative loss-specific) activations in orbital and dorsolateral prefrontal cortex, anterior cingulate and putamen (Coricelli et al., 2005). Spheres with a radius of 10 mm around the previously reported peak activations in these structures served as regions of interest for the contrast: 1 ⁎ relative loss − 0.5 ⁎ (relative gain + relative zero). This model also served to identify basic relief-related (relative gain-specific) activations with the contrast: 1 ⁎ relative gain − 0.5 ⁎ (relative loss + relative zero). The present study asked whether prefrontal cortex would not only show regret- but also relief-related activations and whether the two would overlap. We therefore
restricted analysis in the first model to the frontal lobe. Specifically, we controlled for multiple comparisons by using the frontal lobe as an anatomically defined mask as implemented by the Pickatlas toolbox (Maldijan et al., 2003). Briefly, the frontal lobe region of interest encompassed all grey matter anterior of the central sulcus, including precentral, superior, middle and inferior frontal gyri and sulci, anterior poles, cingulate and orbitofrontal cortex. In a second model we further investigated regions identified by the first model. We defined two event types of interest, absolute outcomes (onset at chosen outcome) and relative outcomes (onset at unchosen outcome). These events were defined irrespective of whether participants experienced gains or losses in a given trial. We then used parametric modulation of these two events of interest. The parametric modulators assumed the actual absolute and relative values experienced in any particular trial and spanned the full range of gains and losses. Thus, the parametric modulators allowed us to search for linear activation changes across the range of values used (relevant contrast = +1 on the parametric modulator, corresponding to a monotonic increase in relief reflecting the amount of relative gain).
1166
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
In a third model, we further interrogated relative gain (relief) regions identified with the first two models. We defined two trial types of interest (onset at unchosen outcome), relative gain trials preceded by an absolute gain and relative gain trials preceded by an absolute loss. We then tested whether there was a difference in activation between these two cases. A genuinely relief coding region should show similar and non-differential activation for relative gains incurred after absolute gains and after absolute losses. The effects of interest (betas, percent of signal change) were calculated relative to an implicit baseline. Using random-effects analysis, the relevant contrasts of parameter estimates were entered into a series of one-way t-tests and simple ANOVAs with nonsphericity correction where appropriate. For each contrast, statistical parametric maps of the t-statistic were generated on a voxel-byvoxel basis, and these t-values were transformed into z-scores of the standard normal distribution. We used activations surviving correction for multiple comparisons in the frontal lobe (P b 0.05, false discovery rate (FDR)) in the first model as regions of interest for the second model. Thus for the second model, we report activations surviving small volume correction for multiple comparisons in the voxels identified by the first model (P b 0.05, FDR). The third model tested for a significant main effect of relief trials both after an absolute gain and an absolute loss in the region identified by the first two models and small volume correction was applied. The dependent measure in scatter plots is percentage signal change measured at peak voxels. Reported voxels conform to MNI (Montreal Neurological Institute) coordinate space, with the right side of the image corresponding to the right side of the brain. Results We used functional magnetic resonance imaging (fMRI) to investigate the neuronal correlates of absolute and relative value processing in a counterbalanced design that fully dissociated absolute and relative reward value. Different card decks were associated with an equal probability of gains and losses. In each trial (expected value = 0 yen), participants chose between one of two card decks. Subsequently, the outcome of the chosen and then the unchosen deck were presented, which informed participants about the absolute and relative value of their choice, respectively. Absolute and relative values ranged from +800 to −800 yen (800 yen correspond to about 8 US dollars; Fig. 1A). Positive and negative relative values (accompanied by the higher-order emotions of relief and regret, respectively) occurred with equal probability, for each experienced absolute value except −800 and 800 yen. Trials in which the absolute value was either −800 or 800 yen were excluded from the analysis of relative value. The task was designed such that the experienced gains and losses from the chosen option (absolute gains and losses) were the same in relief (relative gain) and regret (relative loss) trials. Behavioral performance We measured how participants felt after each trial. Happiness ratings increased both as a function of absolute value (pooling over different levels of relative value) and as a function of relative value (pooling over different levels of absolute value) (Figs. 1B and C; r (absolute) = 0.98, P b 0.0001; r (relative) = 0.96, P b 0.0001). Correlations for absolute and relative value did not differ significantly from each other (P = 0.32, z-test). There was no evidence to suggest that the experience of absolute or relative losses or gains in the previous trial influenced which option individual participants chose on the current trial (all P N = 0.093). To test whether participants made random choices, we performed a runs test on the sequence of each participant's choices of the two options. Out of 16 participants, 8 showed significant deviations from randomness in choosing the two options (P b 0.05), without showing simple left/right biases (chi-
square for proportion of choosing left or right: all P N 0.1). Of the remaining 8 participants, 1 showed a left-right bias (P b 0.001, chisquare) and 3 used win-stay/loose-shift or loose-stay/win-shift strategies (P b 0.05, chi-square test). These strategies were not detected by the runs test because they depend on the outcome in the previous trial, while the runs test only considers patterns in choice sequence, irrespective of outcomes. Thus, participants expressed increasing happiness with increasing joy and relief and the majority of them made non-random choices. We measured choice reaction time in each trial and regressed it to the absolute and relative value participants experienced in the previous trial. Reaction times were shorter in the current trial with increasing relative value experienced in the previous trial, but not with increasing absolute value in the previous trial (Figs. 1D and E; r (absolute) = 0.03, P = 0.96; r (relative) = −0.71, P = 0.03). Taken together, these data suggest that participants processed both absolute and relative value. Relief of having experienced a better outcome than what would have ensued from choosing the alternative option in the previous trial accelerated responses in the next trial, whereas simple joy of having won in the previous trial did not. Neuroimaging We used three separate general linear models to analyze brain activation at the time when the chosen and the unchosen outcome were presented to investigate absolute and relative value, respectively. The first model comprised six separate regressors for positive, negative and zero absolute and relative values, thus pooling over different levels of relative and absolute value. This model allowed us to replicate basic regret coding regions and to identify potential regions involved in relief. In the second model we used the full range of gains and losses as parametric absolute and relative value modulators of chosen and unchosen outcomes. We used this second model to search for regions showing more fine-grained and linear relief-specific activation changes within the basic relief-related regions identified with the first model. Finally we interrogated the anterior ventrolateral prefrontal region identified by the first two models in a third model that separated relief trials in those preceded by absolute gains and those preceded by absolute losses. This last step served to ensure that relief coding occurs both after primary joy and disappointment, respectively. Basic replication of regret findings (relative loss) In order to investigate whether our task produced comparable activation in trials that elicit regret, we constructed regions of interest around previously reported peak activations in orbitofrontal cortex, anterior cingulate, dorsolateral prefrontal cortex and putamen (Coricelli et al., 2005). For these spheres of interest we compared the activation in trials in which participants experienced a relative loss against activation in trials in which they experienced a relative gain and a relative value of zero and found significantly stronger activations for relative loss in all regions (P b 0.05, small volume correction for multiple comparisons in regions of interest; Fig. 2; Table 2). These data suggest that our task was successful in eliciting activations that reflect a relative loss in the comparison of chosen and unchosen outcome, which is the defining component of the counterfactual emotion of regret. Basic relief areas (relative gain) Next, we tested for regions that showed a basic relation to relief in the lateral prefrontal cortex by testing for stronger activation in relative gain compared to relative zero and relative loss trials (P b 0.05, full frontal lobe volume correction with False Discovery Rate). Fig. 3 shows the pattern of activation in a right anterior ventrolateral prefrontal region (x/y/z coordinates: 40/42/12; anterior
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
1167
Fig. 2. Region of interest analysis in previously reported regret coding areas. Peak activations from Coricelli et al., (2005) served as origins for 10 mm spheres of interest in the comparison of relative loss versus relative gain and relative zero (Table 2). ACC, anterior cingulate cortex; DLPFC, dorsolateral prefrontal cortex; OFC, orbitofrontal cortex.
inferior frontal gyrus (Brodmann area 46); incidentally, this region was the only one also with a cluster size larger than expected by chance when correcting at the whole brain level, P = 0.01). Outside prefrontal cortex, there were no significant activations at P b 0.05, whole braincorrected. Both mean effects and time course plots indicate that the identified ventrolateral prefrontal region shows activation increases specifically when the difference between chosen and unchosen outcome is positive, but not when it is zero or negative. Thus, the effects result from activations in relief trials rather than from deactivation in regret or zero trials. Finally, exclusive masking showed that the anterior ventrolateral prefrontal region was preferentially activated by relative rather than absolute gains (see next section for a direct comparison). More dorsal prefrontal cortex regions showed also a specific basic relief signal (e.g. −34/40/26; P b 0.05, frontal lobe correction; Table 3). Conversely, activations in the cingulate cortex showed common relations to both relative and absolute gains (Table 3). These data suggest that ventral and dorsal prefrontal regions satisfy basic requirements of relief encoding.
Fig. 3. Basic relief coding regions. (A) Location in anterior ventrolateral prefrontal cortex showing stronger activation in positive relative gain compared to zero and negative relative gain trials (40/40/8). (B) Average effect size in anterior ventrolateral prefrontal region shown in (A). Activation in trials with positive relative value was significantly stronger than in other trials (P b 0.05, small volume correction). Error bars correspond to 90% confidence intervals. (C) Average time courses for conditions shown in (B). Time courses were obtained by fitting stimulus-induced activity with a finite impulse response set (time = 0 corresponds to onset of unchosen outcome). Activations show the typical delay of the hemodynamic response with a latency to peak of 4–6 seconds.
In a second general linear model we tested for a more quantitative relief signal. The basic relief-related activations identified by the first model served as regions of interest for this analysis. We used a parametric modulator that increased with the difference between chosen and unchosen outcome and found that the anterior ventrolateral prefrontal region displayed in Fig. 3 was the only region surviving small volume correction for multiple comparisons (40/40/8; P b 0.05; z = 3.7). Activation in this region increased primarily across the range of positive relative values and was unchanged with regretrelated negative relative values (Fig. 4A). We investigated this notion further by separately contrasting parametric relief with parametric regret and parametric relief with relative value of zero. We found an overlapping and significantly stronger relation to relief in both
comparisons, in a similar region as the one shown in Fig. 3A (peak: 38/32/8). The expression of parametric relief in anterior ventrolateral prefrontal cortex activation remained significant even when reaction time was included into the general linear model as a covariate of no interest (peak: 40/38/0). These data suggest that the anterior ventrolateral prefrontal cortex processes a quantitative, parametric relief signal. Next we examined whether participants' choice patterns would result in differential relief-related activation in anterior ventrolateral prefrontal cortex. We separately tested for parametric relief signals in participants showing random choice behavior and in those showing non-random choice behavior as determined by the runs test (both groups n = 8). A repeated measures ANOVA with level of relief as within-subject and group as between-subject factors revealed a significant main effect for relief, as reported earlier in the result section (P b 0.001). There was no significant interaction (P = 0.42) between group and relief level. Thus, we have no evidence to assume a difference in anterior ventrolateral prefrontal cortex activation depending on the specific choice strategy used by participants.
Table 2 Coordinates of previously reported regret coding regions (Coricelli et al., 2005) and replication in current study within 10 mm spheres around peaks (P b 0.05, small volume correction. with false discovery rate)
Table 3 Coordinates of frontal regions showing a basic relief signal as detected with the first model (P b 0.05, corrected for frontal cortex, clusters with more than 10 voxels)
Parametric relief signal in anterior ventrolateral prefrontal cortex
Anterior cingulate cortex Putamen Lateral orbitofrontal cortex Medial orbitofrontal cortex Dorsolateral prefrontal cortex Inferior parietal lobule
R L R R L L R R R
x
y
z
Replication
10 −14 42 42 −8 −10 46 54 54
24 0 42 26 32 30 28 −50 −58
34 6 − 18 −16 −14 −12 38 36 48
Yes Yes No Yes Yes Yes Yes Yes No
Ventrolateral prefrontal cortex
Dorsal prefrontal cortex Anterior cingulate cortex Supplementary motor cortex
R L L L L R R
x
y
z
Voxels
z-score
40 −44 − 50 − 34 − 14 2 6
40 24 12 40 42 44 −16
8 14 2 26 12 4 52
193 17 52 12 33 87 53
3.51 3.35 3.46 3.41 3.73 3.46 3.84
Specific Specific Specific Specific Common Common Common
The last column indicates whether regions showed preferential relations to relief rather than joy (“specific”) or common coding of both relief and joy (“common”).
1168
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
Fig. 4. Activation of anterior ventrolateral prefrontal cortex as a function of full range of relative and absolute reward value. (A) Activation increase of region shown in Fig. 3A with increasing relief but not with increasing regret. Basic activations obtained in simple comparison of positive relative value versus other trials (Fig. 3) were further tested by requiring parametric modulation of activation with the level of relief (P b 0.05, small volume correction). (B) No change in activation as function of absolute value of chosen outcome in region shown in Fig. 3A. (C) Stronger parametric modulation of anterior ventrolateral prefrontal activation by relief than joy. Levels of relief for parametric modulation corresponded to difference between chosen and unchosen outcome; levels of joy were determined as levels of chosen outcome (P b 0.05, small volume correction). Error bars correspond to 90% confidence intervals.
The relief coding activation in anterior ventrolateral region changed only to a limited degree with the absolute value of the chosen outcome (Fig. 4B). In a direct comparison we found stronger parametric modulation for positive relative than positive absolute value (P b 0.05, small volume correction; Fig. 4C). There was no overlap in anterior ventrolateral prefrontal cortex between activations reflecting absolute gains and those reflecting relative gains. Thus, activation in the anterior ventrolateral prefrontal cortex is less related to the joy elicited by gains from the chosen outcome than to relief proper, elicited by counterfactual gains from the unchosen outcome. Anterior ventrolateral prefrontal relief-related activation both after gains and losses It is an important feature of relief that it occurs irrespective of whether people actually experienced joy or a disappointment from their chosen outcome. In a third general linear model we therefore modeled relief trials that followed an absolute gain separately from relief trials that followed an absolute loss. For this analysis, the anterior ventrolateral prefrontal region identified by the first two models served as region of interest. In accordance with the prediction, activation in this region increased with positive relative value irrespective of whether participants earned or lost money on an absolute scale (Fig. 5). These data suggest that relief-related anterior
Fig. 5. Anterior ventrolateral prefrontal activation irrespective of positive or negative absolute value of chosen outcome. For this model, relative value trials were separated according to whether participants experienced an actual gain or loss in that trial. Reliefrelated activation in the region shown in Fig. 3A occurred both in gain and loss trials (P b 0.05, small volume correction) and there was no significant difference between relief-related activation in absolute gain and loss trials (P N 0.05, uncorrected).
ventrolateral prefrontal activation occurs independently of positive or negative primary emotion experience. Discussion Relief is a higher-order emotion that occurs when the counterfactual comparison between a chosen and unchosen outcome is beneficial. The current paper shows that the neuronal basis of relief is specific both anatomically and functionally: A region in anterior ventrolateral prefrontal cortex codes the level of relief as quantified by the positive difference between chosen and unchosen outcomes. Activations in this region are not influenced by the absolute value of the chosen outcome (first outcome) and relief-related activation occurs irrespective of whether participants' choice results in an absolute gain or loss. Behavioral relevance of relief Participants were not formally required to process the unchosen outcome. Nevertheless they felt similarly happy with joy-eliciting positive absolute values as with relief-eliciting positive relative values. Moreover, increasing levels of relief in the previous trial accelerated reaction time in the next trial whereas increasing absolute gains in the previous trial did not. These behavioral data suggest that the kind of counterfactual comparisons proposed by theories of higher emotion (Table 1; Gilovich and Medvec 1995; Mellers and McGraw 2001; Byrne 2002; Zeelenberg and Pieters 2007) and by behavioral economic theories (Bell 1982; Loomes and Sugden 1982) occur relatively automatically but influence people's behavior considerably. Even though participants could not learn anything specific about the two options over and above the general reward probabilities (see below), the behavioral effect of the unchosen option was relatively strong. This finding lends indirect support to the notion that the effects of anticipated higher-order emotions may be sufficient to explain exceptions to standard microeconomic theory (Bell 1982; Loomes and Sugden 1982). In the present task, the probability of experiencing relief or regret in each trial was fixed at p = 1/3. However, participants had no explicit knowledge of these probabilities or of the probability with which each outcome would occur. One may ask whether the notion of relief still holds in the absence of explicit predictability. We propose it does for several reasons. First, outcome probabilities are usually unknown in natural decisions outside the lab (and casino), but relief is not restricted to such artificial situations. In natural situations, just like in
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
the present experiment, people use experience to form a subjective impression of how probable it is to win or lose. Accordingly, the formal theories of regret and relief hold both with subjective and objective probabilities (e.g. Loomes and Sugden 1982; Mellers and McGraw 2001). Furthermore, the behavioral finding discussed above that participants paid considerable amounts of attention to the unchosen outcome suggests that they were not only experiencing primary joy and disappointment from chosen outcomes but also higher-order emotions. Participants indeed adjusted their behavior as a function of the relative value of the preceding trial (Fig. 1E), believed they actively competed with the computer to obtain reward, showed substantial deviations from random choice and employed outcome-dependent strategies. Finally, the differential activations for relative and absolute reward value suggest that two outcomes were not processed similarly. We should note tough that the present data, analyzed at the time of outcomes, primarily concern experienced rather than predicted relief. We studied relief in a choice situation as traditionally proposed by experimental psychology (Mellers and McGraw 2001). Participants employed different choice strategies but the relief-related brain activations were similar irrespective of the strategy employed. Future research may therefore want to investigate under what conditions behavioral strategies influence relief signals and whether reliefrelated activation of anterior ventrolateral prefrontal cortex also occurs in situations not involving choice. Dissociation of absolute and relative value An important feature of the current study is that it fully disentangles joy related to absolute and relief related to relative reward value. For example, in each trial participants were equally likely to experience relief or regret, irrespective of whether they won or lost money in that trial. In agreement with the requirement of a true dissociation between absolute and relative value coding, we observed stronger parametric coding of relative gain (relief) than absolute gain in anterior ventrolateral prefrontal cortex, but similar relief-related activation after both absolute gains and losses. These findings concur with the notion that relief should occur irrespective of whether the chosen outcome gives reason for joy or disappointment (Mellers and McGraw 2001). Early theories of economic value assumed value to be represented on an absolute scale (Bernoulli 1954). Conversely, more recent theories such as prospect theory (Kahneman and Tversky 1979) emphasize the relative nature of value. Specifically, prospect theory suggests that value is encoded relative to reference points that could for example correspond to a person's current wealth or to the stakes of a gamble. The present study defined relative value by the difference of the chosen and unchosen outcome. In accordance with prospect theory we propose that the unchosen outcome provided a reference relative to which the chosen outcome was dynamically reevaluated in the form of a quantitative difference signal. The data suggest that positive outcomes of this reevaluation (relief) are coded specifically by anterior ventrolateral prefrontal cortex and that there is some regional specificity in terms of absolute and relative value coding in the brain. Distinct value signals in prefrontal cortex The present study suggests that anterior ventrolateral prefrontal cortex preferentially codes positive relative reward value or relief, rather than positive absolute reward value or negative relative value. Conversely, previous studies have shown separate prefrontal activation increases with absolute reward value e.g. in medial orbital and medioventral prefrontal cortex (Knutson et al., 2001; O'Doherty et al., 2001; Ramnani et al., 2004) and in more posterior regions of lateral prefrontal cortex (Tobler et al., 2007). Thus, distinct regions of prefrontal cortex represent reward value in absolute and in relative terms.
1169
The current study used monetary rewards with constant probabilities to elicit relief. Chandrasekhar et al. (2008) recently employed foot shocks with varying probabilities to study relief and regret. They found a number of relief-related structures throughout the brain, including prefrontal cortex, parietal cortex, insula, and striatum. The different design and the use of motivationally more salient foot shocks may explain the more widespread relief-related activations in Chandrasekhar et al. (2008) compared to the current paper. Interestingly however, one of the regions described in that study (Chandrasekhar et al., 2008) was in close vicinity, just dorsal of the anterior ventrolateral prefrontal region described in the present study. The present results add to those of Chandrasekhar et al. (2008) by showing that relief coding in ventrolateral prefrontal cortex occurs irrespective of whether the chosen outcome was negative, positive or neutral. Taken together, these data suggest that relief processing in the lateral prefrontal cortex is modality-independent. Ventrolateral prefrontal cortex has been proposed to detect the absence of unpleasant events (Bender et al., 2007). Experimental psychology has long suggested that the absence or end of a negative event may be rewarding (e.g. Konorski, 1967; Solomon and Corbit, 1974; conversely the absence or end of a positive event may be punishing). The present results concur with and extend this view in that they show graded, quantitative relief coding and preferential relations to relief rather than joy in anterior ventrolateral prefrontal cortex. Relief-related activation was parametrically graded and occurred also when participants experienced joy due to absolute gains. Thus, this region of prefrontal cortex appears to be a higherorder reward processing region, which is activated not only when unpleasant events fail to occur. This notion concurs with neurophysiological studies in the non-human primate, which have found specific and context-dependent reward functions in lateral prefrontal cortex (e.g. Watanabe 1996; Watanabe et al., 2002; Kobayashi et al., 2006). Our findings add to previous reports of relative reward value coding (e.g. Breiter et al., 2001; Kuhnen and Knutson 2005; Lohrenz et al., 2007). The present study points to a role of right anterior ventrolateral prefrontal cortex in processing a parametric relief signal. We did not have any hypothesis about laterality and found more basic relief signals also in the left hemisphere (for other examples of bilateral relative value coding, see Kuhnen and Knutson, 2005; Lohrenz et al., 2007; Chandrasekhar et al., 2008). Previous lesion and imaging studies point to a role of orbital, cingulate and dorsolateral prefrontal cortex in coding of negative relative reward value or regret (Camille et al., 2004; Coricelli et al., 2005). Taken together, negative and positive relative value appear to be coded in distinct prefrontal regions and the prefrontal cortex thus seems to make distinct contributions to the processing of higher emotions such as regret and relief. Acknowledgments This research was supported by Grant-in-Aid for Scientific Research (KAKENHI) from the MEXT (#17680027, #19673002), Grant-in-Aid for Scientific Research on Priority Areas-System study on higher-order brain functions, from the MEXT (#17022009, #18020005, #20019005), and Academic Frontier Project for Private Universities: ‘Brain Mechanisms for Cognition, Memory and Behavior’ at Nihon University: a matching fund subsidy from the MEXT. J.F. was supported by Global COE Program ‘Basic and Translational Research Center for Global Brain Science’ at Tohoku University from the MEXT. References Bell, D.E., 1982. Regret in decision making under uncertainty. Oper. Res. 30, 961–981. Bender, S., Hellwig, S., Resch, F., Weisbrod, M., 2007. Am I safe? The ventrolateral prefrontal cortex ‘detects’ when an unpleasant event does not occur. NeuroImage 38, 367–385.
1170
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
Bernoulli, D., 1954. Exposition of a new theory on the measurement of risk (translated from Latin). Econometrica 22, 23–36. Breiter, H.C., Aharon, I., Kahneman, D., Dale, A., Shizgal, P., 2001. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619–639. Byrne, R.M., 2002. Mental models and counterfactual thoughts about what might have been. Trends Cogn. Sci. 6, 426–431. Camille, N., Coricelli, G., Sallet, J., Pradat-Diehl, P., Duhamel, J.R., Sirigu, A., 2004. The involvement of the orbitofrontal cortex in the experience of regret. Science 304, 1167–1170. Chandrasekhar, P.V., Capra, C.M., Moore, S., Noussair, C., Berns, G.S., 2008. Neurobiological regret and rejoice functions for aversive outcomes. NeuroImage 39, 1472–1484. Coricelli, G., Critchley, H.D., Joffily, M., O'Doherty, J.P., Sirigu, A., Dolan, R.J., 2005. Regret and its avoidance: a neuroimaging study of choice behavior. Nat. Neurosci. 8, 1255–1262. Critchley, H., 2003. Emotion and its disorders. Br. Med. Bull. 65, 35–47. Friston, K.J., Holmes, A.P., Worsley, K.J., Poline, J.P., Frith, C.D., Frackowiak, R.S.J., 1995. Statistical parametric maps in functional imaging: a general linear approach. Hum. Brain Mapp. 2, 189–210. Gilovich, T., Medvec, V.H., 1995. The experience of regret: what, when, and why. Psychol. Rev. 102, 379–395. Kahneman, D., Tversky, A., 1979. Prospect theory: an analysis of decision under risk. Econometrica 47, 263–291. Knutson, B., Fong, G.W., Adams, C.M., Varner, J.L., Hommer, D., 2001. Dissociation of reward anticipation and outcome with event-related fMRI. NeuroReport 12, 3683–3687. Konorski, J., 1967. Integrative Action of the Brain. University of Chicago Press, Chicago. Kobayashi, S., Nomoto, K., Watanabe, M., Hikosaka, O., Schultz, W., Sakagami, M., 2006. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51, 861–870. Kuhnen, C.M., Knutson, B., 2005. The neural basis of financial risk taking. Neuron 47, 763–770.
Lohrenz, T., McCabe, K., Camerer, C.F., Montague, P.R., 2007. Neural signature of fictive learning signals in a sequential investment task. Proc. Natl. Acad. Sci. U. S. A. 104, 9493–9498. Loomes, G., Sugden, R., 1982. Regret theory: an alternative theory of rational choice under uncertainty. Econ. J. 92, 805–824. Maldijan, J.A., Laurienti, P.J., Kraft, R.A., Burdette, J.H., 2003. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. NeuroImage 19, 1233–1239. Mellers, B.A., McGraw, A.P., 2001. Anticipated emotions as guides to choice. Curr. Dir. Psychol. Sci. 10, 210–214. O'Doherty, J., Kringelbach, M.L., Rolls, E.T., Hornak, J., Andrews, C., 2001. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95–102. Ramnani, N., Elliott, R., Athwal, B.S., Passingham, R.E., 2004. Prediction error for free monetary reward in the human prefrontal cortex. NeuroImage 23, 777–786. Rolls, E.T., 2000. Précis of the brain and emotion. Behav. Brain Sci. 23, 177–191. Sakagami, M., Watanabe, M., 2007. Integration of cognitive and motivational information in the primate lateral prefrontal cortex. Ann. N.Y. Acad. Sci. 1104, 89–107. Solomon, R.L., Corbit, J.D., 1974. An opponent-process theory of motivation: I. Temporal dynamics of affect. Psychol. Rev. 81, 119–145. Tobler, P.N., O'Doherty, J.P., Dolan, R.J., Schultz, W., 2007. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 97, 1621–1632. Watanabe, M., 1996. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632. Watanabe, M., Hikosaka, K., Sakagami, M., Shirakawa, S., 2002. Coding and monitoring of motivational context in the primate prefrontal cortex. J. Neurosci. 22, 2391–2400. Zeelenberg, M., Pieters, R., 2007. A theory of regret regulation 1.0. J. Consum. Psychol. 17, 3–18.
Contents lists available at ScienceDirect
NeuroImage j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / y n i m g
A parametric relief signal in human ventrolateral prefrontal cortex Juri Fujiwara a, Philippe N. Tobler b, Masato Taira c,d, Toshio Iijima a, Ken-Ichiro Tsutsui a,⁎ a
Division of Systems Neuroscience, Tohoku University Graduate School of Life Sciences, 2-1-1 Katahira, Aoba-ku, Sendai 980-8577, Japan Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge, UK c ARISH, Nihon University, Tokyo, Japan d Division of Applied System Neuroscience, Department of Advanced Medical Sciences, Nihon University School of Medicine, Tokyo, Japan b
a r t i c l e
i n f o
Article history: Received 2 May 2008 Revised 12 September 2008 Accepted 29 September 2008 Available online 18 October 2008 Keywords: Counterfactual reasoning Emotion fMRI Relative coding Reward Rejoice
a b s t r a c t People experience relief whenever outcomes are better than they would have been, had an alternative course of action been chosen. Here we investigated the neuronal basis of relief with functional resonance imaging in a choice task in which the outcome of the chosen option and that of the unchosen option were revealed sequentially. We found parametric activation increases in anterior ventrolateral prefrontal cortex with increasing relief (chosen outcomes better than unchosen outcomes). Conversely, anterior ventrolateral prefrontal activation was unrelated to the opposite of relief, increasing regret (chosen outcomes worse than unchosen outcomes). Furthermore, the anterior ventrolateral prefrontal activation was unrelated to primary gains and increased with relief irrespective of whether the chosen outcome was a loss or a gain. These results suggest that the anterior ventrolateral prefrontal cortex encodes a higher-order reward signal that lies at the core of current theories of emotion. © 2008 Elsevier Inc. All rights reserved.
Emotions are elicited by external events and result in behavioral adaptation (Rolls, 2000; Critchley, 2003). Primary emotions such as joy and disappointment arise from motivationally significant outcomes such as reward and punishment. However, we experience also higher-order emotions, such as relief and regret (“rejoice”, is sometimes used synonymously with “relief”, (e.g. Chandrasekhar et al., 2008) but we use “relief” here to avoid confusion of “joy” and “rejoice”). These emotions are not entirely determined by chosen outcomes but also take into consideration the outcomes of unchosen courses of action. For example, consider an investor who decided to invest 400 dollars in one of two companies and subsequently loses 100 dollars because of a fall in the share price. Despite the disappointment of her loss, the investor might still experience higher-order relief when she comes to know that the alternative company went bankrupt and she would have lost the entire 400 dollars, had she chosen to invest in that company. Thus, higher-order emotions occur in addition and independently of primary emotions such that primary and secondary emotions can have opposing valence. Current theories of emotion emphasize the counterfactual nature of higher-order emotions: they require reasoning about what would have happened had one chosen otherwise (Bell 1982; Loomes and Sugden 1982; Gilovich and Medvec 1995; Mellers and McGraw 2001; Byrne 2002; Zeelenberg and Pieters 2007). Formally, the sign and relative difference between the factually chosen and the counter⁎ Corresponding author. Fax: +81 22 217 5048. E-mail addresses: [email protected], [email protected] (K.-I. Tsutsui). 1053-8119/$ – see front matter © 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2008.09.050
factually unchosen outcome serve to define quality and strength of higher-order emotions. If the unchosen outcome is worse than the chosen outcome, one experiences a relative gain accompanied by the higher-order emotion of relief of having done the right thing (Table 1). Conversely, if the unchosen outcome is better than the chosen outcome, one experiences a relative loss accompanied by the higherorder emotion of regret of having done the wrong thing. Importantly, the formal definition of secondary emotions such as relief in terms of relative value difference allows a precise quantification and separation of secondary from primary emotions. Thus, in the example above, the investor's disappointment (−100 dollars) is smaller than her relief (+300 dollars; i.e. chosen − unchosen = −100 − (−400) = +300). In the present study we used the formal definition of higher-order emotion to quantify relief in a parametric fashion. Lesions in orbitofrontal cortex impair regret processing (Camille et al., 2004). Functional magnetic resonance imaging (fMRI) has confirmed orbitofrontal involvement in regret processing and found regret-related signals also in the dorsolateral prefrontal cortex (Coricelli et al., 2005). Thus, these data suggest that prefrontal cortex fulfils a prime role in regret processing. However, it is largely unknown whether prefrontal cortex also plays a major role in relief processing and whether relief and regret are processed by the same structures. The counterfactual nature of relief requires a higher-order comparison of obtained with unobtained outcomes. The lateral prefrontal cortex, particularly its ventral part, is an important higher-order reward structure (Sakagami and Watanabe 2007). Lateral prefrontal neurons track which specific rewards are currently available in blocks of differentially rewarded trials (Watanabe 1996;
1164
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
Table 1 Primary and secondary emotions associated with positive and negative absolute and relative value
Absolute value (chosen outcome) Relative value (chosen − unchosen outcome)
Positive
Negative
Joy Relief
Disappointment Regret
Watanabe et al., 2002). Thus, lateral prefrontal cortex neurons code the motivational reward context of current actions. The computation of a quantitative relief signals requires that unchosen rewards serve as contexts to be integrated with chosen rewards. We therefore hypothesized that the lateral prefrontal cortex should process relief. In agreement with this hypothesis, we found that a region in anterior ventrolateral prefrontal cortex specifically encoded parametric relief. Materials and methods Participants Seventeen right-handed adults (12 males and 5 females, aged 20– 29 years) with no history of neurological, psychiatric, or auditory symptoms participated in the imaging study. Written informed consent was obtained from each participant. This experiment was conducted in accordance with the Declaration of Helsinki and was approved by the Ethical Committee of the Nihon University School of Medicine. Experimental design and task Prior to the experiment, participants were instructed to compete with a computer to maximize their income in a task in which, for each trial, they had to choose one of two decks of cards. Each deck contained the same cards, which were associated with 0 yen or a gain or loss of 200, 400, or 800 yen (100 yen correspond to about 1 US dollar). Both decks also contained blank control cards. Participants had to learn the properties of the decks by trial and error, and were not given any information about the distribution and size of outcomes of the two decks. Questioning of participants after the experiment revealed that all participants except one male believed that they were competing with a computer. Data obtained from the participant who did not believe that he was competing with a computer were excluded from analyses. The sequence of events in a trial is shown in Fig 1A. At the beginning of each trial, the two decks were presented until participants made their choice, but for a maximum of 4 s. If no response occurred during that interval, the computer chose one of the decks randomly, which happened very rarely (0–1 trials for each participant). Reported results correspond only to trials in which participants responded in time. Once a choice was made, the chosen deck was highlighted (1 s). After a variable delay of 3–4 s, the outcome of the chosen deck (chosen outcome) was presented for 2 s. After a second delay of 3–4 s, the outcome of the unchosen deck (unchosen outcome) was presented for 2 s, followed by an intertrial interval of 3–4 s. The absolute value of a trial corresponded to the chosen outcome, whereas the relative value was a function of both the chosen and the unchosen outcome, and could thus only be computed at the time of the unchosen outcome according to the formula: Relative value = chosen outcome
unchosen outcome:
For example a relative value of +200 yen would arise from +400 yen (chosen outcome) and +200 yen (unchosen outcome), or from 0 yen (chosen outcome) and −200 yen (unchosen outcome), or from −200 yen (chosen outcome) and −400 yen (unchosen outcome). Importantly, the absolute and relative values were completely counterbalanced experimentally. In other words, the probability of the unchosen outcome being lower, the same as, or higher than the chosen outcome was constant (p = 1/3). Absolute and relative values
both ranged within −800 and +800 yen with a mean of 0 yen. Absolute values were 0, +/−200, +/−400, or +/−800, and relative values were 0, +/−200, +/−400, +/−600, or +/−800. The trial sequence was predetermined for each participant, and different sequence were used across participants. Each participant started the task with 3000 yen, and was told that any losses they incurred during the experiment would be subtracted from this total, whereas any gains would be added to the total. Thus, participants knew that their eventual payments at the end of the experiment would consist of a base amount of 3000 plus or minus a variable amount earned during the experiment. The actual payoff of the variable amount earned and lost during the experiment was zero in all participants. This was ensured by pre-determining the sequence of chosen and unchosen outcomes such that the actual mean payoff corresponded to the theoretical mean payoff of 0 yen after every 12 to 14 trials. However, for ethical reasons, each participant received 5000 yen (about 50 US dollars), according to the duration of their participation, approximately 3 months after scanning. Participants were not informed of this before the experiment. The experiment comprised a training phase (20 trials), a scanning phase (122 trials), and a behavioral testing phase (122 trials). Earnings of the training phase did not contribute to the total earnings of participants. In the behavioral testing phase, participants were required to indicate how happy or unhappy the chosen and unchosen outcomes of each trial made them on a scale from 1 (very happy) to 9 (very unhappy). In order to avoid cumulative wealth effects (Kahneman and Tversky 1979), accumulated earnings were not displayed and both absolute and relative gains and losses were counterbalanced such that the cumulative wealth of all participants was 0 yen after every 12 to 14 trials. Participants reported that they did not try to add up their earnings during the experiment. Image acquisition All images were acquired using a 1.5-T Siemens Symphony MRI scanner (Siemens, Erlangen, Germany) at Nihon University. Gradientrecalled echo planar imaging (EPI) was used for the fMRI sequence to obtain blood oxygen level-dependent (BOLD) contrast. A total of 976 functional scans per participant were acquired using a gradient echo EPI sequence (20 axial slices in the AC–PC plane; repetition time (TR) = 2000 ms, echo time (TE) = 50 ms, flip angle = 90°, field of view (FOV) = 192 mm, 64 × 64 matrix, voxels size= 3 × 3 × 5 mm, slice thickness= 4 mm, gap= 1 mm). A T1 anatomical scan of each participant was obtained (192 sagittal slices; TR= 2000 ms, TE = 3.93 ms, flip angle= 15°, FOV = 256× 224 mm, in-plane resolution = 1 × 1 mm, slice thickness= 1 mm). fMRI analysis Preprocessing and data analysis were performed using the SPM2 software. The first two functional scans were discarded to allow for magnetic saturation. The individual slices of a functional volume were temporally corrected for their acquisition time difference, with reference to the middle (tenth) slice. The functional images of each participant were realigned with reference to the first image to correct for head motion. The anatomical images were coregistered with the mean functional images and normalized to the MNI brain template. Functional data were then normalized using the same transformation parameters and smoothed in the spatial domain (isotropic Gaussian kernel of 8 mm at full width-half maximum). Functional data were analyzed in an event-related design. In general linear models we modeled each condition by the canonical hemodynamic response function and its temporal derivative. Participant-specific movement parameters were modeled as covariates of no interest. A high pass filter with a cutoff period of 128 s was used to remove the low-frequency noise. Global scaling was not applied.
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
1165
Fig. 1. Experimental design and behavior. (A) Behavioral task. In each trial, two decks were presented on the left and the right side of a monitor and participants chose one of the two decks as identified by different letters. Each deck contained the same cards, which were associated with 0 yen or a gain or loss of 200, 400, or 800 yen (100 yen correspond to about 1 US dollar). Participants' choice was highlighted for 1 s and after a variable delay the outcome of the chosen deck was presented for 2 s. After a delay of 3–4 s the unchosen outcome was presented for 2 s, followed by an intertrial interval of 3–4 s. The likelihood of the unchosen outcome being higher, the same or lower than the chosen outcome remained constant and trial types were randomly interleaved. (B) Average happiness rating in all participants as a function of the absolute value of the chosen outcome (error bars represent standard deviation). For each level of absolute gain or loss, ratings were pooled over different levels of relative gain or loss. The scale ranged from 1 (happy) to 9 (unhappy). Ratings were taken at the end of each trial. (C) Average happiness rating as a function of relative value (chosen outcome − unchosen outcome). For each relative value, ratings were pooled over different levels of absolute value. (D) Absence of relation between reaction time in current trial and absolute value in previous trial. (E) Faster reaction times in current trial with increasing relative value in previous trial (r = − 0.71, P = 0.03). For (C) and (D) reaction times were measured from time of deck presentation.
General linear models served to compute trial type-specific betas, reflecting the strength of covariance between the brain activation and the canonical response function for a given condition at each voxel for each participant (see Friston et al., 1995 for detailed descriptions). In a first general linear model we defined 6 event types of interest, negative, zero and positive absolute value (onset at chosen outcome) and relative value (onset at unchosen outcome; value determined as difference of chosen-unchosen outcome). This model pooled over different levels of positive or negative value, and was used to replicate basic regret-related (relative loss-specific) activations in orbital and dorsolateral prefrontal cortex, anterior cingulate and putamen (Coricelli et al., 2005). Spheres with a radius of 10 mm around the previously reported peak activations in these structures served as regions of interest for the contrast: 1 ⁎ relative loss − 0.5 ⁎ (relative gain + relative zero). This model also served to identify basic relief-related (relative gain-specific) activations with the contrast: 1 ⁎ relative gain − 0.5 ⁎ (relative loss + relative zero). The present study asked whether prefrontal cortex would not only show regret- but also relief-related activations and whether the two would overlap. We therefore
restricted analysis in the first model to the frontal lobe. Specifically, we controlled for multiple comparisons by using the frontal lobe as an anatomically defined mask as implemented by the Pickatlas toolbox (Maldijan et al., 2003). Briefly, the frontal lobe region of interest encompassed all grey matter anterior of the central sulcus, including precentral, superior, middle and inferior frontal gyri and sulci, anterior poles, cingulate and orbitofrontal cortex. In a second model we further investigated regions identified by the first model. We defined two event types of interest, absolute outcomes (onset at chosen outcome) and relative outcomes (onset at unchosen outcome). These events were defined irrespective of whether participants experienced gains or losses in a given trial. We then used parametric modulation of these two events of interest. The parametric modulators assumed the actual absolute and relative values experienced in any particular trial and spanned the full range of gains and losses. Thus, the parametric modulators allowed us to search for linear activation changes across the range of values used (relevant contrast = +1 on the parametric modulator, corresponding to a monotonic increase in relief reflecting the amount of relative gain).
1166
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
In a third model, we further interrogated relative gain (relief) regions identified with the first two models. We defined two trial types of interest (onset at unchosen outcome), relative gain trials preceded by an absolute gain and relative gain trials preceded by an absolute loss. We then tested whether there was a difference in activation between these two cases. A genuinely relief coding region should show similar and non-differential activation for relative gains incurred after absolute gains and after absolute losses. The effects of interest (betas, percent of signal change) were calculated relative to an implicit baseline. Using random-effects analysis, the relevant contrasts of parameter estimates were entered into a series of one-way t-tests and simple ANOVAs with nonsphericity correction where appropriate. For each contrast, statistical parametric maps of the t-statistic were generated on a voxel-byvoxel basis, and these t-values were transformed into z-scores of the standard normal distribution. We used activations surviving correction for multiple comparisons in the frontal lobe (P b 0.05, false discovery rate (FDR)) in the first model as regions of interest for the second model. Thus for the second model, we report activations surviving small volume correction for multiple comparisons in the voxels identified by the first model (P b 0.05, FDR). The third model tested for a significant main effect of relief trials both after an absolute gain and an absolute loss in the region identified by the first two models and small volume correction was applied. The dependent measure in scatter plots is percentage signal change measured at peak voxels. Reported voxels conform to MNI (Montreal Neurological Institute) coordinate space, with the right side of the image corresponding to the right side of the brain. Results We used functional magnetic resonance imaging (fMRI) to investigate the neuronal correlates of absolute and relative value processing in a counterbalanced design that fully dissociated absolute and relative reward value. Different card decks were associated with an equal probability of gains and losses. In each trial (expected value = 0 yen), participants chose between one of two card decks. Subsequently, the outcome of the chosen and then the unchosen deck were presented, which informed participants about the absolute and relative value of their choice, respectively. Absolute and relative values ranged from +800 to −800 yen (800 yen correspond to about 8 US dollars; Fig. 1A). Positive and negative relative values (accompanied by the higher-order emotions of relief and regret, respectively) occurred with equal probability, for each experienced absolute value except −800 and 800 yen. Trials in which the absolute value was either −800 or 800 yen were excluded from the analysis of relative value. The task was designed such that the experienced gains and losses from the chosen option (absolute gains and losses) were the same in relief (relative gain) and regret (relative loss) trials. Behavioral performance We measured how participants felt after each trial. Happiness ratings increased both as a function of absolute value (pooling over different levels of relative value) and as a function of relative value (pooling over different levels of absolute value) (Figs. 1B and C; r (absolute) = 0.98, P b 0.0001; r (relative) = 0.96, P b 0.0001). Correlations for absolute and relative value did not differ significantly from each other (P = 0.32, z-test). There was no evidence to suggest that the experience of absolute or relative losses or gains in the previous trial influenced which option individual participants chose on the current trial (all P N = 0.093). To test whether participants made random choices, we performed a runs test on the sequence of each participant's choices of the two options. Out of 16 participants, 8 showed significant deviations from randomness in choosing the two options (P b 0.05), without showing simple left/right biases (chi-
square for proportion of choosing left or right: all P N 0.1). Of the remaining 8 participants, 1 showed a left-right bias (P b 0.001, chisquare) and 3 used win-stay/loose-shift or loose-stay/win-shift strategies (P b 0.05, chi-square test). These strategies were not detected by the runs test because they depend on the outcome in the previous trial, while the runs test only considers patterns in choice sequence, irrespective of outcomes. Thus, participants expressed increasing happiness with increasing joy and relief and the majority of them made non-random choices. We measured choice reaction time in each trial and regressed it to the absolute and relative value participants experienced in the previous trial. Reaction times were shorter in the current trial with increasing relative value experienced in the previous trial, but not with increasing absolute value in the previous trial (Figs. 1D and E; r (absolute) = 0.03, P = 0.96; r (relative) = −0.71, P = 0.03). Taken together, these data suggest that participants processed both absolute and relative value. Relief of having experienced a better outcome than what would have ensued from choosing the alternative option in the previous trial accelerated responses in the next trial, whereas simple joy of having won in the previous trial did not. Neuroimaging We used three separate general linear models to analyze brain activation at the time when the chosen and the unchosen outcome were presented to investigate absolute and relative value, respectively. The first model comprised six separate regressors for positive, negative and zero absolute and relative values, thus pooling over different levels of relative and absolute value. This model allowed us to replicate basic regret coding regions and to identify potential regions involved in relief. In the second model we used the full range of gains and losses as parametric absolute and relative value modulators of chosen and unchosen outcomes. We used this second model to search for regions showing more fine-grained and linear relief-specific activation changes within the basic relief-related regions identified with the first model. Finally we interrogated the anterior ventrolateral prefrontal region identified by the first two models in a third model that separated relief trials in those preceded by absolute gains and those preceded by absolute losses. This last step served to ensure that relief coding occurs both after primary joy and disappointment, respectively. Basic replication of regret findings (relative loss) In order to investigate whether our task produced comparable activation in trials that elicit regret, we constructed regions of interest around previously reported peak activations in orbitofrontal cortex, anterior cingulate, dorsolateral prefrontal cortex and putamen (Coricelli et al., 2005). For these spheres of interest we compared the activation in trials in which participants experienced a relative loss against activation in trials in which they experienced a relative gain and a relative value of zero and found significantly stronger activations for relative loss in all regions (P b 0.05, small volume correction for multiple comparisons in regions of interest; Fig. 2; Table 2). These data suggest that our task was successful in eliciting activations that reflect a relative loss in the comparison of chosen and unchosen outcome, which is the defining component of the counterfactual emotion of regret. Basic relief areas (relative gain) Next, we tested for regions that showed a basic relation to relief in the lateral prefrontal cortex by testing for stronger activation in relative gain compared to relative zero and relative loss trials (P b 0.05, full frontal lobe volume correction with False Discovery Rate). Fig. 3 shows the pattern of activation in a right anterior ventrolateral prefrontal region (x/y/z coordinates: 40/42/12; anterior
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
1167
Fig. 2. Region of interest analysis in previously reported regret coding areas. Peak activations from Coricelli et al., (2005) served as origins for 10 mm spheres of interest in the comparison of relative loss versus relative gain and relative zero (Table 2). ACC, anterior cingulate cortex; DLPFC, dorsolateral prefrontal cortex; OFC, orbitofrontal cortex.
inferior frontal gyrus (Brodmann area 46); incidentally, this region was the only one also with a cluster size larger than expected by chance when correcting at the whole brain level, P = 0.01). Outside prefrontal cortex, there were no significant activations at P b 0.05, whole braincorrected. Both mean effects and time course plots indicate that the identified ventrolateral prefrontal region shows activation increases specifically when the difference between chosen and unchosen outcome is positive, but not when it is zero or negative. Thus, the effects result from activations in relief trials rather than from deactivation in regret or zero trials. Finally, exclusive masking showed that the anterior ventrolateral prefrontal region was preferentially activated by relative rather than absolute gains (see next section for a direct comparison). More dorsal prefrontal cortex regions showed also a specific basic relief signal (e.g. −34/40/26; P b 0.05, frontal lobe correction; Table 3). Conversely, activations in the cingulate cortex showed common relations to both relative and absolute gains (Table 3). These data suggest that ventral and dorsal prefrontal regions satisfy basic requirements of relief encoding.
Fig. 3. Basic relief coding regions. (A) Location in anterior ventrolateral prefrontal cortex showing stronger activation in positive relative gain compared to zero and negative relative gain trials (40/40/8). (B) Average effect size in anterior ventrolateral prefrontal region shown in (A). Activation in trials with positive relative value was significantly stronger than in other trials (P b 0.05, small volume correction). Error bars correspond to 90% confidence intervals. (C) Average time courses for conditions shown in (B). Time courses were obtained by fitting stimulus-induced activity with a finite impulse response set (time = 0 corresponds to onset of unchosen outcome). Activations show the typical delay of the hemodynamic response with a latency to peak of 4–6 seconds.
In a second general linear model we tested for a more quantitative relief signal. The basic relief-related activations identified by the first model served as regions of interest for this analysis. We used a parametric modulator that increased with the difference between chosen and unchosen outcome and found that the anterior ventrolateral prefrontal region displayed in Fig. 3 was the only region surviving small volume correction for multiple comparisons (40/40/8; P b 0.05; z = 3.7). Activation in this region increased primarily across the range of positive relative values and was unchanged with regretrelated negative relative values (Fig. 4A). We investigated this notion further by separately contrasting parametric relief with parametric regret and parametric relief with relative value of zero. We found an overlapping and significantly stronger relation to relief in both
comparisons, in a similar region as the one shown in Fig. 3A (peak: 38/32/8). The expression of parametric relief in anterior ventrolateral prefrontal cortex activation remained significant even when reaction time was included into the general linear model as a covariate of no interest (peak: 40/38/0). These data suggest that the anterior ventrolateral prefrontal cortex processes a quantitative, parametric relief signal. Next we examined whether participants' choice patterns would result in differential relief-related activation in anterior ventrolateral prefrontal cortex. We separately tested for parametric relief signals in participants showing random choice behavior and in those showing non-random choice behavior as determined by the runs test (both groups n = 8). A repeated measures ANOVA with level of relief as within-subject and group as between-subject factors revealed a significant main effect for relief, as reported earlier in the result section (P b 0.001). There was no significant interaction (P = 0.42) between group and relief level. Thus, we have no evidence to assume a difference in anterior ventrolateral prefrontal cortex activation depending on the specific choice strategy used by participants.
Table 2 Coordinates of previously reported regret coding regions (Coricelli et al., 2005) and replication in current study within 10 mm spheres around peaks (P b 0.05, small volume correction. with false discovery rate)
Table 3 Coordinates of frontal regions showing a basic relief signal as detected with the first model (P b 0.05, corrected for frontal cortex, clusters with more than 10 voxels)
Parametric relief signal in anterior ventrolateral prefrontal cortex
Anterior cingulate cortex Putamen Lateral orbitofrontal cortex Medial orbitofrontal cortex Dorsolateral prefrontal cortex Inferior parietal lobule
R L R R L L R R R
x
y
z
Replication
10 −14 42 42 −8 −10 46 54 54
24 0 42 26 32 30 28 −50 −58
34 6 − 18 −16 −14 −12 38 36 48
Yes Yes No Yes Yes Yes Yes Yes No
Ventrolateral prefrontal cortex
Dorsal prefrontal cortex Anterior cingulate cortex Supplementary motor cortex
R L L L L R R
x
y
z
Voxels
z-score
40 −44 − 50 − 34 − 14 2 6
40 24 12 40 42 44 −16
8 14 2 26 12 4 52
193 17 52 12 33 87 53
3.51 3.35 3.46 3.41 3.73 3.46 3.84
Specific Specific Specific Specific Common Common Common
The last column indicates whether regions showed preferential relations to relief rather than joy (“specific”) or common coding of both relief and joy (“common”).
1168
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
Fig. 4. Activation of anterior ventrolateral prefrontal cortex as a function of full range of relative and absolute reward value. (A) Activation increase of region shown in Fig. 3A with increasing relief but not with increasing regret. Basic activations obtained in simple comparison of positive relative value versus other trials (Fig. 3) were further tested by requiring parametric modulation of activation with the level of relief (P b 0.05, small volume correction). (B) No change in activation as function of absolute value of chosen outcome in region shown in Fig. 3A. (C) Stronger parametric modulation of anterior ventrolateral prefrontal activation by relief than joy. Levels of relief for parametric modulation corresponded to difference between chosen and unchosen outcome; levels of joy were determined as levels of chosen outcome (P b 0.05, small volume correction). Error bars correspond to 90% confidence intervals.
The relief coding activation in anterior ventrolateral region changed only to a limited degree with the absolute value of the chosen outcome (Fig. 4B). In a direct comparison we found stronger parametric modulation for positive relative than positive absolute value (P b 0.05, small volume correction; Fig. 4C). There was no overlap in anterior ventrolateral prefrontal cortex between activations reflecting absolute gains and those reflecting relative gains. Thus, activation in the anterior ventrolateral prefrontal cortex is less related to the joy elicited by gains from the chosen outcome than to relief proper, elicited by counterfactual gains from the unchosen outcome. Anterior ventrolateral prefrontal relief-related activation both after gains and losses It is an important feature of relief that it occurs irrespective of whether people actually experienced joy or a disappointment from their chosen outcome. In a third general linear model we therefore modeled relief trials that followed an absolute gain separately from relief trials that followed an absolute loss. For this analysis, the anterior ventrolateral prefrontal region identified by the first two models served as region of interest. In accordance with the prediction, activation in this region increased with positive relative value irrespective of whether participants earned or lost money on an absolute scale (Fig. 5). These data suggest that relief-related anterior
Fig. 5. Anterior ventrolateral prefrontal activation irrespective of positive or negative absolute value of chosen outcome. For this model, relative value trials were separated according to whether participants experienced an actual gain or loss in that trial. Reliefrelated activation in the region shown in Fig. 3A occurred both in gain and loss trials (P b 0.05, small volume correction) and there was no significant difference between relief-related activation in absolute gain and loss trials (P N 0.05, uncorrected).
ventrolateral prefrontal activation occurs independently of positive or negative primary emotion experience. Discussion Relief is a higher-order emotion that occurs when the counterfactual comparison between a chosen and unchosen outcome is beneficial. The current paper shows that the neuronal basis of relief is specific both anatomically and functionally: A region in anterior ventrolateral prefrontal cortex codes the level of relief as quantified by the positive difference between chosen and unchosen outcomes. Activations in this region are not influenced by the absolute value of the chosen outcome (first outcome) and relief-related activation occurs irrespective of whether participants' choice results in an absolute gain or loss. Behavioral relevance of relief Participants were not formally required to process the unchosen outcome. Nevertheless they felt similarly happy with joy-eliciting positive absolute values as with relief-eliciting positive relative values. Moreover, increasing levels of relief in the previous trial accelerated reaction time in the next trial whereas increasing absolute gains in the previous trial did not. These behavioral data suggest that the kind of counterfactual comparisons proposed by theories of higher emotion (Table 1; Gilovich and Medvec 1995; Mellers and McGraw 2001; Byrne 2002; Zeelenberg and Pieters 2007) and by behavioral economic theories (Bell 1982; Loomes and Sugden 1982) occur relatively automatically but influence people's behavior considerably. Even though participants could not learn anything specific about the two options over and above the general reward probabilities (see below), the behavioral effect of the unchosen option was relatively strong. This finding lends indirect support to the notion that the effects of anticipated higher-order emotions may be sufficient to explain exceptions to standard microeconomic theory (Bell 1982; Loomes and Sugden 1982). In the present task, the probability of experiencing relief or regret in each trial was fixed at p = 1/3. However, participants had no explicit knowledge of these probabilities or of the probability with which each outcome would occur. One may ask whether the notion of relief still holds in the absence of explicit predictability. We propose it does for several reasons. First, outcome probabilities are usually unknown in natural decisions outside the lab (and casino), but relief is not restricted to such artificial situations. In natural situations, just like in
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
the present experiment, people use experience to form a subjective impression of how probable it is to win or lose. Accordingly, the formal theories of regret and relief hold both with subjective and objective probabilities (e.g. Loomes and Sugden 1982; Mellers and McGraw 2001). Furthermore, the behavioral finding discussed above that participants paid considerable amounts of attention to the unchosen outcome suggests that they were not only experiencing primary joy and disappointment from chosen outcomes but also higher-order emotions. Participants indeed adjusted their behavior as a function of the relative value of the preceding trial (Fig. 1E), believed they actively competed with the computer to obtain reward, showed substantial deviations from random choice and employed outcome-dependent strategies. Finally, the differential activations for relative and absolute reward value suggest that two outcomes were not processed similarly. We should note tough that the present data, analyzed at the time of outcomes, primarily concern experienced rather than predicted relief. We studied relief in a choice situation as traditionally proposed by experimental psychology (Mellers and McGraw 2001). Participants employed different choice strategies but the relief-related brain activations were similar irrespective of the strategy employed. Future research may therefore want to investigate under what conditions behavioral strategies influence relief signals and whether reliefrelated activation of anterior ventrolateral prefrontal cortex also occurs in situations not involving choice. Dissociation of absolute and relative value An important feature of the current study is that it fully disentangles joy related to absolute and relief related to relative reward value. For example, in each trial participants were equally likely to experience relief or regret, irrespective of whether they won or lost money in that trial. In agreement with the requirement of a true dissociation between absolute and relative value coding, we observed stronger parametric coding of relative gain (relief) than absolute gain in anterior ventrolateral prefrontal cortex, but similar relief-related activation after both absolute gains and losses. These findings concur with the notion that relief should occur irrespective of whether the chosen outcome gives reason for joy or disappointment (Mellers and McGraw 2001). Early theories of economic value assumed value to be represented on an absolute scale (Bernoulli 1954). Conversely, more recent theories such as prospect theory (Kahneman and Tversky 1979) emphasize the relative nature of value. Specifically, prospect theory suggests that value is encoded relative to reference points that could for example correspond to a person's current wealth or to the stakes of a gamble. The present study defined relative value by the difference of the chosen and unchosen outcome. In accordance with prospect theory we propose that the unchosen outcome provided a reference relative to which the chosen outcome was dynamically reevaluated in the form of a quantitative difference signal. The data suggest that positive outcomes of this reevaluation (relief) are coded specifically by anterior ventrolateral prefrontal cortex and that there is some regional specificity in terms of absolute and relative value coding in the brain. Distinct value signals in prefrontal cortex The present study suggests that anterior ventrolateral prefrontal cortex preferentially codes positive relative reward value or relief, rather than positive absolute reward value or negative relative value. Conversely, previous studies have shown separate prefrontal activation increases with absolute reward value e.g. in medial orbital and medioventral prefrontal cortex (Knutson et al., 2001; O'Doherty et al., 2001; Ramnani et al., 2004) and in more posterior regions of lateral prefrontal cortex (Tobler et al., 2007). Thus, distinct regions of prefrontal cortex represent reward value in absolute and in relative terms.
1169
The current study used monetary rewards with constant probabilities to elicit relief. Chandrasekhar et al. (2008) recently employed foot shocks with varying probabilities to study relief and regret. They found a number of relief-related structures throughout the brain, including prefrontal cortex, parietal cortex, insula, and striatum. The different design and the use of motivationally more salient foot shocks may explain the more widespread relief-related activations in Chandrasekhar et al. (2008) compared to the current paper. Interestingly however, one of the regions described in that study (Chandrasekhar et al., 2008) was in close vicinity, just dorsal of the anterior ventrolateral prefrontal region described in the present study. The present results add to those of Chandrasekhar et al. (2008) by showing that relief coding in ventrolateral prefrontal cortex occurs irrespective of whether the chosen outcome was negative, positive or neutral. Taken together, these data suggest that relief processing in the lateral prefrontal cortex is modality-independent. Ventrolateral prefrontal cortex has been proposed to detect the absence of unpleasant events (Bender et al., 2007). Experimental psychology has long suggested that the absence or end of a negative event may be rewarding (e.g. Konorski, 1967; Solomon and Corbit, 1974; conversely the absence or end of a positive event may be punishing). The present results concur with and extend this view in that they show graded, quantitative relief coding and preferential relations to relief rather than joy in anterior ventrolateral prefrontal cortex. Relief-related activation was parametrically graded and occurred also when participants experienced joy due to absolute gains. Thus, this region of prefrontal cortex appears to be a higherorder reward processing region, which is activated not only when unpleasant events fail to occur. This notion concurs with neurophysiological studies in the non-human primate, which have found specific and context-dependent reward functions in lateral prefrontal cortex (e.g. Watanabe 1996; Watanabe et al., 2002; Kobayashi et al., 2006). Our findings add to previous reports of relative reward value coding (e.g. Breiter et al., 2001; Kuhnen and Knutson 2005; Lohrenz et al., 2007). The present study points to a role of right anterior ventrolateral prefrontal cortex in processing a parametric relief signal. We did not have any hypothesis about laterality and found more basic relief signals also in the left hemisphere (for other examples of bilateral relative value coding, see Kuhnen and Knutson, 2005; Lohrenz et al., 2007; Chandrasekhar et al., 2008). Previous lesion and imaging studies point to a role of orbital, cingulate and dorsolateral prefrontal cortex in coding of negative relative reward value or regret (Camille et al., 2004; Coricelli et al., 2005). Taken together, negative and positive relative value appear to be coded in distinct prefrontal regions and the prefrontal cortex thus seems to make distinct contributions to the processing of higher emotions such as regret and relief. Acknowledgments This research was supported by Grant-in-Aid for Scientific Research (KAKENHI) from the MEXT (#17680027, #19673002), Grant-in-Aid for Scientific Research on Priority Areas-System study on higher-order brain functions, from the MEXT (#17022009, #18020005, #20019005), and Academic Frontier Project for Private Universities: ‘Brain Mechanisms for Cognition, Memory and Behavior’ at Nihon University: a matching fund subsidy from the MEXT. J.F. was supported by Global COE Program ‘Basic and Translational Research Center for Global Brain Science’ at Tohoku University from the MEXT. References Bell, D.E., 1982. Regret in decision making under uncertainty. Oper. Res. 30, 961–981. Bender, S., Hellwig, S., Resch, F., Weisbrod, M., 2007. Am I safe? The ventrolateral prefrontal cortex ‘detects’ when an unpleasant event does not occur. NeuroImage 38, 367–385.
1170
J. Fujiwara et al. / NeuroImage 44 (2009) 1163–1170
Bernoulli, D., 1954. Exposition of a new theory on the measurement of risk (translated from Latin). Econometrica 22, 23–36. Breiter, H.C., Aharon, I., Kahneman, D., Dale, A., Shizgal, P., 2001. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30, 619–639. Byrne, R.M., 2002. Mental models and counterfactual thoughts about what might have been. Trends Cogn. Sci. 6, 426–431. Camille, N., Coricelli, G., Sallet, J., Pradat-Diehl, P., Duhamel, J.R., Sirigu, A., 2004. The involvement of the orbitofrontal cortex in the experience of regret. Science 304, 1167–1170. Chandrasekhar, P.V., Capra, C.M., Moore, S., Noussair, C., Berns, G.S., 2008. Neurobiological regret and rejoice functions for aversive outcomes. NeuroImage 39, 1472–1484. Coricelli, G., Critchley, H.D., Joffily, M., O'Doherty, J.P., Sirigu, A., Dolan, R.J., 2005. Regret and its avoidance: a neuroimaging study of choice behavior. Nat. Neurosci. 8, 1255–1262. Critchley, H., 2003. Emotion and its disorders. Br. Med. Bull. 65, 35–47. Friston, K.J., Holmes, A.P., Worsley, K.J., Poline, J.P., Frith, C.D., Frackowiak, R.S.J., 1995. Statistical parametric maps in functional imaging: a general linear approach. Hum. Brain Mapp. 2, 189–210. Gilovich, T., Medvec, V.H., 1995. The experience of regret: what, when, and why. Psychol. Rev. 102, 379–395. Kahneman, D., Tversky, A., 1979. Prospect theory: an analysis of decision under risk. Econometrica 47, 263–291. Knutson, B., Fong, G.W., Adams, C.M., Varner, J.L., Hommer, D., 2001. Dissociation of reward anticipation and outcome with event-related fMRI. NeuroReport 12, 3683–3687. Konorski, J., 1967. Integrative Action of the Brain. University of Chicago Press, Chicago. Kobayashi, S., Nomoto, K., Watanabe, M., Hikosaka, O., Schultz, W., Sakagami, M., 2006. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51, 861–870. Kuhnen, C.M., Knutson, B., 2005. The neural basis of financial risk taking. Neuron 47, 763–770.
Lohrenz, T., McCabe, K., Camerer, C.F., Montague, P.R., 2007. Neural signature of fictive learning signals in a sequential investment task. Proc. Natl. Acad. Sci. U. S. A. 104, 9493–9498. Loomes, G., Sugden, R., 1982. Regret theory: an alternative theory of rational choice under uncertainty. Econ. J. 92, 805–824. Maldijan, J.A., Laurienti, P.J., Kraft, R.A., Burdette, J.H., 2003. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. NeuroImage 19, 1233–1239. Mellers, B.A., McGraw, A.P., 2001. Anticipated emotions as guides to choice. Curr. Dir. Psychol. Sci. 10, 210–214. O'Doherty, J., Kringelbach, M.L., Rolls, E.T., Hornak, J., Andrews, C., 2001. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95–102. Ramnani, N., Elliott, R., Athwal, B.S., Passingham, R.E., 2004. Prediction error for free monetary reward in the human prefrontal cortex. NeuroImage 23, 777–786. Rolls, E.T., 2000. Précis of the brain and emotion. Behav. Brain Sci. 23, 177–191. Sakagami, M., Watanabe, M., 2007. Integration of cognitive and motivational information in the primate lateral prefrontal cortex. Ann. N.Y. Acad. Sci. 1104, 89–107. Solomon, R.L., Corbit, J.D., 1974. An opponent-process theory of motivation: I. Temporal dynamics of affect. Psychol. Rev. 81, 119–145. Tobler, P.N., O'Doherty, J.P., Dolan, R.J., Schultz, W., 2007. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 97, 1621–1632. Watanabe, M., 1996. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632. Watanabe, M., Hikosaka, K., Sakagami, M., Shirakawa, S., 2002. Coding and monitoring of motivational context in the primate prefrontal cortex. J. Neurosci. 22, 2391–2400. Zeelenberg, M., Pieters, R., 2007. A theory of regret regulation 1.0. J. Consum. Psychol. 17, 3–18.