- Email: [email protected]
Using computational neuroscience to investigate the neural correlates of cognitive-affective integration during covert decision making
Brain and Cognition 53 (2003) 398–402 www.elsevier.com/locate/b&c
Using computational neuroscience to investigate the neural correlates of cognitive-affective integration during covert decision making Brandon M. Wagar and Paul Thagard Department of Psychology, University of Waterloo, Waterloo, Ont., Canada Accepted 7 May 2003
Abstract We presented a proposed neural level mechanism for the integration of cognitive and affective information during covert decision making. The central idea is that the ventromedial prefrontal cortex establishes predicted outcomes of responses through its connections with the amygdala, and that this information is passed through the context-moderated gateway in the nucleus accumbens in order to promote behaviours that are most beneficial to the long term survival of the organism. We then implemented the proposed mechanism in a network of spiking neurons, and tested one of its central claims. Results showed that the model was capable of producing behaviour similar to that observed in normal humans, as well as that exhibited during VMPFC damage. Ó 2003 Elsevier Inc. All rights reserved.
1. Introduction What is of particular interest in this paper is the neural level mechanism responsible for cognitive-affective integration. The neural basis for the production of emotional signals in decision making has most likely evolved to ensure the survival of the organism. By biasing the organism to avoid decisions which will lead to negative future outcomes, and seek decisions which will lead to positive future outcomes, such a mechanism pretunes the organism to behave in ways that promote achievement and long term survival. To this end, the mechanism must recruit brain regions involved in the processing and storage of bodily states (in order to avoid unpleasant states and promote homeostatic ones), as well as regions responsible for cognitive processes (in order to process sensory representations). To investigate this issue, we will employ a network of spiking neurons and attempt to replicate the deficit resulting from damage to ventromedial prefrontal cortex (VMPFC). In general, VMPFC damage is characterized by insensitivity to future consequences (see Damasio, 1995). While the impairment is usually discussed as an impairment in the ability to predict the consequences of oneÕs actions within a complex social environment, the 0278-2626/$ - see front matter Ó 2003 Elsevier Inc. All rights reserved. doi:10.1016/S0278-2626(03)00153-2
impairment extends to other decision making tasks that involve distinctions between long-term and short-term consequences in an environment containing punishment and reward. According to the somatic marker hypothesis (Damasio, 1995), sensory representations of a given response to the current situation activate knowledge tied in with previous emotional experience. These somatic markers are the feelings that have become associated, through experience, with the predicted long-term outcomes of certain responses to a given situation. Somatic markers act as covert biases, or gut reactions, influencing the mechanisms responsible for higher level cognitive processes and/or the motor effector sites. These emotions assist us during the decision making process by rapidly highlighting those options that have positive predicted outcomes, while eliminating those options that have negative predicted outcomes from further consideration, thereby allowing the organism to reason according to the long-term predicted outcomes of its actions. 1.1. Neural mechanism In this section we describe a possible neural mechanism for cognitive-affective integration during covert
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
decision making. We suggest that interconnections between VMPFC and the amygdala are responsible for the formation of memory traces that allow the organism to predict the future outcome of a given response. This information is then passed on to the nucleus accumbens (NAcc), which acts as a gateway allowing only context consistent behaviour (as determined by hippocampal inputs to NAcc), to pass through. Information that passes through NAcc is then redirected back to VMPFC and other prefrontal and neocortical sites, providing a nonconscious, emotionally laden response that feeds in to higher level cognitive processes and/or the motor effector sites. In the following sections we elaborate briefly on the processes of establishing predicted outcomes and gating of NAcc throughput (see Wagar & Thagard, in press, for a more detailed description). 1.2. Establishing predicted outcomes Damasio (1995), highlights two key structures as being primarily responsible for the production and utilization of somatic markers: VMPFC and the amygdala. Ventromedial prefrontal cortex receives input from sensory cortices (representing the behavioural options) and limbic structures, most notably the amygdala (which processes somatic states). Through these interconnections between cognitive and emotional processes VMPFC records the signals that define a given response by encoding the representations of certain stimuli and the behavioural significance of the somatic states that have been previously associated with the given response, thereby laying down a memory trace (somatic marker) that represents a given action and the expected consequences of the action. Once the memory trace is encoded, VMPFC houses the critical output for somatic markers to influence decision making. When a given set of inputs to VMPFC elicit activation, VMPFC, through its reciprocal connections with the amygdala, elicits a reenactment of the bodily state consistent with the predicted future outcome of the given behaviour. This emotionally laden signal is then passed on to NAcc. 1.3. Gating of nucleus accumbens throughput Hippocampal gating of NAcc throughput in our model is based primarily on work by OÕDonnell and Grace (1995), and Mogenson, Jones, and Yim (1980). The nucleus accumbens is responsible for mediating basic locomotor and appetitive behaviours that are driven by the affective state of the organism. Thus, it provides the ideal location for cognitive-affective integration during covert decision making tasks. The nucleus accumbens receives afferent connections from VMPFC, the amygdala, the hippocampus and massive dopamine (DA) input from VTA. The nucleus
399
accumbens in turn sends efferent projections, via the ventral palladium, to the mediodorsal nucleus of the thalamus (MD). Importantly, MD is the thalamic region that provides major regulatory control over prefrontal cortex neurons. Ventromedial prefrontal cortex and the amygdala produce brief, small amplitude membrane depolarizations in NAcc, which themselves cannot cause NAcc neurons to fire. This is because NAcc neurons are typically in a hyperpolarized state due to the massive inhibitory DA input from VTA. So, NAcc neurons are being constantly bombarded with VMPFC driven excitatory postsynaptic potentials (epspÕs), yet none of the information is getting through. Hippocampal input, on the other hand, produces large amplitude, long duration plateau-like depolarizations. For the subset of NAcc neurons receiving hippocampal input, their typical hyperpolarized state is disrupted by temporary depolarization plateaus that bring NAcc neuronsÕ activity levels close to firing threshold, thereby allowing any coincidental VMPFC activity to elicit spike activity in NAcc neurons and pass through the NAcc gateway. The hippocampus controls VMPFC and amygdala throughput in the NAcc by allowing only those patterns active in NAcc neurons that are consistent with the current context to elicit spike activity in NAcc neurons. Ventromedial prefrontal cortex input to NAcc neurons represents the multitude of potentially effective responses to a given situation. The hippocampus influences the selection of a given response by facilitating within NAcc only those responses that are congruent with the current context. Because, as was described in the previous section, VMPFC elicits a predicted outcome through reciprocal connections with the amygdala, the VMPFC signal representing a given response and the amygdala signal representing the emotionally laden predicted outcome of that response are generated together and will arrive at NAcc neurons simultaneously. This provides the means by which an emotional valence representing the predicted outcome of a given response can be passed through NAcc to higher level cognitive processes, thereby creating the emotionally laden predicted future outcome to a given response. Given the above description, we test the proposed mechanism using a computer simulation. To do so, we implement the mechanism in a network of spiking neurons. We hypothesize that the network will make decisions based on the predicted outcome of a given response, even though the immediate outcome contradicts the future outcome. We specifically predict that—consistent with the performance of human patients—when presented with a given response, an intact network will behave as described above, while a network in which VMPFC has been lesioned will make decisions based on immediate outcomes rather than future outcomes.
400
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
1.4. Overview of the model In this section, we describe a network of spiking neurons based on the proposed neural mechanism for cognitive-affective integration during covert decision making described above. The model had 700 spiking neurons with approximately 670 connections. The modeled regions consisted of the ventromedial prefrontal cortex, the amygdala, the nucleus accumbens, the hippocampus and the ventral tegmental area. Each region contained 100 neurons (which received input from other regions and/or external input and passed information on to other regions), as well as 40 inhibitory interneurons. The pattern of afferent, efferent and internal connectivity follows that of the proposed neural mechanism. The model included intraregional connections and interregional connections. Individual neurons were modeled as spike response model units, and synaptic interactions followed a Hebbian-based learning rule. In the following sections we describe the implementation of the neuronal and synaptic learning properties. We then describe the stimuli used to activate the model, and the procedures for the collection and analysis of data. 1.5. Spiking neuron model We used a slightly modified leaky integrate-and-fire neuron, based heavily on GerstnerÕs Spike Response Model (1999, Gerstner & van Hemman, 1994). In our model, a neuronÕs reaction to spike emission and dendritic integration are described by two response functions. First, spike emission induces refractoriness, which is modeled by an internal response function gðsÞ. Second, incoming spikes from afferent connections evoke postsynaptic potentials modeled by a response function eðsÞ. Thus, the total membrane potential ai of a given neuron i at time t is given by X X X ðf Þ ðf Þ ai ¼ gi t t i xij eij t tj : þ ð1Þ ðf Þ
ti 2Fi
j2Ci tðf Þ 2F j j
The right-hand side of Eq. (1) can be decomposed into the two response functions mentioned above. Each will be described in more detail below. The left term on the right-hand side of Eq. (1) represents the response of neuron i to the set Fi of all its ðf Þ own previous spike emissions ti . In the event of a spike, a short-term, decaying negative contribution gðsÞ is added to the total membrane potential a, representing the refractoriness, or reduced excitability of a neuron after spike emission. The refractoriness of a given neuron i at time t is given by s gðsÞ ¼ lhi exp H ðsÞ; ð2Þ s
ðf Þ
where s is the time ðt ti Þ since the last spike of neuron i; h is the firing threshold of neuron i, l and s are constants that scale the amplitude and the decay rate of gðsÞ, respectively. Finally, H ðsÞ is the Heaviside step function defined by 0 for s < 0; H ðsÞ ¼ ð3Þ 1 otherwise: The last term on the right-hand side of Eq. (1) represents the response of neuron i to the set Fj of all ðf Þ previous spike emissions tj for the set of all afferent connections Cj to neuron i. The variable xij represents the connection strength between the postsynaptic neuron i and a given presynaptic neuron j. The postsynaptic potential induced in neuron i at time t in response to the firing of presynaptic neuron j at time ðf Þ tI is given by s s eðsÞ ¼ d exp H ðsÞ; ð4Þ 2 ss ss where d is a constant representing the valence of the postsynaptic potential (1 if excitatory, )1 if inhibitory), s ðf Þ is the time ðt tj Þ since the last spike of neuron j, and ss is a time constant representing the rise time of the postsynaptic potential. Again, H ðsÞ is the Heaviside step function described in Eq. (3). If at time t the total membrane potential a of a given neuron reaches the threshold h, a spike is emitted which is transmitted along the axon to other neurons. Each ðf Þ spike results in a new spike time ti being added to the set Fi of all spike times for neuron i. and the total membrane potential a is reset to 0 for one time step (thereby enforcing absolute refractoriness). 1.6. Learning At each time step the connection strengths, xij , are updated synchronously according to a Hebbian learning rule, modeled roughly after Kempter, Gerstner, and van Hemmen (1999). In our model the connection strength, xij , between the postsynaptic neuron, i, and a given presynaptic neuron, j, is adjusted according to the delay ðf Þ between the time, tj , of the most recent presynaptic ðf Þ spike arrival and the time, ti , of the most recent postsynaptic firing. If the delay between presynaptic spike arrival and postsynaptic firing is negative or zero (coincidental), the connection strength, xij , is increased. This change in connection strength is greatest when the delay is zero, and asymptotes rapidly to null as the delay decreases. If the delay between presynaptic spike arrival and postsynaptic firing is positive (j fires after i), the connection strength, xij , is decreased. This change in connection strength is greatest when the delay is near zero (j fires immediately after i), and asymptotes rapidly to null as the delay increases. The formulation of the change of connection strength, xij , is given by:
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
s
s 1 sþ
Dxiij ¼ kij exp Aþ ssyn s þ A 1 for s < 0; s
kij for s ¼ 0; ð5Þ
kij Aþ exp
s sþ
þ A exp ðf Þ
s s
for s > 0;
ðf Þ
where s is the delay (ti ti ) between presynaptic spike arrival and postsynaptic firing. ssyn ; sþ ; s ; sþ ; s are time constants, and Aþ ; A are constants which scale the strength of synaptic potentiation and depression, respectively. Finally, kij is the learning rate for the synapse connecting the presynaptic neuron j to the postsynaptic neuron i. 1.7. Stimuli In a typical experiment, the model was presented with a set of stimulus patterns consisting of activation vectors feeding in to those regions that received external input during the given simulation. Each pattern consisted of 50 active units (thereby exciting 50% of the neurons in the receiving region). The hippocampus and VTA received a single activation pattern each for a given experiment, though these patterns were different from one another and between experiments. VMPFC and the amygdala each required two separate activation patterns for a given experiment. For VMPFC and the amygdala, these patterns represented the two choices (good vs. bad) and two body states (positive vs. negative), respectively. To simplify the task, each set of patterns form orthoganalized pairs. Stimulation was achieved by applying constant synaptic excitation independently to each neuron corresponding to nodes active in the stimulus pattern. This resulted in a constant input signal of approximately 25 spikes per second to those neurons activated by the stimulus pattern. 1.8. Data collection and analysis In a typical run, the model was trained on two combinations of stimuli: good choice (VMPFC) with positive body state (amygdala), and bad choice with negative body state. Training occurred in an interleaved fashion over 4000 time steps, alternating stimulus combination every 400 time steps. Once training was complete, the model was presented for epochs of 2000 time steps with each of the training stimuli in order to establish baseline ensemble activity in NAcc for representations of the good choice and bad choice decisions. Finally, once baseline representation for NAcc had been obtained the model was presented for epochs of 2000
401
time steps with each of the test stimulus combinations. These consisted of a given response (e.g., good choice) and an inconsistent body state (e.g., negative). This allowed us to test whether the network would decide based on future outcomes (e.g., make the good choice even though the immediate outcome is negative) or immediate outcomes.1 The above experiments were performed with an intact network, and with a network in which VMPFC had been removed after training. The membrane potential ai of each neuron in the network was recorded for the entire history of the experiment. In order to characterize the ensemble activation pattern of the NAcc population, membrane potential histories were converted into rate graphs, averaging over 20 time step windows with single step increments. Once the rate graphs were acquired, cluster analysis using PearsonÕs contingency coefficient was used in order to separate the population according to those neurons which were active in response to the stimuli and those which were not responsive (see Everitt, Landau, & Leese, 2001). This was done for the baseline representations and each of the test representations. The test representations were then compared to the baseline representations to determine which choice the network had opted for in a given experiment.
2. Results and discussion To obtain a measure of the networks overall performance, all results are averaged over 50 replications of each experiment (with activation patterns generated randomly for each experiment). After having been trained on the predicted affective outcomes of each response, the network was tested by presenting VMPFC and the amygdala with activation patterns which simulated a condition where future outcome and immediate outcome were in opposition. The goal was to determine whether an intact system would make a decision based on predicted future outcomes, while a VMPFC lesioned system would decide solely on immediate outcomes. As expected, memory traces representing the predicted affective outcome of a given response are what drive behaviour in an intact network. Stored associations between VMPFC and the amygdala are able to elicit a representation of the predicted future outcome of a given response. This information is then fed forward into NAcc, and if it is consistent with the current context it is passed on to higher level cognitive processes and/or the motor effector sites. 1 Given the scope of the current paper, we only present data for NAcc activity here. For a full analysis of the network, see Wagar and Thagard (in press). We note, however, that interpretation of the entire network dynamics is consistent with the conclusions drawn here.
402
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
Fig. 1. Mean similarity to baseline for test stimuli as a function of baseline emotional reaction when GAGE is intact (top), and when VMPFC is lesioned in GAGE (bottom).
Fig. 1 (top) shows the similarity between the pattern of activity in NAcc neurons representing the baseline decisions and those chosen by the network. As can be seen, even in the presence of a conflicting emotional signal representing the immediate affective outcome of the response, the network makes a decision based on the predicted future outcome of the response. This is the most efficient decision, as it ensures that an organism will behave in ways that promote long term survival, rather than short term satisfaction. However, in the event of VMPFC damage the mechanism no longer functions effectively. Rather than deciding based on long term benefits, immediate affective outcomes drive behaviour. Damage to VMPFC prevents stored associations from eliciting a representation of the predicted future outcome of a given response. Thus, the only information fed into NAcc is that which is initiated by amygdala response to current body states. Because this information is consistent with the current context the amygdala driven activity is passed on to higher cognitive processes and/or the motor effector sites.
Fig. 1 (bottom) shows the similarity between the pattern of activity in NAcc neurons representing the baseline decisions and those chosen by the network. As can be seen, the immediate affective reward of a given response now dominates information flow through Nacc. Rather than behaving in an efficient manner and promoting decisions based on long term survival, the organism will behave in ways that appear impulsive and even irrational. Thus, the model presented here is capable of producing behaviour similar to that observed in normal humans, as well as the deficits exhibited by VMPFC damage. It does so by exhibiting a phenomenon like cognitive-affective integration during covert decision making. The central idea is that VMPFC establishes predicted outcomes of responses through its connections with the amygdala, and that this information is passed through the context-moderated gateway in NAcc in order to promote behaviours that are most beneficial to the long term survival of the organism within the current environment.
References Damasio, A. (1995). Descartes error: Emotion, reason and the human brain. New York: Avon Books Inc. Everitt, B., Landau, S., & Leese, M. (2001). Cluster analysis (4th ed.). New York: Oxford University Press. Gerstner, W. (1999). Spiking neurons. In Maass, & Bishop (Eds.), Pulsed neural networks (pp. 3–54). Cambridge: MIT press. Gerstner, W., & van Hemmen, J. (1994). Coding and information processing in neural networks. In Domnay, van Hemmen, & Schulten (Eds.), Models of Neural Networks II (pp. 1–93). New York: Springer-Verlag. Kempter, R., Gerstner, W., & van Hemmen, J. (1999). Hebbian learning and spiking neurons. Physical Review E, 59, 4498–4514. Mogenson, G., Jones, D., & Yim, C. (1980). From motivation to action: Functional interface between the limbic system and the motor system. Progress in Neurobiology, 14, 69–97. OÕDonnell, P., & Grace, A. (1995). Synaptic interactions among excitatory afferents to nucleus accumbens neurons: Hippocampal gating of prefrontal cortical input. Journal of Neuroscience, 15, 3622–3639. Wagar, B. M., & Thagard, P. (in press). Spiking Phineas Gage: A Neurocomputational Theory of Cognitive-Affective Integration in Decision Making. Psychological Review.
Using computational neuroscience to investigate the neural correlates of cognitive-affective integration during covert decision making Brandon M. Wagar and Paul Thagard Department of Psychology, University of Waterloo, Waterloo, Ont., Canada Accepted 7 May 2003
Abstract We presented a proposed neural level mechanism for the integration of cognitive and affective information during covert decision making. The central idea is that the ventromedial prefrontal cortex establishes predicted outcomes of responses through its connections with the amygdala, and that this information is passed through the context-moderated gateway in the nucleus accumbens in order to promote behaviours that are most beneficial to the long term survival of the organism. We then implemented the proposed mechanism in a network of spiking neurons, and tested one of its central claims. Results showed that the model was capable of producing behaviour similar to that observed in normal humans, as well as that exhibited during VMPFC damage. Ó 2003 Elsevier Inc. All rights reserved.
1. Introduction What is of particular interest in this paper is the neural level mechanism responsible for cognitive-affective integration. The neural basis for the production of emotional signals in decision making has most likely evolved to ensure the survival of the organism. By biasing the organism to avoid decisions which will lead to negative future outcomes, and seek decisions which will lead to positive future outcomes, such a mechanism pretunes the organism to behave in ways that promote achievement and long term survival. To this end, the mechanism must recruit brain regions involved in the processing and storage of bodily states (in order to avoid unpleasant states and promote homeostatic ones), as well as regions responsible for cognitive processes (in order to process sensory representations). To investigate this issue, we will employ a network of spiking neurons and attempt to replicate the deficit resulting from damage to ventromedial prefrontal cortex (VMPFC). In general, VMPFC damage is characterized by insensitivity to future consequences (see Damasio, 1995). While the impairment is usually discussed as an impairment in the ability to predict the consequences of oneÕs actions within a complex social environment, the 0278-2626/$ - see front matter Ó 2003 Elsevier Inc. All rights reserved. doi:10.1016/S0278-2626(03)00153-2
impairment extends to other decision making tasks that involve distinctions between long-term and short-term consequences in an environment containing punishment and reward. According to the somatic marker hypothesis (Damasio, 1995), sensory representations of a given response to the current situation activate knowledge tied in with previous emotional experience. These somatic markers are the feelings that have become associated, through experience, with the predicted long-term outcomes of certain responses to a given situation. Somatic markers act as covert biases, or gut reactions, influencing the mechanisms responsible for higher level cognitive processes and/or the motor effector sites. These emotions assist us during the decision making process by rapidly highlighting those options that have positive predicted outcomes, while eliminating those options that have negative predicted outcomes from further consideration, thereby allowing the organism to reason according to the long-term predicted outcomes of its actions. 1.1. Neural mechanism In this section we describe a possible neural mechanism for cognitive-affective integration during covert
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
decision making. We suggest that interconnections between VMPFC and the amygdala are responsible for the formation of memory traces that allow the organism to predict the future outcome of a given response. This information is then passed on to the nucleus accumbens (NAcc), which acts as a gateway allowing only context consistent behaviour (as determined by hippocampal inputs to NAcc), to pass through. Information that passes through NAcc is then redirected back to VMPFC and other prefrontal and neocortical sites, providing a nonconscious, emotionally laden response that feeds in to higher level cognitive processes and/or the motor effector sites. In the following sections we elaborate briefly on the processes of establishing predicted outcomes and gating of NAcc throughput (see Wagar & Thagard, in press, for a more detailed description). 1.2. Establishing predicted outcomes Damasio (1995), highlights two key structures as being primarily responsible for the production and utilization of somatic markers: VMPFC and the amygdala. Ventromedial prefrontal cortex receives input from sensory cortices (representing the behavioural options) and limbic structures, most notably the amygdala (which processes somatic states). Through these interconnections between cognitive and emotional processes VMPFC records the signals that define a given response by encoding the representations of certain stimuli and the behavioural significance of the somatic states that have been previously associated with the given response, thereby laying down a memory trace (somatic marker) that represents a given action and the expected consequences of the action. Once the memory trace is encoded, VMPFC houses the critical output for somatic markers to influence decision making. When a given set of inputs to VMPFC elicit activation, VMPFC, through its reciprocal connections with the amygdala, elicits a reenactment of the bodily state consistent with the predicted future outcome of the given behaviour. This emotionally laden signal is then passed on to NAcc. 1.3. Gating of nucleus accumbens throughput Hippocampal gating of NAcc throughput in our model is based primarily on work by OÕDonnell and Grace (1995), and Mogenson, Jones, and Yim (1980). The nucleus accumbens is responsible for mediating basic locomotor and appetitive behaviours that are driven by the affective state of the organism. Thus, it provides the ideal location for cognitive-affective integration during covert decision making tasks. The nucleus accumbens receives afferent connections from VMPFC, the amygdala, the hippocampus and massive dopamine (DA) input from VTA. The nucleus
399
accumbens in turn sends efferent projections, via the ventral palladium, to the mediodorsal nucleus of the thalamus (MD). Importantly, MD is the thalamic region that provides major regulatory control over prefrontal cortex neurons. Ventromedial prefrontal cortex and the amygdala produce brief, small amplitude membrane depolarizations in NAcc, which themselves cannot cause NAcc neurons to fire. This is because NAcc neurons are typically in a hyperpolarized state due to the massive inhibitory DA input from VTA. So, NAcc neurons are being constantly bombarded with VMPFC driven excitatory postsynaptic potentials (epspÕs), yet none of the information is getting through. Hippocampal input, on the other hand, produces large amplitude, long duration plateau-like depolarizations. For the subset of NAcc neurons receiving hippocampal input, their typical hyperpolarized state is disrupted by temporary depolarization plateaus that bring NAcc neuronsÕ activity levels close to firing threshold, thereby allowing any coincidental VMPFC activity to elicit spike activity in NAcc neurons and pass through the NAcc gateway. The hippocampus controls VMPFC and amygdala throughput in the NAcc by allowing only those patterns active in NAcc neurons that are consistent with the current context to elicit spike activity in NAcc neurons. Ventromedial prefrontal cortex input to NAcc neurons represents the multitude of potentially effective responses to a given situation. The hippocampus influences the selection of a given response by facilitating within NAcc only those responses that are congruent with the current context. Because, as was described in the previous section, VMPFC elicits a predicted outcome through reciprocal connections with the amygdala, the VMPFC signal representing a given response and the amygdala signal representing the emotionally laden predicted outcome of that response are generated together and will arrive at NAcc neurons simultaneously. This provides the means by which an emotional valence representing the predicted outcome of a given response can be passed through NAcc to higher level cognitive processes, thereby creating the emotionally laden predicted future outcome to a given response. Given the above description, we test the proposed mechanism using a computer simulation. To do so, we implement the mechanism in a network of spiking neurons. We hypothesize that the network will make decisions based on the predicted outcome of a given response, even though the immediate outcome contradicts the future outcome. We specifically predict that—consistent with the performance of human patients—when presented with a given response, an intact network will behave as described above, while a network in which VMPFC has been lesioned will make decisions based on immediate outcomes rather than future outcomes.
400
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
1.4. Overview of the model In this section, we describe a network of spiking neurons based on the proposed neural mechanism for cognitive-affective integration during covert decision making described above. The model had 700 spiking neurons with approximately 670 connections. The modeled regions consisted of the ventromedial prefrontal cortex, the amygdala, the nucleus accumbens, the hippocampus and the ventral tegmental area. Each region contained 100 neurons (which received input from other regions and/or external input and passed information on to other regions), as well as 40 inhibitory interneurons. The pattern of afferent, efferent and internal connectivity follows that of the proposed neural mechanism. The model included intraregional connections and interregional connections. Individual neurons were modeled as spike response model units, and synaptic interactions followed a Hebbian-based learning rule. In the following sections we describe the implementation of the neuronal and synaptic learning properties. We then describe the stimuli used to activate the model, and the procedures for the collection and analysis of data. 1.5. Spiking neuron model We used a slightly modified leaky integrate-and-fire neuron, based heavily on GerstnerÕs Spike Response Model (1999, Gerstner & van Hemman, 1994). In our model, a neuronÕs reaction to spike emission and dendritic integration are described by two response functions. First, spike emission induces refractoriness, which is modeled by an internal response function gðsÞ. Second, incoming spikes from afferent connections evoke postsynaptic potentials modeled by a response function eðsÞ. Thus, the total membrane potential ai of a given neuron i at time t is given by X X X ðf Þ ðf Þ ai ¼ gi t t i xij eij t tj : þ ð1Þ ðf Þ
ti 2Fi
j2Ci tðf Þ 2F j j
The right-hand side of Eq. (1) can be decomposed into the two response functions mentioned above. Each will be described in more detail below. The left term on the right-hand side of Eq. (1) represents the response of neuron i to the set Fi of all its ðf Þ own previous spike emissions ti . In the event of a spike, a short-term, decaying negative contribution gðsÞ is added to the total membrane potential a, representing the refractoriness, or reduced excitability of a neuron after spike emission. The refractoriness of a given neuron i at time t is given by s gðsÞ ¼ lhi exp H ðsÞ; ð2Þ s
ðf Þ
where s is the time ðt ti Þ since the last spike of neuron i; h is the firing threshold of neuron i, l and s are constants that scale the amplitude and the decay rate of gðsÞ, respectively. Finally, H ðsÞ is the Heaviside step function defined by 0 for s < 0; H ðsÞ ¼ ð3Þ 1 otherwise: The last term on the right-hand side of Eq. (1) represents the response of neuron i to the set Fj of all ðf Þ previous spike emissions tj for the set of all afferent connections Cj to neuron i. The variable xij represents the connection strength between the postsynaptic neuron i and a given presynaptic neuron j. The postsynaptic potential induced in neuron i at time t in response to the firing of presynaptic neuron j at time ðf Þ tI is given by s s eðsÞ ¼ d exp H ðsÞ; ð4Þ 2 ss ss where d is a constant representing the valence of the postsynaptic potential (1 if excitatory, )1 if inhibitory), s ðf Þ is the time ðt tj Þ since the last spike of neuron j, and ss is a time constant representing the rise time of the postsynaptic potential. Again, H ðsÞ is the Heaviside step function described in Eq. (3). If at time t the total membrane potential a of a given neuron reaches the threshold h, a spike is emitted which is transmitted along the axon to other neurons. Each ðf Þ spike results in a new spike time ti being added to the set Fi of all spike times for neuron i. and the total membrane potential a is reset to 0 for one time step (thereby enforcing absolute refractoriness). 1.6. Learning At each time step the connection strengths, xij , are updated synchronously according to a Hebbian learning rule, modeled roughly after Kempter, Gerstner, and van Hemmen (1999). In our model the connection strength, xij , between the postsynaptic neuron, i, and a given presynaptic neuron, j, is adjusted according to the delay ðf Þ between the time, tj , of the most recent presynaptic ðf Þ spike arrival and the time, ti , of the most recent postsynaptic firing. If the delay between presynaptic spike arrival and postsynaptic firing is negative or zero (coincidental), the connection strength, xij , is increased. This change in connection strength is greatest when the delay is zero, and asymptotes rapidly to null as the delay decreases. If the delay between presynaptic spike arrival and postsynaptic firing is positive (j fires after i), the connection strength, xij , is decreased. This change in connection strength is greatest when the delay is near zero (j fires immediately after i), and asymptotes rapidly to null as the delay increases. The formulation of the change of connection strength, xij , is given by:
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
s
s 1 sþ
Dxiij ¼ kij exp Aþ ssyn s þ A 1 for s < 0; s
kij for s ¼ 0; ð5Þ
kij Aþ exp
s sþ
þ A exp ðf Þ
s s
for s > 0;
ðf Þ
where s is the delay (ti ti ) between presynaptic spike arrival and postsynaptic firing. ssyn ; sþ ; s ; sþ ; s are time constants, and Aþ ; A are constants which scale the strength of synaptic potentiation and depression, respectively. Finally, kij is the learning rate for the synapse connecting the presynaptic neuron j to the postsynaptic neuron i. 1.7. Stimuli In a typical experiment, the model was presented with a set of stimulus patterns consisting of activation vectors feeding in to those regions that received external input during the given simulation. Each pattern consisted of 50 active units (thereby exciting 50% of the neurons in the receiving region). The hippocampus and VTA received a single activation pattern each for a given experiment, though these patterns were different from one another and between experiments. VMPFC and the amygdala each required two separate activation patterns for a given experiment. For VMPFC and the amygdala, these patterns represented the two choices (good vs. bad) and two body states (positive vs. negative), respectively. To simplify the task, each set of patterns form orthoganalized pairs. Stimulation was achieved by applying constant synaptic excitation independently to each neuron corresponding to nodes active in the stimulus pattern. This resulted in a constant input signal of approximately 25 spikes per second to those neurons activated by the stimulus pattern. 1.8. Data collection and analysis In a typical run, the model was trained on two combinations of stimuli: good choice (VMPFC) with positive body state (amygdala), and bad choice with negative body state. Training occurred in an interleaved fashion over 4000 time steps, alternating stimulus combination every 400 time steps. Once training was complete, the model was presented for epochs of 2000 time steps with each of the training stimuli in order to establish baseline ensemble activity in NAcc for representations of the good choice and bad choice decisions. Finally, once baseline representation for NAcc had been obtained the model was presented for epochs of 2000
401
time steps with each of the test stimulus combinations. These consisted of a given response (e.g., good choice) and an inconsistent body state (e.g., negative). This allowed us to test whether the network would decide based on future outcomes (e.g., make the good choice even though the immediate outcome is negative) or immediate outcomes.1 The above experiments were performed with an intact network, and with a network in which VMPFC had been removed after training. The membrane potential ai of each neuron in the network was recorded for the entire history of the experiment. In order to characterize the ensemble activation pattern of the NAcc population, membrane potential histories were converted into rate graphs, averaging over 20 time step windows with single step increments. Once the rate graphs were acquired, cluster analysis using PearsonÕs contingency coefficient was used in order to separate the population according to those neurons which were active in response to the stimuli and those which were not responsive (see Everitt, Landau, & Leese, 2001). This was done for the baseline representations and each of the test representations. The test representations were then compared to the baseline representations to determine which choice the network had opted for in a given experiment.
2. Results and discussion To obtain a measure of the networks overall performance, all results are averaged over 50 replications of each experiment (with activation patterns generated randomly for each experiment). After having been trained on the predicted affective outcomes of each response, the network was tested by presenting VMPFC and the amygdala with activation patterns which simulated a condition where future outcome and immediate outcome were in opposition. The goal was to determine whether an intact system would make a decision based on predicted future outcomes, while a VMPFC lesioned system would decide solely on immediate outcomes. As expected, memory traces representing the predicted affective outcome of a given response are what drive behaviour in an intact network. Stored associations between VMPFC and the amygdala are able to elicit a representation of the predicted future outcome of a given response. This information is then fed forward into NAcc, and if it is consistent with the current context it is passed on to higher level cognitive processes and/or the motor effector sites. 1 Given the scope of the current paper, we only present data for NAcc activity here. For a full analysis of the network, see Wagar and Thagard (in press). We note, however, that interpretation of the entire network dynamics is consistent with the conclusions drawn here.
402
B.M. Wagar, P. Thagard / Brain and Cognition 53 (2003) 398–402
Fig. 1. Mean similarity to baseline for test stimuli as a function of baseline emotional reaction when GAGE is intact (top), and when VMPFC is lesioned in GAGE (bottom).
Fig. 1 (top) shows the similarity between the pattern of activity in NAcc neurons representing the baseline decisions and those chosen by the network. As can be seen, even in the presence of a conflicting emotional signal representing the immediate affective outcome of the response, the network makes a decision based on the predicted future outcome of the response. This is the most efficient decision, as it ensures that an organism will behave in ways that promote long term survival, rather than short term satisfaction. However, in the event of VMPFC damage the mechanism no longer functions effectively. Rather than deciding based on long term benefits, immediate affective outcomes drive behaviour. Damage to VMPFC prevents stored associations from eliciting a representation of the predicted future outcome of a given response. Thus, the only information fed into NAcc is that which is initiated by amygdala response to current body states. Because this information is consistent with the current context the amygdala driven activity is passed on to higher cognitive processes and/or the motor effector sites.
Fig. 1 (bottom) shows the similarity between the pattern of activity in NAcc neurons representing the baseline decisions and those chosen by the network. As can be seen, the immediate affective reward of a given response now dominates information flow through Nacc. Rather than behaving in an efficient manner and promoting decisions based on long term survival, the organism will behave in ways that appear impulsive and even irrational. Thus, the model presented here is capable of producing behaviour similar to that observed in normal humans, as well as the deficits exhibited by VMPFC damage. It does so by exhibiting a phenomenon like cognitive-affective integration during covert decision making. The central idea is that VMPFC establishes predicted outcomes of responses through its connections with the amygdala, and that this information is passed through the context-moderated gateway in NAcc in order to promote behaviours that are most beneficial to the long term survival of the organism within the current environment.
References Damasio, A. (1995). Descartes error: Emotion, reason and the human brain. New York: Avon Books Inc. Everitt, B., Landau, S., & Leese, M. (2001). Cluster analysis (4th ed.). New York: Oxford University Press. Gerstner, W. (1999). Spiking neurons. In Maass, & Bishop (Eds.), Pulsed neural networks (pp. 3–54). Cambridge: MIT press. Gerstner, W., & van Hemmen, J. (1994). Coding and information processing in neural networks. In Domnay, van Hemmen, & Schulten (Eds.), Models of Neural Networks II (pp. 1–93). New York: Springer-Verlag. Kempter, R., Gerstner, W., & van Hemmen, J. (1999). Hebbian learning and spiking neurons. Physical Review E, 59, 4498–4514. Mogenson, G., Jones, D., & Yim, C. (1980). From motivation to action: Functional interface between the limbic system and the motor system. Progress in Neurobiology, 14, 69–97. OÕDonnell, P., & Grace, A. (1995). Synaptic interactions among excitatory afferents to nucleus accumbens neurons: Hippocampal gating of prefrontal cortical input. Journal of Neuroscience, 15, 3622–3639. Wagar, B. M., & Thagard, P. (in press). Spiking Phineas Gage: A Neurocomputational Theory of Cognitive-Affective Integration in Decision Making. Psychological Review.