Primate orbitofrontal cortex and adaptive behaviour

Review

TRENDS in Cognitive Sciences

Vol.10 No.2 February 2006

Primate orbitofrontal cortex and adaptive behaviour A.C. Roberts Department of Anatomy, University of Cambridge, Cambridge, CB2 3DY, and MRC Centre for Behavioural and Clinical Neuroscience, University of Cambridge, Downing Street, Cambridge, CB2 3EB, UK

Orbitofrontal cortex contributes to behavioural adaptation in response to changes in the contingent relationship and incentive value of positive affective stimuli in the environment. This article integrates early descriptions of the effects of orbitofrontal ablation in monkeys, on object discrimination reversal and extinction, with contemporary theories of animal learning. Studies of incentive devaluation, conditioned reinforcement and changes in reward contingency are reviewed, highlighting the role of the orbitofrontal cortex in processing the affective and non-affective properties of rewarding stimuli, in reward expectation, and in goal selection. It is argued that future studies should focus on the interaction of the orbitofrontal cortex with peripheral arousal systems and the ascending monoamine systems in order to understand fully the role of the orbitofrontal cortex in behavioural adaptation.

Introduction Profoundly disturbed emotional and social behaviour and poor decision making are associated with damage to the orbitofrontal cortex (OFC), the brain region that lies above the orbits in the ventral region of the frontal lobe. In humans, such behavioural disturbance can result from brain tumors, surgical interventions for intractable epilepsy, or trauma that induce gross damage within the OFC. It is also present in drug addiction and in a variety of neuropsychiatric disorders including schizophrenia, obsessive–compulsive disorder, autism, depression and sociopathies as well as neurodegenerative disorders including the frontal variant of Pick’s disease. In all these disorders there is no gross damage to the OFC but dysregulation of its functioning. In human and non-human primates the OFC comprises several regions that are differentiable according to their cyto- and neurochemical architecture [1] (see Box 1), their connections both within and outside the frontal lobes [2] and to some extent, the behavioural deficits that have been shown to arise as a result of their damage [3,4]. The most medial region of the OFC is intimately connected with the adjacent, ventral region of the medial prefrontal cortex (PFC); the medial PFC being that area of cortex lying within the cingulate gyrus anterior to the genu of the corpus callosum. In humans, damage to much of the OFC Corresponding author: Roberts, A.C. ([email protected]). Available online 27 December 2005

together with the ventral aspects of the medial PFC is associated with disturbed emotional and social behaviour [5,6]. Although similar changes in social and emotional behaviour have also been observed in monkeys with orbitofrontal damage (for review see [7]), the most well described behavioural characteristic of such monkeys had been, until recently, their inflexible responding across a variety of different contexts [7,8]. Recent progress in our understanding of the role of the OFC in emotional and social behaviour has been stimulated by (i) a greater understanding of the interaction between Pavlovian and instrumental learning processes that have been proposed to underlie much of behaviour and (ii) a renewed emphasis on comparing the role of the OFC with related structures including the neighbouring medial PFC and the amygdala. This review will consider the unique contribution of the OFC to specific associative learning mechanisms in the context of positive affective behaviour, and how such mechanisms might provide the flexibility necessary for complex emotional and social interactions within sophisticated human and non-human primate societies. Although the focus will be on non-human primates, relevant evidence from human neuroimaging and rodent experimental studies will also be discussed. Early studies on discrimination learning in monkeys The importance of the primate OFC in adaptive behaviour has been recognized since the publication of a series of influential experiments in the 1960s and early 1970s. Based on ablation studies in old world monkeys using a range of discrimination tests including ‘go, no-go’, onetrial object discrimination, object discrimination reversal as well as instrumental extinction it was hypothesized that the medial sector of the OFC (mOFC: including areas 13, orbital 14, much of area 11 and the tip of orbital 10) was involved in processing affective information, consistent with its dense reciprocal connections with the amygdala [4], whereas the inferior prefrontal convexity (area 12/47 and parts of area 45), which included the lateral sector of OFC, was involved in the suppression of inappropriate cognitive sets [8]. More recent findings in New World monkeys have shown that the failure to suppress inappropriate cognitive sets can be fractionated; the ability to shift attentional sets and to reverse stimulus–reward associations being differentially dependent upon distinct regions of PFC [9]. Moreover, ablations of the posteromedial sector of OFC but to a much lesser

www.sciencedirect.com 1364-6613/$ - see front matter Q 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2005.12.002

84

Review

TRENDS in Cognitive Sciences

Vol.10 No.2 February 2006

Box 1. The anatomy of the primate orbitofrontal cortex and the medial wall The orbital surface of the frontal lobes in non-human primates is composed of several cytoarchitectonically distinct regions that have differing patterns of connections both within and outside the frontal lobes. These regions include areas 10, 11, 12/47, 13 and 14, according to the maps of Petrides and Pandya [68] and Carmichael and Price [1]. Based on an extensive analysis of the connections of the OFC and medial PFC Price and colleagues [2] have divided these regions into two networks (Figure I). The orbital network includes area 12 (m, r and l), the majority of 13 (l, m and b), 11I and most of the agranular insula. This network has been proposed to function as a system for sensory integration because these regions receive highly processed

information from all the major senses: visual, auditory, somatosensory, gustatory, olfactory and visceral. The remaining regions on the orbital surface, 14r and 14c, 10o, 11m and an intermediate section of agranular insula (Iai) have extensive connections with the medial wall of the frontal lobes, including areas 10m, 32, 24 and 25, and together have been proposed to form a medial network (shaded grey). This is thought to function as a visceromotor system because its component regions have little sensory input but provide major cortical output to visceromotor structures in the hypothalamus and brainstem. Areas13a and 12o (hatched regions) have substantial connections with both networks.

Figure I. The anatomy of primate orbitofrontal cortex and medial wall. See text for details and abbreviations. Redrawn with permission from [1]. Copyright 1994 John Wiley & Sons.

Box 2. Learning processes contributing to performance on object discrimination tests In a typical object discrimination task for primates, two objects are presented on each trial, with the location of each object, on the animals right and left, varying pseudorandomly across trials. A response to one object leads to reward and a response to the other is unrewarded. An interaction of Pavlovian and instrumental learning processes underlie a monkey’s performance on this simple task although their relative control over responding will depend upon the precise task parameters including the salience of the objects, the incentive value of the reward and the precise contingencies between objects, responses and reward.

Pavlovian associations The repeated pairing of one of the objects with reward can lead to that object becoming a conditioned stimulus (CS) such that it will elicit Pavlovian, that is, automatic or reflexive, conditioned responses when presented alone in the absence of the reward; responses that include behavioural approach towards the CS (object) itself [69], as well as approach towards the food. Often the location of the CS and the food are spatially overlapping, as in the object-discrimination task, but when spatially segregated, approach responses to the CS can occur independently of approach responses to the food source itself [70]. Such responses are under differential neural control; excitotoxic lesions of the central nucleus of the amygdala disrupting CS, but not US directed responses [26]. Such CS elicited approach responses to both the food and the CS itself could conceivably underlie performance of primates in the object discrimination task where only a simple displacement response of an object is required to gain access to reward. Besides the formation of object (stimulus)–response associations, associations also form between the object and the sensory and affective properties of the reward such that Pavlovian responding is sensitive to changes in the contingent relationship between an object and reward and also to changes in the incentive value of that reward

www.sciencedirect.com

(for review, see [71]). In addition, the object itself, by virtue of its association with reward, can become a conditioned reinforcer, and thus the animal selects that object as a consequence of the objects own reinforcing efficacy [72]. This effect could be due to the CS acting to retrieve the reinforcing value of the primary reward with which it has been paired and/or might be a consequence of the CS acting as a general positive reinforcer. That conditioned stimuli possess such general motivational properties has been demonstrated, and recently it has been shown that a conditioned reinforcer can indeed support new learning in rats despite devaluation of the primary reinforcer with which it was originally paired [for discussion, see 22]. Under some circumstances however it is the predictive and discriminative properties of the object with respect to the sensory and informational properties of the reward that might guide responding independent of the rewards incentive value [73,74].

Instrumental associations The act of displacing the object and retrieving food reward will also lead to instrumental conditioning involving at least two different forms of learning [75]. One, a form of stimulus–response, procedural or habit learning is dependent upon the retrieval of food reward (following displacement of the object) strengthening the association between the object and that displacement response but the reward per se does not enter into the association. This type of stimulus-induced responding is insensitive to changes in the incentive value of the reward or changes in the contingency between the response and reward unless new learning is allowed to take place. The second fulfils the criterion of goal directed actions in that animals learn the contingent relationship between object displacement (the response) and the outcome (reward) as well as the incentive value of that outcome and consequently are sensitive to degradation of the contingency or devaluation of the outcome.

Review

TRENDS in Cognitive Sciences

extent, the inferior prefrontal convexity, caused a prolonged response upon extinction of an instrumental response [3]. Although these findings highlighted the role of the OFC in adaptive behaviour in general, and specifically, that involving reward processing, they left the precise nature of its contributions unclear, not least because the tests of discrimination and instrumental responding so often used involved multiple, interacting, Pavlovian and instrumental learning processes (see Box 2). More recently, the specific contribution of the primate OFC to some of these associative learning processes has begun to be explored.

CS+

CS omission

PFo

CS–

Post CS omission

Con

0.7

*

*

0.6

**

30

0.5

20 15 10 5

0.4 0.3 0.2

0

0 OFC lesion

Medial PFC lesion

25 20 15 10 5

0.1

Control

(c) Reinforcer devaluation test

Difference scores

25

(b) Omission of the CS on a second order schedule of food reward

Response ratio

Mean responses

30

85

presentation of an appetitive CS [16], although the latter do show impaired acquisition (but not performance) of conditioned approach towards an appetitive CS itself [17] (see [18] for comparison of orbital and medial regions in rats and monkeys). As highlighted in Box 2, many of these approach responses are the result of the formation of stimulus–response associations, strengthened by reward. However, a food associated CS can also potentiate feeding in food-satiated, OFC lesioned rats indicating that certain motivational properties of the CS also remain intact after such lesions and can enhance consummatory responding [19]. By contrast, marmosets with OFC lesions fail to select a novel object to gain access to a conditioned reinforcer, that is, a stimulus that has gained reinforcing efficacy in its own right as a consequence of previous pairing with food reward (see Figure 1a; [20]). In addition, their ability to maintain responding across protracted periods of time under lean schedules of primary reward is no longer dependent upon intermittent presentations of a response contingent CS (Figure 1b; [20]). This is in marked contrast to control monkeys whose responding on such a second-order schedule of reinforcement decreases markedly upon removal of the CS (for a review of second-order schedules, see [21]). The specificity of this effect is shown by the spared performance of monkeys with lesions of the medial PFC confirming that it is the OFC that is important in guiding goal directed actions based on the incentive value of conditioned stimuli. To the extent that recent evidence supports the hypothesis that acquisition of a new response for conditioned reinforcement might depend upon the CS evoking a general positive

Orbitofrontal cortex and incentive value Electrophysiological recording studies in animals and functional neuroimaging studies in humans have highlighted the sensitivity of OFC responses to the incentive value of conditioned and unconditioned stimuli [10–13]. These responses reflect changes in incentive value brought about by (i) alterations in general motivational state (i.e. deprivation or satiety) [10], and (ii) rewardspecific devaluation or inflation [10,14]. In addition, OFC responses reflect (iii) the relative value of the reward compared with other, currently available reward alternatives [15]. Given this responsivity of the OFC to the incentive value of stimuli what are the specific behavioural consequences if the OFC is damaged? Following OFC ablations monkeys can still learn to select a novel object to gain access to food reward [4,9] and rats with excitotoxic lesions of the OFC exhibit intact conditioned approach towards a food cup upon

(a) Acquisition of a new response for conditioned reinforcement

Vol.10 No.2 February 2006

0 Control

OFC lesion

Medial PFC lesion

Test 3 Test 4 Objects + Food Food only TRENDS in Cognitive Sciences

Figure 1. The effects of lesions of the monkey orbitofrontal cortex on behaviour guided by conditioned stimuli. (a) Mean number of responses made by control, OFC and medial PFC lesioned groups to a visual stimulus associated with a CSC compared with a stimulus associated with a CS–. Control and medial PFC-lesioned groups showed differential responding to the stimulus associated with the CSC compared with that associated with the CS– (*, p!0.05; **, p!0.01), but OFC-lesioned animals responded equally to the two stimuli. (Adapted from [20].) (b) Performance by control, OFC and medial PFC lesioned groups on a second-order schedule for food reward following removal of the CS for one day only (CS omission) and upon its return (post-CS omission). Whereas control and medial-PFC lesioned groups showed a decline in performance upon CS omission, the OFC-lesioned group did not. Performance is presented as the ratio of responses relative to the two baseline days immediately before CS omission. A score below 0.5 indicates a decrease in responding. (*, response ratio during CS omission in the OFC lesioned group was significantly greater than that of controls at p!0.05.) (Adapted from [20].) (c) Mean difference scores of control and OFC-lesioned groups on two tests of reinforcer devaluation: a test with objects overlying the food rewards (test 3 in [23]) and a test with food rewards presented alone (test 4). The OFC-lesioned group was impaired at avoiding the objects associated with the devalued food reward but was unimpaired at avoiding the devalued food itself. (Redrawn with permission from [23]. Copyright 2004, Society for Neuroscience.) www.sciencedirect.com

86

Review

TRENDS in Cognitive Sciences

affective state [22], this would suggest that the OFC contributes to the process by which general affective value can guide behaviour. The OFC also contributes to the process by which a CS accesses a specific representation (or memory) of the incentive value of a particular reinforcer and guides behaviour accordingly. Thus, if sometime after having received different rewards for selecting particular objects, the incentive value of the rewards change, rhesus monkeys with mOFC ablations do not spontaneously use this information to guide their selection of these objects on a subsequent occasion (see Figure 1c; [23]). The finding that rats with OFC lesions show a similar impairment in a Pavlovian setting [16] shows that the involvement of the OFC in this process is not specific to instrumental responding, but also impacts on Pavlovian responding. Orbitofrontal cortex, expectation and goal selection The process by which a CS evokes a representation of the incentive value of the specific reinforcer provides a mechanism for reward expectation. Indeed, Schoenbaum and colleagues have identified a population of OFC neurons in rats, performing a go, no-go odor discrimination test, that are active not only during the presentation of a CS but also following the response, just before the delivery of a motivationally significant outcome. These findings are consistent with the hypothesis that OFC neurons encode the specific incentive value of the expected outcome predicted by the cue. Of interest here is the finding that a distinct population of neurons that also fire during CS presentation do not show such anticipatory firing with respect to the outcome; these neurons have been hypothesized to encode instead the acquired affective value of the CS itself, that is, the CS’s conditioned reinforcing properties [24]. As highlighted by the authors, such an interpretation of their findings is consistent with the results of lesion studies in that, following an OFC lesion, not only is behaviour insensitive to changes in the current value of a reward associated with a particular CS [16,23], but a CS no longer acts as a conditioned reinforcer [20]. Further insight into the role of the OFC in outcome expectation has come from the novel approach of studying neuronal activity in the OFC in the absence of a specific class of inputs, in this case, those from the basolateral nucleus of the amygdala [24]. Lesions of the basolateral amygdala induce similar impairments in processes of reward expectancy [25,26] and conditioned reinforcement [27,28] to that of OFC lesions, with reward expectancy having been shown to depend upon an interaction between the OFC and amygdala [29]. However, a recent disconnection study has revealed that orbitofrontal neurons still generate expectancies about outcomes after a basolateral amygdala lesion [24]. These expectancies appear to be devoid of information about the incentive value of the outcome associated with a particular CS, but they might reflect instead the sensory aspects of the outcome (see Box 2 for a discussion of the sensory aspects of conditioning, and [30] for a review) – information that, as proposed by Izquierdo et al. [23] could be obtained from other brain regions such as the perirhinal cortex, which itself has been www.sciencedirect.com

Vol.10 No.2 February 2006

implicated in stimulus–stimulus associations ([31], and see discussion in next section). Taken together these findings support the hypothesis that through the generation of expectancies, the OFC contributes to goal selection. Indeed, based on such findings, Montague and Berns [32] have developed a predictor-valuation model of decision-making in which the OFC integrates information relating to various rewards and punishments and their predictors, and despite their variable nature, which might include food, money, sex, produces a common neural currency on which decisions

(a)

Incentive activation: Amygdala

(b)

Incentive activation: Medial OFC

(c)

Choice activation: Medial OFC

(d) Incentive x choice activation: Lateral OFC

Figure 2. Incentive and choice in a restaurant task. (a) A region of the left amygdala (xZK16, yZK4, zZK14) and (b) a region of the left medial orbitofrontal cortex (xZK8, yZ44, zZK20) showed significantly increased rCBF in a high-incentive condition compared with a low-incentive condition in the restaurant menu task, the rCBF in the amygdala being positively correlated with subjective ratings of incentive value. (c) An area of left medial orbitofrontal cortex (xZK8, yZ36, zZ 16) also showed significantly greater activity in choice trials compared with nochoice trials, activity in this region correlating positively with subjective ratings of difficulty. (d) Finally, a region of lateral orbitofrontal cortex (xZ48, yZ52, zZK14) showed significantly increased activity for the incentive ! choice interaction, specifically when subjects selected between high-incentive alternatives. Data redrawn from [33].

Review

TRENDS in Cognitive Sciences

can be made as to desirable future outcomes. Consequently, it would be predicted that neural activations within the OFC, but not the amygdala, would be differentially activated in a condition in which a subject is presented with multiple incentive items and has to make an explicit expectancy-based preference judgement compared with a condition when the same items are presented but no decision has to be made. This is exactly the pattern that was generated when activity in the OFC and amygdala was compared in humans presented with a series of individually tailored high or low incentive restaurant menus and asked to either make a selection from the menu or just consider the menu items [33] (Figure 2). Discrimination reversal and extinction revisited Underlying learning mechanisms The inability of a CS to access representational information about the incentive value of associated reinforcers and guide behaviour after OFC damage is clearly one example of inflexible behaviour. As described earlier, other examples include the impairments in discrimination reversal and extinction that follow OFC damage. However, tests of reversal and extinction do not involve a change in incentive value. Instead there is an alteration in the contingent relationship between the CS and the outcome. Thus to understand the important contribution of the OFC to discrimination reversal and extinction it is necessary to re-consider the associative learning mechanisms that underlie performance following a change in reward contingencies. Both involve new learning such that a particular cue or signal (CS) acquires a second meaning that becomes available along with the first meaning. As discussed by Bouton [34] such cues are ambiguous and as such their current meaning is determined by the current context. Thus, whereas the associations formed during initial learning appear relatively independent of the context in which they were presented, the new learning that takes place during extinction (or reversal) is relatively contextspecific with the type of new associations formed being again dependent upon the particular task used. There are several important factors that will contribute to extinction or reversal of a response, whether it be Pavlovian or instrumental in nature (for reviews, see [35,36]). First, a negative affective response is likely to develop following the loss of reward and will function to produce various associative changes opposite to those induced by the original positive affective event; with the intensity of this negative event depending in part upon the intensity of the previously obtained reward (contrast effect). Evidence that loss of reward acts as an aversive event is supported by the finding that animals will learn a response to escape from reward loss and that neutral stimuli presented at the time of reward loss become themselves capable of promoting escape learning. Second, as there is no explicit sensory event associated with the loss of reward then any associations formed will be dominated by the responses generated by the animal itself in response to reward loss. Consequently, extinction can result in an inhibitory association forming specifically between the CS and the www.sciencedirect.com

Vol.10 No.2 February 2006

87

previously rewarded response; that is, the response most likely to be emitted early in extinction or reversal and which becomes associated with the aversive event (for a detailed review of this hypothesis, see [35]). Although an inhibitory association might also form between the CS and the previously rewarded outcome this hypothesis is inconsistent with the finding that an extinguished CS that is no longer able to provoke its original food-box approach response nevertheless can potentiate instrumental responding for the same rewarded outcome [35]. Comparison of OFC and medial PFC lesions Although a disruption of any of these mechanisms will lead to impaired reversal or extinction performance the pattern of impairment might well differ. For example, intact suppression of responding by non-reinforcement during the initial extinction session, but a loss of memory for that extinction after a delay, is characteristic of rats with lesions of the infralimbic area, a ventral region of medial PFC suggested to be homologous to area 25 in monkeys [18]. It has been proposed that this impairment, first described for Pavlovian conditioned fear [37] but recently replicated in an appetitive setting [38] is a consequence of an impairment in the contextual modulation of extinction learning [38]. Such an impairment could underlie the instrumental discrimination reversal deficit that has also been reported to occur following infralimbic lesions [17]. It is unlikely however that this same impairment contributes to the deficits in discrimination learning and extinction induced by lesions of OFC [4,9,39,40]. When information about ‘within’ and ‘between’ session performance has been provided, the deficit has been seen on the first day of extinction (in monkeys) [3] or reversal (in rats) [40]. Moreover, when the deficit has been described as perseverative, more often than not, the impairment arose early, rather than later, in reversal training [9]. Consequently, such orbitofrontal lesion-induced reversal deficits are more likely to result from either a specific loss of the ability to inhibit the previously rewarded response or a failure to use the new negative affective information to counter the previous positive affective information. Additionally, a loss of reward expectation that has been proposed to accompany orbitofrontal lesions or a loss of contingency information between the stimulus and sensory properties of the reward might also be expected to delay the effects of extinction. It is quite possible that several of these processes rely upon distinct sectors of the OFC, a proposal that could explain the differential effects of ablations of the medial sector of OFC and inferior prefrontal convexity (including lateral OFC) on discrimination reversal and extinction [3,4]. For example, a loss of inhibitory control or impaired processing of negative affect might be associated with damage to the lateral OFC [12,13], whereas impaired processing of reward expectation, resulting in reduced impact of non-reward, might contribute to the impairment induced by medial orbitofrontal ablations. Such a proposal would also be consistent with recent functional neuroimaging studies that have identified multiple regions of activation within OFC specifically linked to the reversal or extinction stage of a Pavlovian or instrumental discrimination [41–44] although the

Review

TRENDS in Cognitive Sciences

importance of response inhibition (even in a Pavlovian discrimination) has sometimes been overlooked [43]. Based on electrophysiological findings of slowed and inflexible responses to a CS in the basolateral nucleus of the amygdala following orbitofrontal lesions it has been proposed that following a reversal in an orbitofrontal lesioned animal, incomplete or erroneous incentive information from the amygdala affects other brain areas and thereby slows the rate at which old response patterns are abandoned in favor of new strategies [45]. This interaction between the OFC and the amygdala might parallel that proposed to occur between the infralimbic region and the amygdala in extinction of fear conditioning [37,46] except that presumably the latter interaction is specific to the contextual recall of extinction memory as opposed to the initial extinction process per se. Moreover, the involvement of the amygdala in reversal learning might well depend upon whether responding is under the control of associations between stimuli and the affective properties of reward or alternatively the sensory and informational properties of reward. If the latter, which could be the case in studies of object discrimination learning and reversal in rhesus monkeys [23] then it might be erroneous information coming from elsewhere in the brain, such as the perirhinal cortex (as a consequence of the loss of orbitofrontal modulation of that region), that slows the rate at which old response patterns are abandoned. (For a related proposal for the OFC involving the habituation of event related potentials in response to repeated presentations of aversive somatosensory stimuli, see [47]). Future directions Continued focus on the nature of the interactions between the OFC and related brain structures will be essential to our overall understanding of OFC functioning. These interactions should include not only those between the OFC and structures such as the amygdala, perirhinal cortex, hippocampus and striatum but also the peripheral arousal systems. Thus, changes in peripheral autonomic, somatic and endocrine activity are important for preparing the body to support ongoing affective behaviour, and feedback from such peripheral changes has been hypothesized to contribute to the perception of arousal, to engage appraisal processes including the self perception of emotional state (for reviews see [48–50]), to enhance emotional memories [51], to modulate attention [52] and to influence response selection and hence decision making ([5], but see [53] for counter-arguments). However, the relationship of peripheral arousal mechanisms with positive affective behaviour and their potential role in response selection has been relatively ignored in studies of orbitofrontal function in animals (but see [54] with respect to the amygdala and negative affect), despite this region providing a major focus of integration for visceral feedback. Recently, though, a primate model for investigating peripheral arousal has been developed that uses a telemetric device implanted into the decending aorta of behaving marmosets [55]. Using this model, increases in blood pressure and heart rate have been shown to accompany the anticipation and subsequent consumption of food reward, the expression of which is dependent upon www.sciencedirect.com

Vol.10 No.2 February 2006

the amygdala [55] but not OFC or medial PFC [56]. Future investigations of peripheral arousal could provide insight into the negative consequences of non-reward following reversal or extinction of reward contingencies and should facilitate investigations into the role of peripheral feedback in adaptive behaviour. Another interaction of crucial importance to our understanding of OFC function is that between the OFC and the ascending monoaminergic systems, dopamine, noradrenaline and serotonin. These systems have been linked with mood states, reward-related processing and stress and are also involved in mediating some of the central effects of feedback from peripheral arousal [51]. The interactive nature of the relationship between the OFC and the monoamines is highlighted by the fact that not only do these brainstem systems modulate OFC processing but the OFC, with the medial PFC, has a relatively privileged position amongst cortical areas in its regulation of these brainstem pathways. That depression and OCD, disorders associated with OFC dysregulation [57–59], are treated with selective serotonin re-uptake inhibitors and changes in OFC DA regulation have been identified in human drug abusers [60] are factors that have contributed to this relationship gaining more prominence. A comprehensive discussion of this topic is outside the scope of this review (for a review see [61]) but a recent finding, of particular relevance to the current discussion, is the selective contribution of serotonergic afferents within OFC to reversal learning. Thus, serotonergic (Figure 3), but not dopaminergic depletions, within the PFC produce perseverative responding on a visual discrimination reversal task [62–64]. By contrast, dopamine in the OFC has been implicated in response choice involving delayed rewards [65,66]. Such differential modulation of distinct aspects of reward processing within the OFC by these chemically selective systems, although clearly needing further study, is an important step forward in determining the specific relationship between these individual neurochemical systems and the OFC. Crucial to our understanding of this relationship is the determination of the role of the OFC in modulating these ascending brainstem pathways. Recent findings have implicated the medial PFC in the 15

Control **

Errors √x

88

10

Prefrontal 5-HT lesion *

5

0

Rev 1

Rev 2

Rev 3

Rev 4

TRENDS in Cognitive Sciences

Figure 3. Cognitive inflexibility after prefrontal serotonin depletion. Mean number of errors (C/K SEM) made by groups of control (nZ3) and 5,7, DHT induced prefrontal serotonin depleted (nZ3) subjects during the perseverative stage of a series of reversals of a pattern discrimination (Rev. 1–4). The serotonin depleted group made significantly more perseverative errors to the previously rewarded pattern than controls on reversals 2 and 3. (* and ** indicate that the lesioned group differed from controls with significance pZ0.026 and 0.001, respectively.) Data redrawn from [62].

Review

TRENDS in Cognitive Sciences

Box 3. Questions for future research † What are the specific contributions of the OFC to behavioural adaptation, in line with changing reward contingencies and how do the medial and lateral sectors of the OFC differ in this respect? † How does the OFC interact with related brain structures such as the amygdala and perirhinal cortex to alter behaviour in line with changing reward contingencies? † What is the relative contribution of the OFC and medial PFC to affective behaviour? † How does the modulation of OFC function by the different monoamines differ and, in turn, what is the basis of their modulation by the OFC? † Under what circumstances, if any, do the peripheral arousal mechanisms modulate OFC function and what is the nature of that modulation?

mechanism by which stressor controllability affects not only behaviour but also serotonergic activity in the dorsal raphe [67]. Whether the OFC plays a complementary role in determining, for example, how the predictability of reward and punishment by external cues affects activity in these ascending monoamine pathways remains to be addressed. Conclusion Although there is more to be learned (see Box 3), there is accruing evidence consistent with an involvement of the primate OFC in integrating information about the desirability of expected outcomes based on interoceptive and exteroceptive cues and the use of this information to select goals for action. Access to the current incentive value of outcomes, along with the ability to detect changes in contingencies between stimuli in the environment are central to the OFC’s role in behavioural adaptation. This role is crucially dependent upon the ability of the OFC to control processing, not only within the amygdala and other temporal lobe structures such as the perirhinal cortex, but also the brainstem monoamine systems that have widespread influence over learning and attention. Acknowledgements I am very grateful to Professors B.J. Everitt and T.W. Robbins for their helpful comments on an earlier draft of this manuscript.

References 1 Carmichael, S.T. and Price, J.L. (1994) Architectonic subdivision of the orbital and medial prefrontal cortex in the macaque monkey. J. Comp. Neurol. 346, 366–402 2 Price, J.L. et al. (1996) Networks related to the orbital and medial prefrontal cortex; a substrate for emotional behavior? Prog. Brain Res. 107, 523–536 3 Butter, C.M. (1969) Perseveration in extinction and in discrimination reversal tasks following selective frontal ablations in macaca mulatta. Physiol. Behav. 4, 163–171 4 Iversen, S.D. and Mishkin, M. (1970) Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity. Exp. Brain Res. 11, 376–386 5 Damasio, A.R. (1998) The somatic marker hypothesis and the possible functions of the prefrontal cortex. In Prefrontal Cortex: Cognitive and Executive Functions (Roberts, A.C. et al., eds), pp. 27–50, Oxford University Press 6 Bechara, A. et al. (1999) Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J. Neurosci. 19, 5473–5481 www.sciencedirect.com

Vol.10 No.2 February 2006

89

7 Fuster, J.M. (1989) The Prefrontal Cortex: Anatomy, Physiology and Neuropsychology of the Frontal Lobe, 2nd edn, Raven Press 8 Mishkin, M. (1964) Perseveration of central sets after frontal lesions in man. In The Frontal Granular Cortex and Behavior (Warren, J.M. and Akert, K., eds), pp. 219–294, McGraw-Hill 9 Dias, R. et al. (1996) Dissociation in the prefrontal cortex of affective and attentional shifting. Nature 380, 69–72 10 Rolls, E. (1998) Neurophysiology and functions of the primate amygdala and the neural basis of emotion. In Prefrontal Cortex: Cognitive and Executive Functions (Roberts, A.C. et al., eds), pp. 67–86, Oxford University Press 11 Roesch, M.R. and Olson, C.R. (2004) Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307–310 12 Kringelbach, M.L. (2004) The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog. Neurobiol. 72, 341–372 13 Elliott, R. and Deakin, B. (2005) Role of the orbitofrontal cortex in reinforcement processing and inhibitory control: evidence from functional magnetic resonance imaging studies in healthy human subjects. Int. Rev. Neurobiol. 65, 89–116 14 O’Doherty et al. (2000) Sensory-specific satiety related olfactory activation of the human orbitofrontal cortex. Neuroreport 11, 893–897 15 Tremblay, L. and Schultz, W. (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398, 704–708 16 Gallagher, M. et al. (1999) Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610–6614 17 Chudasama, Y. and Robbins, T.W. (2003) Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J. Neurosci. 23, 8771–8780 18 Ongur, D. and Price, J.L. (2000) The organisation of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb. Cortex 10, 206–219 19 McDannald, M.A. et al. (2005) Lesions of orbitofrontal cortex impair rats differential outcome expectancy learning but not conditioned stimulus-potentiated feeding. J. Neurosci. 25, 4626–4632 20 Pears, A. et al. (2003) Lesions of the orbitofrontal, but not medial prefrontal cortex, disrupt conditioned reinforcement in primates. J. Neurosci. 23, 11189–11201 21 Everitt, B.J. and Robbins, T.W. (2000) Second-order schedules of drug reinforcement in rats and monkeys: measurement of reinforcing efficacy and drug seeking behaviour. Psychopharmacology (Berl.) 153, 17–30 22 Parkinson, J.A. et al. (2005) Acquisition of instrumental conditioned reinforcement is resistant to the devaluation of the unconditioned stimulus. Q. J. Exp. Psychol B. 58, 19–30 23 Izquierdo, A. et al. (2004) Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24, 7540–7548 24 Schoenbaum, G. et al. (2003) Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855–867 25 Hatfield, T. et al. (1996) Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256–5265 26 Malkova, L. et al. (1997) Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. J. Neurosci. 17, 6011–6020 27 Parkinson, J.A. et al. (2001) The role of the primate amygdala in conditioned reinforcement. J. Neurosci. 21, 7770–7780 28 Everitt, B.J. et al. (2003) Appetitive behavior: impact of amygdaladependent mechanisms of emotional learning. Ann. N. Y. Acad. Sci. 985, 233–250 29 Baxter, M.G. et al. (2000) Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. J. Neurosci. 20, 4311–4319 30 Roesch, M. and Schoenbaum, G. From associations to expectancies: orbitofrontal cortex as gateway between the limbic system and

Review

90

31

32 33

34 35

36

37 38

39

40

41

42

43

44

45

46

47 48 49 50 51

52 53

54

TRENDS in Cognitive Sciences

representational memory. In The Orbitofrontal Cortex (Zald, D and Rauch, S., eds), Oxford University Press (in press) Murray, E.A. and Wise, S.P. (2004) What, if anything, is the medial temporal lobe, and how can the amygdala be part of it if there is no such thing? Neurobiol. Learn. Mem. 82, 178–198 Montague, P.R. and Berns, G.S. (2002) Neural economics and the biological substrates of valuation. Neuron 36, 265–284 Arana, F.S. et al. (2003) Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J. Neurosci. 23, 9632–9638 Bouton, M. (2004) Context and behavioral processes in extinction. Learn. Mem. 11, 485–494 Rescorla, R.A. (2001) Experimental extinction. In Handbook of Contemporary Learning Theories (Mowrer, R.R. and Klein, S.B., eds), pp. 119–154, Erlbaum Delamater, A.R. (2004) Experimental extinction in Pavlovian conditioning: behavioural and neuroscience perspectives. Q. J. Exp. Psychol B. 57, 97–132 Quirk, G.J. and Gehlert, D.R. (2003) Inhibition of the amygdala: key to pathological states. Ann. N. Y. Acad. Sci. 985, 263–272 Rhodes, S.E. and Killcross, S. (2004) Lesions of rat infralimbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. Learn. Mem. 11, 611–616 McAlonan, K. and Brown, V.J. (2003) Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behav. Brain Res. 146, 97–103 Schoenbaum, G. et al. (2003) Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 10, 129–140 Cools, R. et al. (2002) Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci. 22, 4563–4567 Kringelbach, M.L. and Rolls, E.T. (2003) Neural correlates of rapid reversal learning in a simple model of human social interaction. Neuroimage 20, 1371–1383 O’Doherty, J.P. et al. (2003) Dissociating valence of outcome from behavioral control in orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931–7939 Gottfried, J.A. and Dolan, R.J. (2004) Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value. Nat. Neurosci. 7, 1144–1152 Saddoris, M.P. et al. (2005) Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46, 321–331 Sotres-Bayon, F. et al. (2004) Emotional perseveration: an update on prefrontal-amygdala interactions in fear extinction. Learn. Mem. 11, 525–535 Rule, R.R. et al. (2002) Orbitofrontal cortex and dynamic filtering of emotional stimuli. Cogn. Affect. Behav. Neurosci. 2, 264–270 Craig, A.D. (2002) How do you feel? Interoception: the sense of the physiological condition of the body. Nat. Rev. Neurosci. 3, 655–662 Levenson, R.W. (2003) Blood, sweat and fears. The autonomic architecture of emotion. Ann. N. Y. Acad. Sci. 1000, 348–366 Dolan, R.J. (2002) Emotion, cognition and behavior. Science 298, 1191–1194 McGaugh, J.L. (2004) The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annu. Rev. Neurosci. 27, 1–28 Berntson, G.G. et al. (2003) Ascending visceral regulation of cortical affective information processing. Eur. J. Neurosci. 18, 2103–2109 North, N.T. and O’Carroll, R.E. (2001) Decision making in patients with spinal cord damage: afferent feedback and the somatic marker hypothesis. Neuropsychologia 39, 521–524 Davis, M. (2000) The role of the amygdala in conditioned and unconditioned fear and anxiety. In The Amygdala: Neurobiological

www.sciencedirect.com

Vol.10 No.2 February 2006

55

56

57

58

59

60

61

62 63

64

65

66

67

68

69 70

71 72 73

74 75

Aspects of Emotion, Memory and Mental Dysfunction (Aggleton, J.P., ed.), pp. 255–306, Wiley-Liss Braesicke, K. et al. (2005) Autonomic arousal in an appetitive context in primates: a behavioural and neural analysis. Eur. J. Neurosci. 21, 1733–1740 Reekie, Y. et al. (2004) Autonomic and behavioural arousal in appetitive pavlovian conditioning in monkeys: effects of amygdala and orbitofrontal lesions. Soc. Neurosci. Abstr. Program No. 84.7, Abstract viewer/itinerary planner Drevets, W.C. (2001) Neuroimaging and neuropathological studies of depression: implications for the cognitive-emotional features of mood disorders. Curr. Opin. Neurobiol. 11, 240–249 Mayberg, H.S. (2003) Modulating dysfunctional limbic-cortical circuits in depression: towards development of brain-based algorithms for diagnosis and optimised treatment. Br. Med. Bull. 65, 193–207 Saxena, S. and Rauch, S.L. (2000) Functional neuroimaging and the neuroanatomy of obsessive-compulsive neurosis. Psychiatr. Clin. North Am. 23, 563–586 Volkow, N.D. and Fowler, J.S. (2000) Addiction, a disease of compulsion and drive: involvement of the orbitofrontal cortex. Cereb. Cortex 10, 318–325 Robbins, T.W. et al. Neurochemical modulation of orbitofrontal cortex. In The Orbitofrontal Cortex (Zald, D and Rauch, S, eds), Oxford University Press (in press) Clarke, H.F. et al. (2004) Cognitive inflexibility after prefrontal serotonin depletion. Science 304, 878–880 Roberts, A.C. et al. (1994) 6-hydroxydopamine lesions of the prefrontal cortex in monkeys enhance performance on an analogue of the Wisconsin Card Sort test: possible interactions with subcortical dopamine. J. Neurosci. 14, 2531–2544 Clarke, H.F. et al. (2004) A neuropsychological and neurochemical investigation into the cognitive inflexibility induced by prefrontal serotonin depletion. Soc. Neurosci. Abstr. Program No. 87.16 Abstract viewer/itinerary planner Kheramin, S. et al. (2004) Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology (Berl.) 175, 206–214 Winstanley, C.A. et al. (2005) Double dissociation between serotonergic and dopaminergic modulation of medial prefrontal and orbitofrontal cortex during a test of impulsive choice. Cereb. Cortex doi: 10. 1093/cercor/bhi088 (epub ahead of print) Amat, J. et al. (2005) Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus. Nat. Neurosci. 8, 365–371 Petrides, M. and Pandya, D.N. (1994) Comparative cytoarchitectonic analysis of the human and the macaque frontal cortex. In Handbook of Neuropsychology, Vol 9 (Boller, F and Grafman, J., eds), pp. 17–58, Elsevier Hearst, E. and Jenkins, H.M. (1974) Sign Tracking: The Stimulus– Reinforcer Relation and Directed Action, The Psychonomic Society Holland, P.C. and Straub, J.J. (1977) Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. J. Exp. Psychol Anim Behav Process. 5, 65–78 Mackintosh, N.J. (1974) The Psychology of Animal Learning, Academic Press Williams, B.A. (1994) Conditioned reinforcement: neglect or outmoded explanatory construct? Psych. Bull. Rev 1, 457–475 Medin, D.L. (1977) Information processing and discrimination learning set. In Behavioral Primatology (Schrier, A.M., ed.), pp. 33–69, Erlbaum Gaffan, D. (1979) Acquisition and forgetting in monkeys’ memory of informational object-reward associations. Learn. Motiv. 10, 419–444 Dickinson, A. and Balleine, B. (1994) Motivational control of goaldirected action. Anim. Learn. Behav. 22, 1–18