- Email: [email protected]
Decision making and risk attitude of the common marmoset in a gambling task
Neuroscience Research 71 (2011) 260–265
Contents lists available at ScienceDirect
Neuroscience Research journal homepage: www.elsevier.com/locate/neures
Decision making and risk attitude of the common marmoset in a gambling task Hironobu Tokuno ∗ , Ikuko Tanaka Laboratory of Brain Structure, Tokyo Metropolitan Institute of Medical Science, Kamikitazawa, Setagaya, Tokyo 156-8506, Japan
a r t i c l e
i n f o
Article history: Received 5 June 2011 Received in revised form 12 July 2011 Accepted 13 July 2011 Available online 23 July 2011 Keywords: Decision making Gambling task Risk Reward Primate Common marmoset Neuroeconomics
a b s t r a c t To analyze decision making under uncertainty of monkeys, common marmosets were trained to choose and remove one of two colored caps on wells arranged side by side. Each well contained constant reward (3 grains of puffed rice) or risky reward (0 or 6 grains; probability, 50%:50%). For each marmoset, white or black color was assigned randomly as a symbol of non-risky or risky choice. Arrangement of white and black caps was determined randomly in each trial. After 200 trials (5 trials per day), the marmosets were classified according to the pattern of their choice. Eight of 18 marmosets (44.4%) were risk-aversive, whereas 5 marmosets (27.8%) were risk-prone. The remaining 5 marmosets (27.8%) preferred to choose one side (left n = 4, right n = 1). These results showed individual differences in decision making of marmosets. An additional task with reduction in the expected value of the preferred choice revealed that risk-aversive marmosets were slower to adjust their choices to such reductions than risk-prone animals. © 2011 Elsevier Ireland Ltd and the Japan Neuroscience Society. All rights reserved.
1. Introduction In recent years, much interest has been paid to ‘neuroeconomics’, an approach to understand behavior and decision making based on unified economical, psychological and neuroscientific frameworks. An important problem in this emerging field (Knutson and Bossaerts, 2007; Doya, 2008) is whether animals, including humans, avoid or seek risk under various circumstances (Behar, 1961; Hill and Riopelle, 1985; Kahneman and Tversky, 2000; Marsh and Kacelnik, 2002; McCoy and Platt, 2005; Hayden and Platt, 2007), since this type of decision making is closely related to economical activities like gambling, speculation and investment. Thus, evaluation of utility, value and probability by the brain has been discussed repeatedly (for review, see Glimcher, 2003; Glimcher and Rustichini, 2004; Kable and Glimcher, 2009). During such exploration, McCoy and Platt (2005) have demonstrated risk-related neuronal activities in the posterior cingulate cortex of macaque monkeys. In addition, they have described that both of 2 monkeys used in their experiments showed riskprone behavior. However, utilization of 2 monkeys seems to be not enough to generalize preference for risky choice in monkeys. Since there has been increasing interest in the common marmoset (Callithrix jacchus, a species of New World monkeys) as an alternative primate species in neuroscience in recent years, we have established a breeding colony of the marmoset in our lab-
∗ Corresponding author. E-mail address: [email protected] (H. Tokuno).
oratory and have been using it for neuroscientific experiments (Tokuno et al., 2009, 2011; Paxinos et al., 2011). The common marmosets are small primates which can be easily maintained on middle to large scale (approximately 50–500 marmosets) in laboratory because of their availability, costs, size, and ease of breeding in laboratory conditions. In the present study, we examined decision making in the gambling task using 18 common marmosets as a basis for future neuroeconomical studies in the marmoset. We additionally tested the effect of reward reduction in the preferred choice on the pattern of decision making of the marmosets.
2. Materials and methods 2.1. Subjects Experiments were performed in 10 male and 8 female young to adult common marmosets (C. jacchus) weighing 250–510 g. The age of the animals ranged from 9 months to 6 years 2 months (mean, 2 years 3 months) at the start of the training. These marmosets were derived from a breeding colony at Tokyo Metropolitan Institute for Neuroscience (now Tokyo Metropolitan Institute of Medical Science). The use of the marmosets in the present study followed the Guidelines approved by the Animal Experiment Committee at Tokyo Metropolitan Institute for Neuroscience. A single or a pair of marmosets was housed in home cages. The light cycle was 12:12 h with light starting at 07.00 h, and room temperature was kept at 28 ◦ C. Water was available ad lib.
0168-0102/$ – see front matter © 2011 Elsevier Ireland Ltd and the Japan Neuroscience Society. All rights reserved. doi:10.1016/j.neures.2011.07.1822
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
261
limited to 3 min in the preparation period and early 2 w. However, we observed almost no violation of this rule after the preparation period. Daily session for one marmoset usually took less than 30 s, because each trial took approximately 5 s. We observed that the marmosets became relaxed in the testing cage after the preparation period. After the gambling task during 8 w, the marmosets were finally classified according to the pattern of their choice.
Fig. 1. Photomicrographs showing the testing apparatus used in the present study. Black and white caps were used for the gambling task (a). The marmoset can get grains of sugar-coated puffed rice in the well after removal of the cap (a ). White caps marked with an open circle (left) or a cross (right) were used in the simple visual discrimination task (additional task 2) (b).
2.2. Gambling task Each marmoset was trained to move into an isolated testing cage for the task, and remove one of two colored caps on two horizontally arranged wells attached to a gray testing board in front of them (Fig. 1a). Each well contained constant reward (3 grains of sugar-coated puffed rice) or risky reward (0 or 6 grains; probability, 50%:50%). We chose the reward based on preliminary observation of food preference of marmosets, where all animals rapidly learned the taste of the sugar-coated puffed rice and were ready to take it. Marmosets could easily get grains from the well with its hand after removal of the cap (Fig. 1a ). The mean weight of one grain of the sugar-coated rice was 25.7 mg (n = 100; S.D., 5.1 mg). For each marmoset, white (W) or black (B) color was assigned randomly as a symbol of non-risky or risky choice. Arrangement of white and black caps was determined randomly in each trial. Since it is known that the marmoset shows a sex-linked polymorphism of red–green color vision (Travis et al., 1988; Pessoa et al., 2005), we used only monochromatic symbols throughout the present experiments (Fig. 1). The testing cage, the testing board and the arrangement of wells were designed symmetrically to avoid left–right bias in the present experiments. Each marmoset was trained to perform 5 trials per day. Thereby, the maximum possible number of reward grains per day was limited to 30, since we were afraid that the marmosets lost interest in excessive reward grains. We further took care of the marmosets from nutritional aspects. The tasks were carried out before daily feeding at 15.00 h. Body weight of each marmoset was measured once a week (w) and recorded. After a preparation period of 5 days for teaching the marmoset to remove the cap and take the reward grains, each marmoset was tested for 200 trials (25 trials per week) during 8 w. The time for one trial was recorded and
2.2.1. Addionnal task 1 – effect of reward reduction Additional tasks were carried out after the gambling task. The effect of gradual reduction of reward on decision making was tested in risk-aversive and risk-prone marmosets (additional task 1). These tasks were planned to examine the capability of the marmosets to alter their risk preference according to the change of expected value (EV) of the reward grain number. For risk-aversive marmosets (n = 8), the number of the reward for non-risky choice were reduced during 4 blocks as 3, 2, 1 and a half of grain (Fig. 2, left column). Each block consisted of 150 trials (6 w). For riskprone marmosets (n = 4), the probability of acquiring the reward was changed to reduce the EV of the reward grains if the marmosets continued to prefer the risky choice. The probability of acquiring 6 grains was decreased as follows: 50% (1/2), 33% (1/3), 17% (1/6), and 8% (1/12). Thus, the EV was reduced from 3 to 0.5 during 4 task blocks (Fig. 2, middle column). The additional task 1 was designed to reduce the EV equivalently among marmoset groups (Fig. 2, left and middle column). In each experiments, we did not make preliminary or explicit instruction about the reduction of the reward. The marmosets had to know the change of reward amount by examining their earnings. 2.2.2. Additional task 2 – simple visual discrimination task Furthermore, we trained marmosets which showed side preference (L-type and R-type, see Section 3 for detail) to perform a simple visual discrimination task using the same testing apparatus with two differently marked caps. In this additional task, we used new marks to instruct marmosets on the beginning of a new task. One of the two kinds of marks, i.e. an open circle and a cross, was drawn on the top of the white cap with black ink (Fig. 1b). One mark indicated correct choice (containing 3 grains of sugar-coated puffed rice in the well), while the other mark indicated incorrect choice (0 grain). The number of the reward grains for the correct choice was reduced from 3 to 0.5 during 4 task blocks as shown in Fig. 2 (right column). 2.3. Data analysis The obtained data were analyzed using R (programming language) for statistical computing (R Development Core Team, 2008). This free software was used for cluster analysis (Euclidean distance and complete linkage clustering method) and multiple comparisons method (Tukey’s HSD [honestly significant difference] test.) in the present study. 3. Results 3.1. Gambling task Fig. 3a summarizes the result of 200 trials in 18 marmosets. Each marmoset was plotted according to the percentage of their risky choice (vertical axis) and the percentage of the side of the cap removed by them (horizontal axis). We could classify the marmosets into 3 types (Fig. 3a and b) with the aid of cluster analysis (Euclidean distance and complete linkage clustering method; Fig. 3b). Eight of 18 marmosets (44.4%) were risk-aversive, while 5 marmosets (27.8%) were risk-prone. The remaining 5 marmosets
262
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
Fig. 2. The conditions and sequence in the additional tasks 1 and 2 following the gambling task. They were designed to examine the response to the reduction of reward grains. The expected value (EV) of reward grain number for the preferred choice was gradually reduced during 4 blocks as 3, 2, 1 and 0.5. Four blocks were sequentially carried out in each marmoset. The marmosets were tested for 150 trials during 6 weeks (w) in one block. Different parameters among task blocks are indicated by bold letters.
(27.8%) showed a tendency to prefer the left or right side, and were subdivided into 2 types (L-type, n = 4, [22.2%]; R-type, n = 1, [5.6%]). We found neither sex (Fig. 3a) nor age differences in the present study. Table 1 shows that we could not find significant age effect on the risk attitude of the marmosets. Typical examples of 25 trials in the last week of the task are demonstrated in Fig. 3c, which shows the result of every trial, i.e. the color of the chosen cap, the side of the chosen cap and the number of reward grains. The mean percentage of the risky choice was 18.1% (n = 8; S.D., 4.5%) and 71.4 (n = 5; S.D., 7.6%) in the risk-aversive and risk-prone marmosets, respectively. As seen in Fig. 3a and c, the risk-aversive marmosets showed a more persistent tendency to their favorite choice than the risk-prone marmosets. We calculated the mean percentage of the non-risky choice in the risk-aversive marmosets (81.9%; n = 8; S.D., 4.5%) and compared with that of risky choice in the risk-prone marmosets. The mean percentage was significantly different among these two groups (Student’s t-test, p = 0.0095). 3.1.1. Additional task 1 All 8 risk-aversive marmosets and 4 of the 5 risk-prone marmosets were used for additional task 1. The result of the additional task is summarized in Fig. 4. Both risk-prone and risk-aversive marmosets showed a change in their choice during the reduction of the
Table 1 Number of the marmosets showing the various risk attitude during the gambling task classified based on their age. Extended Fisher’s exact test revealed that there is no significant age effect on their risk attitude (p = 0.959).
0–1 years old 1–3 years old 3–6 years old
Risk-aversive
Risk-prone
L-type, R-type
2 3 3
1 2 2
1 3 1
8
5
5
reward to obtain more reward grains. However, the timing of the change was different among groups. The mean ratio of the risky choice increased in Block 4 of the risk-aversive marmosets. The risk-prone marmosets showed a gradual change during Blocks 2–4. Table 2 summarizes the timing of the change in each marmoset. This table also suggests the time difference of the change among groups. Five (62.5%) of the 8 risk-aversive marmosets changed the ratio of their favorite choice significantly in Block 4. Fig. 5 exemplifies the daily change of the total number of the reward grains in each day (5 trials) during the 4 task blocks in a risk-aversive marmoset (Fig. 5a) and a risk-prone one (Fig. 5b). Riskprone animals showed stronger vertical fluctuations in the number of grains per day and deviated less from the EV of 15 grains/day than risk-aversive animals. Thus, risk-prone animals were better and earlier able to adjust their choices to the changed outcomes (see also Table 2). The risk-aversive marmoset showed week vertical fluctuation during Blocks 1–3, where total grains earned were decreased block by block due to their non-risky choice. The vertical fluctuation was manifest in Block 4 (Fig. 5a), indicating that the marmoset started to prefer the risky choice. On
Table 2 Number of the marmosets showing the first significant change (Tukey’s HSD [honestly significant difference] test, p < 0.05) of their choice in each block of the task. The experimental design is shown in Fig. 2. Additional task 1
Block 1 Block 2 Block 3 Block 4 No change
Additional task 2
Risk-aversive
Risk-prone
L-type, R-type
0 0 2 5 1
0 2 1 1 0
0 2 1 0 1
8
4
4
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
263
Fig. 3. Result of 200 trials (5 trials per day, 25 trials per week, 8 w) in the gambling task. (a) Each marmoset was plotted according to the percentage of their risky choice (vertical axis) and the percentage of the side of the cap (horizontal axis). (a ) The marmosets were classified into 3 groups showing risk-aversive, risk-prone and side preference behavior that was further subclassified into L-type and R-type. The X–Y plot is the same one as in Fig. 1a. (b) Result of the cluster analysis based on the dataset shown in a. We used the Euclidean distance and complete linkage clustering method. The cluster analysis supported the present classification. (c) Samples showing results of 25 trials in the last week of the gambling task in a risk-aversive (top), risk-prone (middle) or L-type (bottom) marmoset. The color of the chosen cap (W, white; B, black), the side of the chosen cap (L, left; R, right) and the number of reward grains in every trial were arranged sequentially from left to right.
the contrary, the risk-prone marmoset showed gradual decrease of vertical fluctuation during Blocks 1–3 and almost no vertical fluctuation in the later half of Block 3 and Block 4 (Fig. 5b). We further analyzed the relationship between the individual degree of risk-proneness and the timing of their change of choice in each marmoset. However, this statistical analysis did not provide significant results.
3.1.2. Additional task 2 Four of 5 L-type and R-type marmosets were tested using simple visual discrimination task (additional task 2). They showed early change in their decision making to obtain reward by choosing the correct cap in Block 2 (Fig. 4, right; Table 2). The ratio of the correct choice was as high as about 90% in Blocks 3 and 4 (Fig. 4, right). Finally, it should be noted here that no marmoset dropped out of the present series of experiments in spite of long time course of the task schedule (total 33 w). Two marmosets that were not
examined in additional tasks were used for other experiments (not shown) after the gambling task. 4. Discussion The present experiments disclosed the individual difference in decision making under risk in 18 marmosets. The present experiments further showed that 8 of 18 marmosets were risk-aversive. Previous studies showed risk-aversive behavior of the macaque monkeys, by demonstrating that they preferred the choice with the more frequent but lesser amount of reward (Behar, 1961; Hill and Riopelle, 1985). On the contrary, a recent study revealed riskprone behavior of the macaque monkeys (McCoy and Platt, 2005). Following experiments in the same macaque monkeys, they further showed that the mean ratio of the risky choice was about 50% after prolongation of the delay time between trials to 60–90 s (Hayden and Platt, 2007). These data suggest that the macaque monkeys also have individual difference in decision making under risk as the
264
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
Fig. 4. Change in percentage of risky choice (left and middle, additional task 1) or correct choice (right, additional task 2). The data were analyzed statistically with the multiple comparisons method based on Tukey’s HSD (honestly significant difference) test.
marmosets. In addition, it is reasonable to consider that the degree of individual risk-aversion (risk-proneness) will change according to the experimental conditions. The present additional tasks revealed that the marmosets can change their selection by evaluating the reward amount. The neural basis for the risk attitude of marmosets is unclear from the present behavioral experiments. However, a recent study by Tobler et al. (2009) has revealed probable human brain regions related to risk attitude. It was reported that risk enhanced reward-related activities of the lateral regions of the prefrontal cortex in risk-prone participants. In contrast, risk reduced such activities in risk-aversive participants. These findings lead us to consider that such activities of prefrontal cortex may play a role in changing choices in the additional task. Since the marmosets are primates and have well-developed prefrontal cortical areas in comparison with rodents (Paxinos et al., 2011), future studies using marmosets would be helpful to analyze neuronal mechanisms in the prefrontal cortex underlying the risk attitude with the aid of invasive experimental techniques including electrophysiology. The third group of marmosets showed different behavior. These L-type and R-type marmosets showed side preferences. To interpret the decision making of these marmosets, the cost for decision making and related process in the brain should be considered. The marmosets showing risk-aversive or risk-prone behavior are required to (1) observe the color of caps, (2) decide the cap to take based on their memory about the symbol, risk, expected reward and preference to risk, (3) execute the motor program for the control of their forelimb in the necessary side to remove caps and take grains, and finally (4) evaluate and remember the results. All of the processes necessitates the work of multiple brain areas related to visual, decision making, motor, memory and reward evaluation system. Since the EV of the number of reward grains did not depend
on selection strategy of the marmosets under the present experimental design, the L-type and R-type marmosets can omit all of the brain process by taking the cap of the single side constantly. As the effort for each trial was much reduced, the behavior of these animals can be interpreted as energy saving. Thus, this type of the marmoset was the most superior in cost performance in the present gambling task. The additional task 2 provided further evidence supporting this scenario. The L-type and R-type marmosets were able to perform simple visual discrimination task (additional task 2). They showed similar side preference in Block 1 following the gambling task. However, they showed slight elevation of the ratio of the correct choice (62.5%) in Block 1. The ratio of the correct choice was finally increased to about 90% in Blocks 3 and 4 (Fig. 4). This fact strongly suggests that the L-type and R-type marmosets can observe, choose and take the correct cap on each side, if necessary. Thus, it is reasonable to consider that these marmosets took the cap of the specific side intentionally in the gambling task. Risk attitude in human behavior is usually categorized into three types: risk-aversive, risk-prone and risk-neutral (for recent review, see Engelmann and Tamir, 2009). The L-type and R-type marmosets observed in the present experiments can be interpreted as riskneutral, because they did not show risk aversion or risk preference in the gambling task. Analysis of information processing in the brain of risk-neutral animals will be necessary to fully understand the risk attitude of animals including humans. In spite of the equivalent EV between non-risky and risky choice except for the uncertainty, the degree of preference to the non-risky or risky choice was different between the risk-aversive and riskprone marmosets behaviorally. The number of the risk-aversive marmosets (n = 8) was larger than that of the risk-prone marmosets (n = 5). The mean percentage (81.9%) of non-risky choice in the risk-aversive marmosets was significantly larger than that (71.4%)
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
265
making, and what kind of hierarchy and chemical modulation exists between these areas. To further elucidate the neural circuitry and neurochemical mechanism underlying the decision making under risk, invasive experimental studies using animals will be required. For example, effect of partial destruction of brain areas or drug application into specific brain areas on decision making should be examined in future studies. Acknowledgements We thank M. Inugami and M. Saitoh in the Animal Center of the Tokyo Metropolitan Institute of Medical Science for their help. References
Fig. 5. Samples of daily change in total number of reward grains earned on each day (5 trials) during the 4 task blocks in a risk-aversive marmoset (a) and a risk-prone one (b). Risk-prone animals showed stronger vertical fluctuations in the number of grains per day and deviated less from the expected value (EV) of 15 grains/day than risk-aversive animals. Risk-prone animals were better and earlier able to adjust their choices to the changed outcomes.
of risky choice in the risk-prone marmosets (see Section 3.1). In additional tasks, the mean percentage of the risky choice in the risk-aversive marmosets did not change until Block 4, while the risk-prone marmosets showed a gradual change of choice during Blocks 2–4. These findings indicate, on the whole, that the degree of avoiding risk is relatively larger than that of seeking risk in the marmoset. Taking into account the present results and natural behavior of wild animals, it is possible that small animals such as the marmoset are more risk sensitive than large animals. Recent brain imaging studies in humans have shown that the insular cortex, lateral orbitofrontal cortex, ventral striatum and midbrain were activated in relation to the risk (Dreher et al., 2006; Preuschoff et al., 2006; Lin et al., 2008). Nevertheless, it is still unclear how these areas cooperate during the process of decision
Behar, I., 1961. Learned avoidance of non-reward. Psychol. Rep. 9, 43–52. Doya, K., 2008. Modulators of decision making. Nat. Neurosci. 11, 410–416. Dreher, J.C., Kohn, P., Berman, K.F., 2006. Neural coding of distinct statistical properties of reward information in humans. Cereb. Cortex 16, 561–573. Engelmann, J.B., Tamir, D., 2009. Individual differences in risk preference predict neural responses during financial decision-making. Brain Res. 1290, 28–51. Glimcher, P.W., 2003. Decisions, Uncertainty, and the Brain. The Science of Neuroeconomics. The MIT Press, Cambridge, MA. Glimcher, P.W., Rustichini, A., 2004. Neuroeconomics: the consilience of brain and decision. Science 306, 447–452. Hayden, B.Y., Platt, M.L., 2007. Temporal discounting predicts risk sensitivity in rhesus macaques. Curr. Biol. 17, 49–53. Hill, C.W., Riopelle, A.J., 1985. Probability-reward preferences of rhesus monkeys. Am. J. Psychol. 98, 77–84. Kable, J.W., Glimcher, P.W., 2009. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745. Kahneman, D., Tversky, A., 2000. Choices, Values, and Frames. Cambridge University Press, Cambridge. Knutson, B., Bossaerts, P., 2007. Neural antecedents of financial decisions. J. Neurosci. 27, 8174–8177. Lin, C.-H., Chiu, Y.-C., Cheng, C.-M., Hsieh, J.-C., 2008. Brain maps pf Iowa gambling task. BMC Neurosci. 8, 72. Marsh, B., Kacelnik, A., 2002. Framing effects and risky decisions in starlings. Proc. Natl. Acad. Sci. U. S. A. 99, 3352–3355. McCoy, A.N., Platt, M.L., 2005. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8, 1220–1227. Paxinos, G., Watson, C., Petrides, M., Rosa, M., Tokuno, H., 2011. The Marmoset Brain in Stereotaxic Coordinates. Academic Press, San Diego. Preuschoff, K., Bossaerts, P., Quartz, S.R., 2006. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51, 381–390. Pessoa, D.M., Tomaz, C., Pessoa, V.F., 2005. Color vision in marmosets and tamarins: behavioral evidence. Am. J. Primatol. 67, 487–495. R Development Core Team, 2008. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org. Travis, D.S., Bowmaker, J.K., Mollon, J.D., 1988. Polymorphism of visual pigments in a callitrichid monkey. Vision Res. 28, 481–490. Tobler, P.N., Christopoulos, G.I., O’Doherty, J.P., Dolan, R.J., Schultz, W., 2009. Riskdependent reward value signal in human prefrontal cortex. Proc. Natl. Acad. Sci. U. S. A. 106, 7185–7190. Tokuno, H., Tanaka, I., Umitsu, Y., Akazawa, T., Nakamura, Y., 2009. Webaccessibledigital brain atlas of the common marmoset (Callithrix jacchus). Neurosci. Res. 64, 128–131. Tokuno, H., Tanaka, I., Senoo, A., Umitsu, Y., Akazawa, T., Nakamura, Y., Watson, C., 2011. Internet-based atlas of the primate spinal cord. Neurosci. Res. 70, 128–132.
Contents lists available at ScienceDirect
Neuroscience Research journal homepage: www.elsevier.com/locate/neures
Decision making and risk attitude of the common marmoset in a gambling task Hironobu Tokuno ∗ , Ikuko Tanaka Laboratory of Brain Structure, Tokyo Metropolitan Institute of Medical Science, Kamikitazawa, Setagaya, Tokyo 156-8506, Japan
a r t i c l e
i n f o
Article history: Received 5 June 2011 Received in revised form 12 July 2011 Accepted 13 July 2011 Available online 23 July 2011 Keywords: Decision making Gambling task Risk Reward Primate Common marmoset Neuroeconomics
a b s t r a c t To analyze decision making under uncertainty of monkeys, common marmosets were trained to choose and remove one of two colored caps on wells arranged side by side. Each well contained constant reward (3 grains of puffed rice) or risky reward (0 or 6 grains; probability, 50%:50%). For each marmoset, white or black color was assigned randomly as a symbol of non-risky or risky choice. Arrangement of white and black caps was determined randomly in each trial. After 200 trials (5 trials per day), the marmosets were classified according to the pattern of their choice. Eight of 18 marmosets (44.4%) were risk-aversive, whereas 5 marmosets (27.8%) were risk-prone. The remaining 5 marmosets (27.8%) preferred to choose one side (left n = 4, right n = 1). These results showed individual differences in decision making of marmosets. An additional task with reduction in the expected value of the preferred choice revealed that risk-aversive marmosets were slower to adjust their choices to such reductions than risk-prone animals. © 2011 Elsevier Ireland Ltd and the Japan Neuroscience Society. All rights reserved.
1. Introduction In recent years, much interest has been paid to ‘neuroeconomics’, an approach to understand behavior and decision making based on unified economical, psychological and neuroscientific frameworks. An important problem in this emerging field (Knutson and Bossaerts, 2007; Doya, 2008) is whether animals, including humans, avoid or seek risk under various circumstances (Behar, 1961; Hill and Riopelle, 1985; Kahneman and Tversky, 2000; Marsh and Kacelnik, 2002; McCoy and Platt, 2005; Hayden and Platt, 2007), since this type of decision making is closely related to economical activities like gambling, speculation and investment. Thus, evaluation of utility, value and probability by the brain has been discussed repeatedly (for review, see Glimcher, 2003; Glimcher and Rustichini, 2004; Kable and Glimcher, 2009). During such exploration, McCoy and Platt (2005) have demonstrated risk-related neuronal activities in the posterior cingulate cortex of macaque monkeys. In addition, they have described that both of 2 monkeys used in their experiments showed riskprone behavior. However, utilization of 2 monkeys seems to be not enough to generalize preference for risky choice in monkeys. Since there has been increasing interest in the common marmoset (Callithrix jacchus, a species of New World monkeys) as an alternative primate species in neuroscience in recent years, we have established a breeding colony of the marmoset in our lab-
∗ Corresponding author. E-mail address: [email protected] (H. Tokuno).
oratory and have been using it for neuroscientific experiments (Tokuno et al., 2009, 2011; Paxinos et al., 2011). The common marmosets are small primates which can be easily maintained on middle to large scale (approximately 50–500 marmosets) in laboratory because of their availability, costs, size, and ease of breeding in laboratory conditions. In the present study, we examined decision making in the gambling task using 18 common marmosets as a basis for future neuroeconomical studies in the marmoset. We additionally tested the effect of reward reduction in the preferred choice on the pattern of decision making of the marmosets.
2. Materials and methods 2.1. Subjects Experiments were performed in 10 male and 8 female young to adult common marmosets (C. jacchus) weighing 250–510 g. The age of the animals ranged from 9 months to 6 years 2 months (mean, 2 years 3 months) at the start of the training. These marmosets were derived from a breeding colony at Tokyo Metropolitan Institute for Neuroscience (now Tokyo Metropolitan Institute of Medical Science). The use of the marmosets in the present study followed the Guidelines approved by the Animal Experiment Committee at Tokyo Metropolitan Institute for Neuroscience. A single or a pair of marmosets was housed in home cages. The light cycle was 12:12 h with light starting at 07.00 h, and room temperature was kept at 28 ◦ C. Water was available ad lib.
0168-0102/$ – see front matter © 2011 Elsevier Ireland Ltd and the Japan Neuroscience Society. All rights reserved. doi:10.1016/j.neures.2011.07.1822
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
261
limited to 3 min in the preparation period and early 2 w. However, we observed almost no violation of this rule after the preparation period. Daily session for one marmoset usually took less than 30 s, because each trial took approximately 5 s. We observed that the marmosets became relaxed in the testing cage after the preparation period. After the gambling task during 8 w, the marmosets were finally classified according to the pattern of their choice.
Fig. 1. Photomicrographs showing the testing apparatus used in the present study. Black and white caps were used for the gambling task (a). The marmoset can get grains of sugar-coated puffed rice in the well after removal of the cap (a ). White caps marked with an open circle (left) or a cross (right) were used in the simple visual discrimination task (additional task 2) (b).
2.2. Gambling task Each marmoset was trained to move into an isolated testing cage for the task, and remove one of two colored caps on two horizontally arranged wells attached to a gray testing board in front of them (Fig. 1a). Each well contained constant reward (3 grains of sugar-coated puffed rice) or risky reward (0 or 6 grains; probability, 50%:50%). We chose the reward based on preliminary observation of food preference of marmosets, where all animals rapidly learned the taste of the sugar-coated puffed rice and were ready to take it. Marmosets could easily get grains from the well with its hand after removal of the cap (Fig. 1a ). The mean weight of one grain of the sugar-coated rice was 25.7 mg (n = 100; S.D., 5.1 mg). For each marmoset, white (W) or black (B) color was assigned randomly as a symbol of non-risky or risky choice. Arrangement of white and black caps was determined randomly in each trial. Since it is known that the marmoset shows a sex-linked polymorphism of red–green color vision (Travis et al., 1988; Pessoa et al., 2005), we used only monochromatic symbols throughout the present experiments (Fig. 1). The testing cage, the testing board and the arrangement of wells were designed symmetrically to avoid left–right bias in the present experiments. Each marmoset was trained to perform 5 trials per day. Thereby, the maximum possible number of reward grains per day was limited to 30, since we were afraid that the marmosets lost interest in excessive reward grains. We further took care of the marmosets from nutritional aspects. The tasks were carried out before daily feeding at 15.00 h. Body weight of each marmoset was measured once a week (w) and recorded. After a preparation period of 5 days for teaching the marmoset to remove the cap and take the reward grains, each marmoset was tested for 200 trials (25 trials per week) during 8 w. The time for one trial was recorded and
2.2.1. Addionnal task 1 – effect of reward reduction Additional tasks were carried out after the gambling task. The effect of gradual reduction of reward on decision making was tested in risk-aversive and risk-prone marmosets (additional task 1). These tasks were planned to examine the capability of the marmosets to alter their risk preference according to the change of expected value (EV) of the reward grain number. For risk-aversive marmosets (n = 8), the number of the reward for non-risky choice were reduced during 4 blocks as 3, 2, 1 and a half of grain (Fig. 2, left column). Each block consisted of 150 trials (6 w). For riskprone marmosets (n = 4), the probability of acquiring the reward was changed to reduce the EV of the reward grains if the marmosets continued to prefer the risky choice. The probability of acquiring 6 grains was decreased as follows: 50% (1/2), 33% (1/3), 17% (1/6), and 8% (1/12). Thus, the EV was reduced from 3 to 0.5 during 4 task blocks (Fig. 2, middle column). The additional task 1 was designed to reduce the EV equivalently among marmoset groups (Fig. 2, left and middle column). In each experiments, we did not make preliminary or explicit instruction about the reduction of the reward. The marmosets had to know the change of reward amount by examining their earnings. 2.2.2. Additional task 2 – simple visual discrimination task Furthermore, we trained marmosets which showed side preference (L-type and R-type, see Section 3 for detail) to perform a simple visual discrimination task using the same testing apparatus with two differently marked caps. In this additional task, we used new marks to instruct marmosets on the beginning of a new task. One of the two kinds of marks, i.e. an open circle and a cross, was drawn on the top of the white cap with black ink (Fig. 1b). One mark indicated correct choice (containing 3 grains of sugar-coated puffed rice in the well), while the other mark indicated incorrect choice (0 grain). The number of the reward grains for the correct choice was reduced from 3 to 0.5 during 4 task blocks as shown in Fig. 2 (right column). 2.3. Data analysis The obtained data were analyzed using R (programming language) for statistical computing (R Development Core Team, 2008). This free software was used for cluster analysis (Euclidean distance and complete linkage clustering method) and multiple comparisons method (Tukey’s HSD [honestly significant difference] test.) in the present study. 3. Results 3.1. Gambling task Fig. 3a summarizes the result of 200 trials in 18 marmosets. Each marmoset was plotted according to the percentage of their risky choice (vertical axis) and the percentage of the side of the cap removed by them (horizontal axis). We could classify the marmosets into 3 types (Fig. 3a and b) with the aid of cluster analysis (Euclidean distance and complete linkage clustering method; Fig. 3b). Eight of 18 marmosets (44.4%) were risk-aversive, while 5 marmosets (27.8%) were risk-prone. The remaining 5 marmosets
262
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
Fig. 2. The conditions and sequence in the additional tasks 1 and 2 following the gambling task. They were designed to examine the response to the reduction of reward grains. The expected value (EV) of reward grain number for the preferred choice was gradually reduced during 4 blocks as 3, 2, 1 and 0.5. Four blocks were sequentially carried out in each marmoset. The marmosets were tested for 150 trials during 6 weeks (w) in one block. Different parameters among task blocks are indicated by bold letters.
(27.8%) showed a tendency to prefer the left or right side, and were subdivided into 2 types (L-type, n = 4, [22.2%]; R-type, n = 1, [5.6%]). We found neither sex (Fig. 3a) nor age differences in the present study. Table 1 shows that we could not find significant age effect on the risk attitude of the marmosets. Typical examples of 25 trials in the last week of the task are demonstrated in Fig. 3c, which shows the result of every trial, i.e. the color of the chosen cap, the side of the chosen cap and the number of reward grains. The mean percentage of the risky choice was 18.1% (n = 8; S.D., 4.5%) and 71.4 (n = 5; S.D., 7.6%) in the risk-aversive and risk-prone marmosets, respectively. As seen in Fig. 3a and c, the risk-aversive marmosets showed a more persistent tendency to their favorite choice than the risk-prone marmosets. We calculated the mean percentage of the non-risky choice in the risk-aversive marmosets (81.9%; n = 8; S.D., 4.5%) and compared with that of risky choice in the risk-prone marmosets. The mean percentage was significantly different among these two groups (Student’s t-test, p = 0.0095). 3.1.1. Additional task 1 All 8 risk-aversive marmosets and 4 of the 5 risk-prone marmosets were used for additional task 1. The result of the additional task is summarized in Fig. 4. Both risk-prone and risk-aversive marmosets showed a change in their choice during the reduction of the
Table 1 Number of the marmosets showing the various risk attitude during the gambling task classified based on their age. Extended Fisher’s exact test revealed that there is no significant age effect on their risk attitude (p = 0.959).
0–1 years old 1–3 years old 3–6 years old
Risk-aversive
Risk-prone
L-type, R-type
2 3 3
1 2 2
1 3 1
8
5
5
reward to obtain more reward grains. However, the timing of the change was different among groups. The mean ratio of the risky choice increased in Block 4 of the risk-aversive marmosets. The risk-prone marmosets showed a gradual change during Blocks 2–4. Table 2 summarizes the timing of the change in each marmoset. This table also suggests the time difference of the change among groups. Five (62.5%) of the 8 risk-aversive marmosets changed the ratio of their favorite choice significantly in Block 4. Fig. 5 exemplifies the daily change of the total number of the reward grains in each day (5 trials) during the 4 task blocks in a risk-aversive marmoset (Fig. 5a) and a risk-prone one (Fig. 5b). Riskprone animals showed stronger vertical fluctuations in the number of grains per day and deviated less from the EV of 15 grains/day than risk-aversive animals. Thus, risk-prone animals were better and earlier able to adjust their choices to the changed outcomes (see also Table 2). The risk-aversive marmoset showed week vertical fluctuation during Blocks 1–3, where total grains earned were decreased block by block due to their non-risky choice. The vertical fluctuation was manifest in Block 4 (Fig. 5a), indicating that the marmoset started to prefer the risky choice. On
Table 2 Number of the marmosets showing the first significant change (Tukey’s HSD [honestly significant difference] test, p < 0.05) of their choice in each block of the task. The experimental design is shown in Fig. 2. Additional task 1
Block 1 Block 2 Block 3 Block 4 No change
Additional task 2
Risk-aversive
Risk-prone
L-type, R-type
0 0 2 5 1
0 2 1 1 0
0 2 1 0 1
8
4
4
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
263
Fig. 3. Result of 200 trials (5 trials per day, 25 trials per week, 8 w) in the gambling task. (a) Each marmoset was plotted according to the percentage of their risky choice (vertical axis) and the percentage of the side of the cap (horizontal axis). (a ) The marmosets were classified into 3 groups showing risk-aversive, risk-prone and side preference behavior that was further subclassified into L-type and R-type. The X–Y plot is the same one as in Fig. 1a. (b) Result of the cluster analysis based on the dataset shown in a. We used the Euclidean distance and complete linkage clustering method. The cluster analysis supported the present classification. (c) Samples showing results of 25 trials in the last week of the gambling task in a risk-aversive (top), risk-prone (middle) or L-type (bottom) marmoset. The color of the chosen cap (W, white; B, black), the side of the chosen cap (L, left; R, right) and the number of reward grains in every trial were arranged sequentially from left to right.
the contrary, the risk-prone marmoset showed gradual decrease of vertical fluctuation during Blocks 1–3 and almost no vertical fluctuation in the later half of Block 3 and Block 4 (Fig. 5b). We further analyzed the relationship between the individual degree of risk-proneness and the timing of their change of choice in each marmoset. However, this statistical analysis did not provide significant results.
3.1.2. Additional task 2 Four of 5 L-type and R-type marmosets were tested using simple visual discrimination task (additional task 2). They showed early change in their decision making to obtain reward by choosing the correct cap in Block 2 (Fig. 4, right; Table 2). The ratio of the correct choice was as high as about 90% in Blocks 3 and 4 (Fig. 4, right). Finally, it should be noted here that no marmoset dropped out of the present series of experiments in spite of long time course of the task schedule (total 33 w). Two marmosets that were not
examined in additional tasks were used for other experiments (not shown) after the gambling task. 4. Discussion The present experiments disclosed the individual difference in decision making under risk in 18 marmosets. The present experiments further showed that 8 of 18 marmosets were risk-aversive. Previous studies showed risk-aversive behavior of the macaque monkeys, by demonstrating that they preferred the choice with the more frequent but lesser amount of reward (Behar, 1961; Hill and Riopelle, 1985). On the contrary, a recent study revealed riskprone behavior of the macaque monkeys (McCoy and Platt, 2005). Following experiments in the same macaque monkeys, they further showed that the mean ratio of the risky choice was about 50% after prolongation of the delay time between trials to 60–90 s (Hayden and Platt, 2007). These data suggest that the macaque monkeys also have individual difference in decision making under risk as the
264
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
Fig. 4. Change in percentage of risky choice (left and middle, additional task 1) or correct choice (right, additional task 2). The data were analyzed statistically with the multiple comparisons method based on Tukey’s HSD (honestly significant difference) test.
marmosets. In addition, it is reasonable to consider that the degree of individual risk-aversion (risk-proneness) will change according to the experimental conditions. The present additional tasks revealed that the marmosets can change their selection by evaluating the reward amount. The neural basis for the risk attitude of marmosets is unclear from the present behavioral experiments. However, a recent study by Tobler et al. (2009) has revealed probable human brain regions related to risk attitude. It was reported that risk enhanced reward-related activities of the lateral regions of the prefrontal cortex in risk-prone participants. In contrast, risk reduced such activities in risk-aversive participants. These findings lead us to consider that such activities of prefrontal cortex may play a role in changing choices in the additional task. Since the marmosets are primates and have well-developed prefrontal cortical areas in comparison with rodents (Paxinos et al., 2011), future studies using marmosets would be helpful to analyze neuronal mechanisms in the prefrontal cortex underlying the risk attitude with the aid of invasive experimental techniques including electrophysiology. The third group of marmosets showed different behavior. These L-type and R-type marmosets showed side preferences. To interpret the decision making of these marmosets, the cost for decision making and related process in the brain should be considered. The marmosets showing risk-aversive or risk-prone behavior are required to (1) observe the color of caps, (2) decide the cap to take based on their memory about the symbol, risk, expected reward and preference to risk, (3) execute the motor program for the control of their forelimb in the necessary side to remove caps and take grains, and finally (4) evaluate and remember the results. All of the processes necessitates the work of multiple brain areas related to visual, decision making, motor, memory and reward evaluation system. Since the EV of the number of reward grains did not depend
on selection strategy of the marmosets under the present experimental design, the L-type and R-type marmosets can omit all of the brain process by taking the cap of the single side constantly. As the effort for each trial was much reduced, the behavior of these animals can be interpreted as energy saving. Thus, this type of the marmoset was the most superior in cost performance in the present gambling task. The additional task 2 provided further evidence supporting this scenario. The L-type and R-type marmosets were able to perform simple visual discrimination task (additional task 2). They showed similar side preference in Block 1 following the gambling task. However, they showed slight elevation of the ratio of the correct choice (62.5%) in Block 1. The ratio of the correct choice was finally increased to about 90% in Blocks 3 and 4 (Fig. 4). This fact strongly suggests that the L-type and R-type marmosets can observe, choose and take the correct cap on each side, if necessary. Thus, it is reasonable to consider that these marmosets took the cap of the specific side intentionally in the gambling task. Risk attitude in human behavior is usually categorized into three types: risk-aversive, risk-prone and risk-neutral (for recent review, see Engelmann and Tamir, 2009). The L-type and R-type marmosets observed in the present experiments can be interpreted as riskneutral, because they did not show risk aversion or risk preference in the gambling task. Analysis of information processing in the brain of risk-neutral animals will be necessary to fully understand the risk attitude of animals including humans. In spite of the equivalent EV between non-risky and risky choice except for the uncertainty, the degree of preference to the non-risky or risky choice was different between the risk-aversive and riskprone marmosets behaviorally. The number of the risk-aversive marmosets (n = 8) was larger than that of the risk-prone marmosets (n = 5). The mean percentage (81.9%) of non-risky choice in the risk-aversive marmosets was significantly larger than that (71.4%)
H. Tokuno, I. Tanaka / Neuroscience Research 71 (2011) 260–265
265
making, and what kind of hierarchy and chemical modulation exists between these areas. To further elucidate the neural circuitry and neurochemical mechanism underlying the decision making under risk, invasive experimental studies using animals will be required. For example, effect of partial destruction of brain areas or drug application into specific brain areas on decision making should be examined in future studies. Acknowledgements We thank M. Inugami and M. Saitoh in the Animal Center of the Tokyo Metropolitan Institute of Medical Science for their help. References
Fig. 5. Samples of daily change in total number of reward grains earned on each day (5 trials) during the 4 task blocks in a risk-aversive marmoset (a) and a risk-prone one (b). Risk-prone animals showed stronger vertical fluctuations in the number of grains per day and deviated less from the expected value (EV) of 15 grains/day than risk-aversive animals. Risk-prone animals were better and earlier able to adjust their choices to the changed outcomes.
of risky choice in the risk-prone marmosets (see Section 3.1). In additional tasks, the mean percentage of the risky choice in the risk-aversive marmosets did not change until Block 4, while the risk-prone marmosets showed a gradual change of choice during Blocks 2–4. These findings indicate, on the whole, that the degree of avoiding risk is relatively larger than that of seeking risk in the marmoset. Taking into account the present results and natural behavior of wild animals, it is possible that small animals such as the marmoset are more risk sensitive than large animals. Recent brain imaging studies in humans have shown that the insular cortex, lateral orbitofrontal cortex, ventral striatum and midbrain were activated in relation to the risk (Dreher et al., 2006; Preuschoff et al., 2006; Lin et al., 2008). Nevertheless, it is still unclear how these areas cooperate during the process of decision
Behar, I., 1961. Learned avoidance of non-reward. Psychol. Rep. 9, 43–52. Doya, K., 2008. Modulators of decision making. Nat. Neurosci. 11, 410–416. Dreher, J.C., Kohn, P., Berman, K.F., 2006. Neural coding of distinct statistical properties of reward information in humans. Cereb. Cortex 16, 561–573. Engelmann, J.B., Tamir, D., 2009. Individual differences in risk preference predict neural responses during financial decision-making. Brain Res. 1290, 28–51. Glimcher, P.W., 2003. Decisions, Uncertainty, and the Brain. The Science of Neuroeconomics. The MIT Press, Cambridge, MA. Glimcher, P.W., Rustichini, A., 2004. Neuroeconomics: the consilience of brain and decision. Science 306, 447–452. Hayden, B.Y., Platt, M.L., 2007. Temporal discounting predicts risk sensitivity in rhesus macaques. Curr. Biol. 17, 49–53. Hill, C.W., Riopelle, A.J., 1985. Probability-reward preferences of rhesus monkeys. Am. J. Psychol. 98, 77–84. Kable, J.W., Glimcher, P.W., 2009. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745. Kahneman, D., Tversky, A., 2000. Choices, Values, and Frames. Cambridge University Press, Cambridge. Knutson, B., Bossaerts, P., 2007. Neural antecedents of financial decisions. J. Neurosci. 27, 8174–8177. Lin, C.-H., Chiu, Y.-C., Cheng, C.-M., Hsieh, J.-C., 2008. Brain maps pf Iowa gambling task. BMC Neurosci. 8, 72. Marsh, B., Kacelnik, A., 2002. Framing effects and risky decisions in starlings. Proc. Natl. Acad. Sci. U. S. A. 99, 3352–3355. McCoy, A.N., Platt, M.L., 2005. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 8, 1220–1227. Paxinos, G., Watson, C., Petrides, M., Rosa, M., Tokuno, H., 2011. The Marmoset Brain in Stereotaxic Coordinates. Academic Press, San Diego. Preuschoff, K., Bossaerts, P., Quartz, S.R., 2006. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51, 381–390. Pessoa, D.M., Tomaz, C., Pessoa, V.F., 2005. Color vision in marmosets and tamarins: behavioral evidence. Am. J. Primatol. 67, 487–495. R Development Core Team, 2008. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org. Travis, D.S., Bowmaker, J.K., Mollon, J.D., 1988. Polymorphism of visual pigments in a callitrichid monkey. Vision Res. 28, 481–490. Tobler, P.N., Christopoulos, G.I., O’Doherty, J.P., Dolan, R.J., Schultz, W., 2009. Riskdependent reward value signal in human prefrontal cortex. Proc. Natl. Acad. Sci. U. S. A. 106, 7185–7190. Tokuno, H., Tanaka, I., Umitsu, Y., Akazawa, T., Nakamura, Y., 2009. Webaccessibledigital brain atlas of the common marmoset (Callithrix jacchus). Neurosci. Res. 64, 128–131. Tokuno, H., Tanaka, I., Senoo, A., Umitsu, Y., Akazawa, T., Nakamura, Y., Watson, C., 2011. Internet-based atlas of the primate spinal cord. Neurosci. Res. 70, 128–132.