Evaluation of an Aβ1–40-induced cognitive deficit in rat using a reward-directed instrumental learning task

Behavioural Brain Research 234 (2012) 323–333

Contents lists available at SciVerse ScienceDirect

Behavioural Brain Research journal homepage: www.elsevier.com/locate/bbr

Research report

Evaluation of an A␤1–40 -induced cognitive deficit in rat using a reward-directed instrumental learning task Zhe Shi a , Xiuping Sun a , Xinmin Liu a,∗ , Shanguang Chen b,∗∗ , Qi Chang a , Lingling Chen a,c , Guangqing Song a , Haiqing Li d a Research Center for Pharmacology & Toxicology, Institute of Medicinal Plant Development (IMPLAD), Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100193, China b Science and Technology on Human Factors Engineering Laboratory, Astronaut Centre of China, Beijing 100193, China c Hunan University of Traditional Chinese Medicine, Changsha, Hunan, 410208, China d Beijing Xin Hai Hua Yi technology Co. Ltd., Beijing 100193, China

h i g h l i g h t s     

Our laboratory is the first to utilize a reward-directed instrumental learning procedure in a dementia study. A␤1–40 injection affects the acquisition of the lever-pressing response rather than its maintenance. A␤1–40 injection impairs the ability to adjust behavior to adapt to action–outcome contingency changes. A␤1–40 injection impairs the ability to make a stimulus–response association from a two-color visual stimulus. This series of instrumental learning tasks has the potential to be applied in dementia research as a new cognitive behavior testing method.

a r t i c l e

i n f o

Article history: Received 30 April 2012 Received in revised form 3 July 2012 Accepted 6 July 2012 Available online 14 July 2012 Keywords: A␤1–40 Dementia symptoms Cognitive deficit Reward Instrumental learning Rat

a b s t r a c t Alzheimer’s disease (AD) is the most common form of dementia. It is a progressive neurodegenerative disorder that leads to gradual loss of cognitive and functional abilities, and development of behavioral disturbances. Previous studies using A␤1–40 microinjection in animal models focused on cognitive deficits in spatial learning and avoidance conditioning. However, no attempt has been made to determine the sensitivity of an A␤1–40 -manipulated animal model in tasks involving reward-directed instrumental learning (RDIL). Thus, the present study was designed to investigate the effects of intra hippocampal microinjection of A␤1–40 on the acquisition and maintenance of a basic instrumental response (leverpressing), then on the goal directed (higher response ratio) and habit (visual signal discrimination and extinction) learning, as well as on neurotransmitter changes which could potentially alter the regulatory processes involved in instrumental learning. Our present findings demonstrated that the focal hippocampal microinjection of A␤1–40 rendered rats unable to process new cue/contextual information in the formation of causal relation, rather than affecting the operant action itself. Although the injected A␤1–40 did not directly influence performance, it did prevent the information from being translated into action. Moreover, the neurotransmitter results indicated that multiple neural signaling might be involved in the regulation of RDIL in the A␤1–40 injection model. In conclusion, results suggested that our series of instrumental learning tasks may have potential in dementia research as a novel method for testing cognitive behavior. © 2012 Elsevier B.V. All rights reserved.

1. Introduction Alzheimer’s disease (AD) is the most common form of dementia in the elderly and has rapidly emerged as a major public

∗ Corresponding author. Tel.: +86 010 6281 2595; fax: +86 010 6281 2595. ∗∗ Corresponding author. Tel.: +86 010 66362029; fax: +86 010 66362029. E-mail addresses: [email protected] (X. Liu), [email protected] (S. Chen). 0166-4328/$ – see front matter © 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.bbr.2012.07.006

health issue throughout the world [1]. It is a progressive neurodegenerative disorder that leads to a gradual loss of cognitive and functional abilities, and the development of behavioral disturbances [2]. Among regions of the AD brain, the hippocampus has one of the highest concentrations of amyloid-containing senile plaques [3]. An in vivo hippocampal microinjection of amyloidbeta peptides (A␤) in rodents induced neurocytic denaturation and apoptosis, as well as cognitive deficits [4–7]. This type of manipulation caused impairments on spatial and non-spatial tasks which were considered to reflect damage to the functional integrity of the

324

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

hippocampus [8]. This highly used laboratory model has contributed enormously to AD research over the years [9]. The principal behavioral tasks described in recent articles have been maze[10–14] and avoidance- [12,15–17] based. Evaluation of spatial memory and avoidance conditioning has been the theme of most of these studies. In addition, punitive stimuli were commonly applied. From one perspective, the tasks employed above not only gave an incomplete description of the fundamental behavioral alterations in dementia but also caused harm to the subjects. Moreover, although some tasks using rewards were conducted, they were still aimed at assessing spatial cue learning [3,18,19]. Therefore, other aspects of cognitive deficits induced by A␤ are worth exploring to gain a better understanding of the disease. Instrumental conditioning, which is also called operant conditioning, provides a very accurate model of goal-directed action. It is a form of associative learning through which an animal learns from the consequences of its behavior [20]. Result using a transgenic AD mouse model showed an age-dependent deficit in acquiring a positive appetite-driven Pavlovian context-outcome, but revealed no instrumental or cue-outcome associations [21]. Since, no attempt has been made as yet to determine how an animal model administrated A␤1–40 would response to a reward-directed instrumental learning (RDIL), we focused on this area in our search of the literature. Reward-guided conditioning is controlled by two memory systems: a goal-directed process and a stimulus–response habit mechanism that involves two forms of learning. The first consists of establishing incentive by introduction to the reward, whereas the second consists of making an association between the response and receiving the reward [22]. According to the associative theory, instrumental learning is mediated by cues (stimulus) that predict the reward (outcome) and goal-directed behavior is learned (response/action) to gain access to the reward. The capacity for instrumental conditioning depends critically on the ability to encode a causal relationship between the action and the outcome. In addition, to the pleasure aspects of natural reward (e.g. sucrose, food, etc.), the learning process itself would be more pleasant [23] and would even have the advantage of promoting neural circuit remodeling in the brain [24]. Substantial evidence from lesion study has delineated the roles of specific brain regions in behavior regulation during instrumental cognitive tasks. In these types of studies, researchers induce lesions in specific brain regions of animal models and subsequently test for operant behavior alterations. If the behavior is compromised, it suggests that the destroyed tissue is part of a brain region that is important to the normal expression of operant behavior. Although lesion studies can elucidate the neural basis of a behavior by disrupting function in a specific brain region, the lesion may lead to permanent tissue damage which is far from mimicking the pathological process in dementia. However, A␤1–40 only impairs neural transmission and does not damage the structure of the hippocampus [14,25]. Therefore, we speculated that the A␤1–40 model may share some aspects in common with the lesion study model but would have distinguishing influence on RDIL. Consequently, the aim of the present study was to evaluate A␤1–40 -induced cognitive deficits in RDIL. We first investigated the effects of pre-training and post-training intra hippocampal microinjection of A␤1–40 on the acquisition and maintenance of instrumental response (Experiment 1). Secondly, the sensitivity to changes in the action–outcome contingency degradation and the identification ability of different stimuli–response associations were assessed in the post-training surgically treated rats (Experiment 2). After completing all the behavioral procedures, the rats were sacrificed and each hippocampus was examined for neurotransmitter alterations.

2. Methods 2.1. Animals Eighty-one male Wistar rats (Vital river, Beijing, China), 10 weeks old, weighing 300–320 g at the beginning of the experiments, were housed 4 to a cage with lights on from 7:00 h to 19:00 h. The rats were maintained at 85% of an adjusted ad libitum body weight throughout the duration of the study by restricting food to approximately 16 g per day. Once training began, they were fed each day after the training sessions, and had free access to water while in their own cages. All rats, regardless of group, received the same handling and feeding during this phase of the experiment. All animal handling procedures were performed in accordance with the “Principles of Laboratory Animal Care” (NIH publication No. 86-23, revised in 1996) and P.R. China legislation for the use and care of laboratory animals. All efforts were made to minimize animal suffering during experiments. The protocols were approved by the committee for the Care and Use of Laboratory Animals of IMPLAD, CAMS & PUMC, China. 2.2. Apparatus Behavioral testing was conducted in four operant chambers (Xin Hai Hua Yi Instrument Co., Beijing China) fitted with a dipper magazine and a retractable lever (4 cm wide, positioned 10 cm from the side walls, 2.5 cm to the magazine and 5 cm from the grid floor). Three-color LED signal lights were located 5 cm above the lever. The chambers could be illuminated by a LED house light located on the ceiling. An infrared beam emission and acceptance device was fixed on the side wall of the magazine to record the nose poke activity of the subjects. Ventilation and a masking noise were provided by an exhaust fan. The operant chambers were housed in sound-attenuated rooms. 2.3. Drug preparation The drug used was amyloid ␤ protein fragment 1–40 (A␤1–40 ) (Sigma–Aldrich, USA) which was dissolved in sterile double-distilled water at a concentration of 5 ␮g/␮l and incubated at 37 ◦ C for 7 days prior to use. 2.4. Surgical procedures Rats were assigned to surgery groups in a quasirandom manner. Initial random group assignments were adjusted using baseline magazine training performance to control for a response bias. Animals were divided into three groups for use in Experiments 1A, 1B and 2 (see Fig. 1). These groups consisted of A␤1–40 -injected (n = 9), sham-injected (n = 9) and non-operated control (n = 9) rats respectively. The surgically manipulated rats received either a sham or A␤1–40 injection into the dorsal hippocampus using the procedures described below. Rats were anesthetized with 10% chloral hydrate diluted in physiological saline (3.5 ml/kg, IP) and placed into a stereotaxic apparatus (Benchmark, USA) with head held horizontally. A midline incision was then made into the scalp and the scalp was retracted. Small holes were then drilled into the skull above the injection sites using a dental burr. The stereotaxic coordinates to conduct a bilateral microinjection in the CA1 region of the hippocampus (anterior-posterior (AP) = −3.3 mm, medial-lateral (ML) = ±2.0 mm from the bregma and dorsal-ventral (DV) = 3.0 mm from cerebral dura mater) were standardized from the stereotaxic atlas of Paxinos and Watson [26]. A flatted-tipped Hamilton syringe lowered into the bilateral hippocampus and 5 ␮l of A␤1–40 were delivered at a rate of 1 ␮l/min. Following the injection, the needle was kept in place for 5 min prior to its slow extraction. Rats of the sham group were infused with the vehicle only. After surgery, animals were placed in heated chambers in a darkened room and allowed to recover with free access to food and water. The experiment was continued after 10 days of recovery as follows. 2.5. Behavioral task 2.5.1. Magazine training When subjects were at 85% of their ad libitum weight, all of them were habituated to the operant box over three consecutive 20 min sessions in which the reward (approximately 0.2 ml) was delivered daily in diminishing amounts (30, 25 and 20 drops) on a random time (RT) (40 s, 48 s and 60 s) schedule. Throughout the experiment, the lever was retracted. Each session began with the onset of the house light and terminated with its offset after 20 min. During the magazine process, rats in the operant chamber were only exposed to a blue cue light and white noise. The blue cue light was simultaneously illuminated with the appearance of the reward which was an 8% (m/v) solution of sucrose in distilled water that was prepared daily before each session. 2.5.2. Experiment 1 A total of 54 rats was used to assess the effect of pre-training and post-training intra hippocampal microinjection of A␤1–40 on the acquisition and maintenance of the lever pressing response.

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

325

Fig. 1. Experimental manipulation of rats during the course of behavioral testing.

2.5.2.1. Experiment 1A: lever pressing response acquisition (FR1 schedule, see Fig. 1A). After three sessions of magazine training, the lever was inserted. Rats were trained to freely press the lever for the sucrose reward under continuous reinforcement (fixedratio 1, FR1 schedule). Ten consecutive 30 min sessions were conducted. The blue cue light was simultaneously illuminated for 10 s following the acquisition of reward. The daily session was terminated after either 50 rewards or 30 min, whichever came first. 2.5.2.2. Experiment 1B: lever pressing response maintenance (FR1 schedule, see Fig. 1B). Post-training groups underwent surgery after 10 days of lever pressing when the subjects reached the criterion of 50 rewards within 30 min in two consecutive sessions. After 10 days to recover from the surgery, they were given 2 days on the FR1 schedule to test for the maintenance of their pre-surgically acquired instrumental abilities. 2.5.3. Experiment 2 (see Fig. 1C) A total of 27 rats was used to assess the effects of post-training intra hippocampal microinjection of A␤1–40 on goal-directed and habit learning cognition. Subjects underwent the injection after 10 days of lever pressing, and after 10 days recovery from the surgery, the following procedures were conducted. 2.5.3.1. Goal-directed learning/contingency degradation (higher fixed ratio schedule). After recovery, the subjects were trained to press the lever at a higher frequency of response under the FR2 and FR4 schedule, meaning that during the session, every 2 or 4 pressings of the lever resulted in one reward delivery. Five consecutive 30 min sessions were conducted. The blue cue light was illuminated following each successful lever pressings and left on for 10 s after the reward was presented. The process was terminated when the subjects earned 50 rewards during a 30 min session or when timed out.

solution (300 ␮g/ml DHBA). Fifty micro-liters of the mixture were injected into a LC–MS/MS system for assay. The LC–MS/MS instrument was equipped with an Agilent 1200 HPLC system (Palo Alto, CA, USA) and an Applied Biosystem 3200 Q-Trap mass spectrometer (Foster City, CA, USA) with an electrospray ionization source. The mobile phase consisted of 6 mM ammonium formate in acetonitrile–water (67.5:32.5, pH 5.50) with a flow rate of 200 ␮l/min. The neurotransmitters and internal standard were detected in multiple reaction monitoring mode. Ratios of the peak areas of the analyte vs. the internal standard were used to quantify the neurotransmitter concentrations. 2.7. Statistical analysis All analyses were performed using SPSS version 16.0 (Chicago, IL, USA). Values were expressed as mean ± SEM, and statistical significance was set at p < 0.05 in all of the evaluations. The results of the analysis of these data were only reported when a significant difference was observed. The behavioral data were first analyzed using repeated measures analysis of variance (ANOVA) with days as the within-subject variable and different treatment groups as the between-subject variable (control vs. sham vs. A␤1–40 ). Mauchley’s test was used to evaluate the sphericity of the within-subject effects and when necessary, Greenhouse–Geisser was applied to adjust the degrees of freedom. When significant effects were detected, post hoc multiple pairwise comparisons were made using the LSD Comparisons test after ANOVA. Furthermore, one-way ANOVA were used to test differences between groups within one session. Values were compared using post hoc comparisons with the results of LSD testing. The contents of neurotransmitter in the hippocampus were analyzed with oneway ANOVA, as described above, to compare the between-groups differences.

3. Results

2.5.3.2. Habit learning (signal discrimination and extinction). 2.5.3.2.1. Discrimination of conditioned cue signaling. During the training course, blue and red cue signals were alternately turned on in 120 s period. Fourteen sessions were conducted daily. The blue light (S+ ) served as a reward predictor while the red light (S− ) was associated with a non-rewarded consequence. Rats were trained to lever press in response to the alternating visual cues. They could earn one reward after one lever pressing in S+ phase, and this action was considered a correct response. The training period lasted 5 days. 2.5.3.2.2. Extinction. Testing under the signal discrimination schedule was followed by three 20 min extinction sessions. Rats were exposed to the signal discrimination environment but experienced no scheduled consequences in response to correct lever pressing.

3.1. Magazine training

2.6. Biochemical analysis of neurotransmitters

3.2. Experiment 1. Acquisition and maintenance of the basic instrumental response (lever pressing) in pre-training and post-training surgically manipulated animals

Immediately upon completion of the extinction session (Section 2.5.3.2.2), the rats were anesthetized and then sacrificed. Their brains were rapidly removed, and the hippocampuses were dissected out on ice. The concentrations of six neurotransmitters, dopamine (DA), noradrenalin (NE), glutamate (Glu), ␥-aminobutyric acid (GABA), acetylcholine (Ach) and serotonin (5HT), were simultaneously determined by a LC–MS/MS method. The tissues were weighed and homogenized in ice-cold 0.2% aqueous formic acid, then mixed with 0.2% formic acid in acetonitrile for protein precipitation. After centrifugation at 12,000 rpm for 10 min at 4 ◦ C, an aliquot of the supernatant (200 ␮l) was collected and mixed with 20 ␮l of internal standard

Before the instrumental task was conducted, magazine training was carried out first to insure that all the subjects could make the simplest Pavlovian S–O association. The sucrose reward was paired with the cue signal which resulted in exploration of the magazine. There were no significant differences in nose pokes (NPs) activities (results not shown) among the groups. This implied that when the training started, the initial incentive motivation aroused by the reward substance was at the same level in all groups.

The statistical analysis results are presented in Table 1. Daily data on instrumental response acquisition are shown in Fig. 2(A)–(C). The LPs and LP/NP ratios increased progressively during the training days in all groups. However the A␤1–40 -treated group performed significant fewer LPs and lower LP/NP ratios than

326

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

Table 1 Statistical analyses results of instrumental response acquisition and maintenance performance in pre-training and post-training surgically treated animals. Parameters

Acquisition LPs NPs LP/NP ratios Maintenance LPs NPs LP/NP ratios

Main effects

Interaction effects

Post hoc analyses (group)

Factor 1 (Day)

Factor 2 (Group)

Day × Group

Controla

Shama

F = 6.551, p < 0.001b F = 5.046, p < 0.001b F = 6.725, p = 0.008b

F = 9.753, p = 0.002b F = 4.309, p = 0.032b F = 10.788, p < 0.001b

F = 1.801, p = 0.031b F = 1.220, p = 0.253 F = 1.633, p = 0.147

p = 0.001b p = 0.016b p = 0.001b

p = 0.018b p = 0.028b p = 0.018b

F = 10.612, p = 0.005b F = 9.445, p = 0.008b F = 3.546, p = 0.079

F = 0.511, p = 0.610 F = 1.102, p = 0.358 F = 0.382, p = 0.689

F = 2.153, p = 0.151 F = 3.397, p = 0.061 F = 0.382, p = 0.689

p = 0.559 p = 0.310 p = 0.865

p = 0.689 p = 0.708 p = 0.852

LPs, lever pressings; NPs, nose pokes. Statistical analyses of all results with RM ANOVA showing the main factor 1 effects (Day), main factor 2 effects (Group) and factor 1 × factor 2 interaction effects (Day × Group). a Compared with A␤1–40 group. b Significant differences.

Fig. 2. Instrumental response acquisition and maintenance performance in pre-training and post-training surgically treated animals. Subparts (A) and (D) represent numbers of daily lever pressings; subparts (B) and (E) are numbers of daily nose pokes; subparts (C) and (F) are daily LP/NP ratios. The data are expressed as mean ± SEM. The final numbers of rats used are as follows: control group, n = 7; sham group, n = 6; and A␤1–40 group, n = 6.

the control or sham group, indicating that the ability to shape instrumental conditioning across training days was impaired after A␤1–40 treatment. Fig. 2(D)–(F) shows the maintenance of the lever-pressing response after the post-training A␤1–40 treatment, and as seen, no change (FR1 schedule) was observed. The three groups engaged in similar numbers of LPs on both days test. Furthermore, no significant differences in the levels of LPs, NPs and LP/NP ratios were evident among the A␤1–40 , sham and control groups at the end of the testing periods. 3.3. Experiment 2 3.3.1. Behavioral performance in goal-directed learning (higher rate of response schedule) in post-training surgically manipulated animals We first used RM to analyze the interaction effects between groups and days of training. The statistical analysis results are presented in Table 2. There was no significant change in any parameter on employing the FR2 schedule. Following the FR4 schedule, the

A␤1–40 group significantly showed a tendency of decreased Rs and P(R/LP) values (Fig. 3(C) and (E)) and increased ILPs (Fig. 3(D)). Moreover, we analyzed the differences between groups on each day using one-way ANOVA. We found a significant difference in the LPs of the A␤1–40 group vs. the sham group on day 4 (F(2,22) = 2.828, p = 0.028).We also found significant differences in the rewards eared by the A␤1–40 vs. the sham group on day 4 (F(2,22) = 4.282, p = 0.008) and day 5 (F(2,22) = 2.395, p = 0.042). Significant differences were observed in the number of incorrect LP responses on day 3 (A␤1–40 vs. sham, F(2,22) = 5.276, p = 0.014; and vs. control, F(2,22) = 5.276, p = 0.008), and on day 4 (A␤1–40 vs. sham, F(2,22) = 2.734, p = 0.043 and vs. control, F(2,22) = 2.734, p = 0.046),and on day 5(A␤1–40 vs. sham, F(2,22) = 3.165, p = 0.024; and vs. control, F(2,22) = 3.165, p = 0.034). Significant differences were found in correct response ratio (R/LP) between the A␤1–40 vs. the sham group on day 4 (F(2,22) = 3.995, p = 0.008) and day 5 (F(2,22) = 5.516, p = 0.042), and between the A␤1–40 vs. the control group on day 4 (F(2,22) = 3.995, p = 0.046) and day 5 (F(2,22) = 5.516, p = 0.049).

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

327

Table 2 Statistical analyses results of behavioral performance in goal-directed learning (higher rate of response schedule). Parameters

LPs NPs Rs ILPs P(R/LP)

Main effects

Interaction effects

Post hoc analyses (group)

Factor 1 (Day)

Factor 2 (Group)

Day × Group

Controla

Shama

F = 1.164, p = 0.327 F = 3.216, p = 0.037b F = 3.031, p = 0.073 F = 0.358, p = 0.701 F = 2.480, p = 0.097

F = 1.217, p = 0.315 F = 0.550, p = 0.585 F = 2.803, p < 0.05b F = 7.873, p = 0.003b F = 3.289, p = 0.049b

F = 1.282, p = 0.286 F = 1.195, p = 0.324 F = 1.984, p = 0.133 F = 0.534, p = 0.712 F = 3.292, p = 0.021b

p = 0.602 p = 0.895 p = 0.281 p = 0.004b p = 0.089

p = 0.140 p = 0.352 p = 0.028b p = 0.002b p = 0.002b

LPs, lever pressings; NPs, nose pokes; Rs, rewards; ILPs, incorrect lever pressings; P(R/LP), probability(R/LP). Statistical analyses of all results with RM ANOVA showing the main factor 1 effects (Day), main factor 2 effects (Group) and factor 1 × factor 2 interaction effects (Day × Group). a Compared with A␤1–40 group. b Significant differences.

Fig. 3. Behavioral performance in goal-directed learning (higher rate of response schedule) in post-training surgically manipulated animals. Subpart (A) is the number of daily LPs; subpart (B) is the number of daily NPs; subpart (C) is the number of Rs earned per day; subpart (D) is the daily number of ILPs and subpart (E) is the daily number of P(R/LP). Data are expressed as mean ± SEM. The final numbers of rats used are as follows. Control group, n = 7; sham group, n = 8; A␤1–40 group, n = 7. Significant differences *p < 0.05, **p < 0.001 compared with the control; # p < 0.05, ## p < 0.001 compared with the sham.

3.3.2. Behavioral performance in habit learning (visual signal discrimination and extinction task) in post-training surgically manipulated animals 3.3.2.1. Training session. The interaction effects of groups and training days were analyzed with RM ANOVA. The statistical analysis results are presented in Table 3. No significant differences in LPs, NPs and CNPs were found. The CLPs and CLPR increased and the ILPs decreased progressively during the training days in all groups. However the A␤1–40 -treated group performed significant fewer CLPs, lower CLPR and more ILPs than control or sham group, indicating that the A␤1–40 -administrated rats unable to adjust their responses to the cue reflecting a correct association with the reward outcome. Furthermore, we analyzed the differences between groups for each day using one-way ANOVA. Significant differences in LPs were found in the A␤1–40 vs. the sham group on day 3 (F(2,21) = 3.473, p = 0.016) and day 5 (F(2,21) = 3.236, p = 0.021). Significant differences were also observed in the number of CLPs on day 1 (A␤1–40 vs. sham (F(2,21) = 3.991, p = 0.026) and control (p = 0.021)), day 2 (A␤1–40 vs. sham (F(2,21) = 3.349, p = 0.037) and control (p = 0.035)), day 4 (A␤1–40 vs. sham (F(2,21) = 8.269, p = 0.003) and control (p = 0.002)), and day 5 (A␤1–40 vs. sham (F(2,21) = 8.487, p = 0.007)). Significant differences were found in the number of ILPs on day 1 (A␤1–40 vs. control (F(2,21) = 2.950, p = 0.030)), day 2 (A␤1–40 vs. sham (F(2,21) = 4.811, p = 0.046) and control (p = 0.007)),

day 3 (A␤1–40 vs. sham (F(2,21) = 13.322, p < 0.001) and control (p < 0.001)), day 4 (A␤1–40 vs. sham (F(2,21) = 14.728, p < 0.001) and control (p < 0.001)), and day 5 (A␤1–40 vs. sham (F(2,21) = 21.011, p < 0.001) and control (p < 0.001)). Significant differences were found in number of INPs on day 3 (A␤1–40 vs. sham (F(2,21) = 2.886, p = 0.039)), day 4 (A␤1–40 vs. sham (F(2,21) = 6.946, p = 0.003) and control (p = 0.009)), and day 5 (A␤1–40 vs. sham (F(2,21) = 3.415, p = 0.019)). Significant differences were found in CLPR on day 1 (A␤1–40 vs. control (F(2,21) = 3.404, p = 0.018)), and day 2 (A␤1–40 vs. sham (F(2,21) = 3.112, p = 0.032)), day 3 (A␤1–40 vs. sham (F(2,21) = 5.847, p = 0.003) and vs. control (p = 0.038)), day 4 (A␤1–40 vs. sham (F(2,21) = 10.537, p < 0.001) and vs. control (p = 0.004)), and day 5 (A␤1–40 vs. sham (F(2,21) = 18.892, p < 0.001) and vs. control (p = 0.001)). Significant differences were found in CNPR on day 3 (A␤1–40 vs. sham (F(2,21) = 4.645, p = 0.007)), and day 4 (A␤1–40 vs. sham (F(2,21) = 6.273, p = 0.004) and vs. control(p = 0.011)), and day 5 (A␤1–40 vs. sham (F(2,21) = 4.439, p = 0.008)). 3.3.2.2. Extinction session. The interaction effects related to group or training days were analyzed with RM ANOVA. No significant differences were found in LPs, NPs, CLPs, CNPs, INPs and CNPR during the extinction test. Fig. 4(E) shows the number of ILPs. The main effects pertained to the group (F(2,21) = 10.027, p = 0.001) and day (F(4,84) = 11.497, p < 0.001), but no interaction effects were observed (F(8,38) = 2.288,

328

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

Table 3 Statistical analyses results of behavioral performance in habit learning (visual signal discrimination task). Parameters

LPs NPs CLPs CNPs ILPs INPs CLPR CNPR

Main effects

Interaction effects

Post hoc analyses (group)

Factor 1 (Day)

Factor 2 (Group)

Day × Group

Controla

Shama

F = 3.074, p = 0.038b F = 0.735, p = 0.492 F = 41.274, p < 0.001b F = 11.133, p < 0.001b F = 5.113, p = 0.006b F = 4.759, p = 0.009b F = 41.177, p < 0.001b F = 18.689, p < 0.001b

F = 0.594, p = 0.562 F = 0.358, p = 0.704 F = 5.908, p = 0.009b F = 1.002, p < 0.385 F = 16.319, p < 0.001b F = 2.859, p = 0.081 F = 10.452, p = 0.001b F = 14.416, p = 0.026b

F = 2.182, p = 0.062 F = 0.900, p = 0.477 F = 0.735, p = 0.094 F = 0.646, p < 0.644 F = 1.813, p = 0.123 F = 1.375, p = 0.251 F = 1.842, p = 0.081 F = 1.864, p = 0.107

p = 0.509 p = 0.941 p = 0.004b p = 0.275 p < 0.001b p = 0.235 p = 0.002 b p = 0.083

p = 0.295 p = 0.448 p = 0.015b p = 0.295 p < 0.001b p = 0.027b p < 0.001 b p = 0.008b

LPs, lever pressings; NPs, nose pokes; CLPs, correct lever pressings; CNPs, correct nose pokes; ILPs, incorrect lever pressings; INPs, incorrect nose pokes; CLPR, correct lever pressing ratio; CNPR, correct nose poke ratio. Statistical analyses of all results with RM ANOVA showing the main factor 1 effects (Day), main factor 2 effects (Group) and factor 1 × factor 2 interaction effects (Day × Group). a Compared with A␤1–40 group. b Significant differences.

p = 0.077). Post hoc analysis revealed that the A␤1–40 group had a higher incorrect LPs during the extinction test (A␤1–40 vs. sham (p = 0.001) and vs. control (p = 0.002)). Fig. 4(G) shows the rate of CLPR. The main effects pertained to the group (F(2,21) = 5.507, p = 0.012) rather than to the day (F(4,84) = 0.888, p = 0.420) and no interaction effects between the two were observed (F(8,38) = 0.287, p = 0.885). Post hoc analysis revealed that the A␤1–40 group had a lower rate of correct LP responses during the extinction test (A␤1–40 vs. sham (p = 0.003)). Furthermore, we examined daily differences between groups using one-way ANOVA. Significant differences were found in the number of incorrect LPs on day 6 (A␤1–40 vs. sham (F(2,21) = 8.650, p = 0.002) and vs. control (p = 0.002)), as well as on day 7(A␤1–40 vs. sham (F(2,21) = 5.170, p = 0.004)). Significant differences were also found in the rate of CLPR on day 6 (A␤1–40 vs. sham (F(2,21) = 3.165, p = 0.027) and vs. control (p = 0.046)), day 7 (A␤1–40 vs. sham (F(2,21) = 2.419, p = 0.040)), and on day 8 (A␤1–40 vs. sham (F(2,21) = 2.797, p = 0.030)). In addition, significant differences were found in the rate of CNPR on day 6 (A␤1–40 vs. sham (F(2,21) = 2.639, p = 0.047)). 3.3.3. Hippocampal neurotransmitter alterations in post-training surgically manipulated animals Differences between groups were analyzed with one-way ANOVA. The DA level of the A␤1–40 group was significantly from that of the sham (F(2,21) = 3.601, p = 0.037) or control (F(2,21) = 3.601, p = 0.046) group. The Ach level was also significantly different on comparing A␤1–40 vs. sham (F(2,21) = 5.555, p = 0.011) and vs. control (F(2,21) = 5.555, p = 0.005). 4. Discussion Our laboratory is the first to utilize RDIL tasks in studying dementia symptoms. The results presented here supported our speculation that acute focal injection of A␤1–40 into the CA1 region of the hippocampus impairs cognition in instrumental learning tasks. The instrumental conditioning is one of the most elementary forms of behavioral adaptation [27]. Since a behavior that has once produced a positive outcome could later produce a negative outcome, it is important to be able to adjust behavior to adapt to changes in consequences. This flexibility allows rapid behavioral alterations in the face of changing outcomes, conferring a survival advantage. A primary disability in learning and retaining new information is one of the initial symptoms of AD [28,29]. This characteristic amnestic symptom renders the patient incapable of drawing on advantages and avoiding disadvantages, resulting in a gradual loss of the ability to take care of themselves [30]. For this

reason, we examined the effects of A␤1–40 injection on RDIL. The main findings were as follows.

4.1. Acquisition and maintenance of the basic instrumental response The initial acquisition and maintenance of a free operant response (A–O association) were first investigated in Experiment 1 (see Fig. 1A and B). Preliminary magazine training was designed to train the rats about the location of the reward, to teach them the signals associated with reward delivery (visual stimuli in the present experiment), to keep the rats aroused and to promote exploration [31]. The established Pavlovian signal–outcome association was then shifted to an A–O association in the lever-pressing training course. The appearance of visual stimuli was regarded as an intermediate step in facilitating the shaping of an A–O association. This association would simultaneously emerge with delivery of the reward when the rat accidentally pressed the lever. Benefiting from the earlier established S–O contingency, rats could be easily diverted to the new association. In contrast to the present work, Pavlovian conditioning was not disturbed in previous lesion studies [32], which likely explains why our A␤1–40 group seemed to be poor at the shaping the A–O (signal) association. It was evident that the approaches used in different lesion studies may bring about different consequences. Corbit and Balleine reported that the acquisition of the instrumental skill was not affected by a pretraining-inflicted electrolytic lesion [33], while other researchers have declared that the learning of lever-pressing was retarded, either when the reward was presented in the absence of a delay or when selective satiation was used after a pre-training-applied excitotoxic hippocampal lesion [32,34]. In addition, a post-traininginflicted lesion of this type had no effect on the maintenance of the instrumental response [34,35]. However, it did affect relearning in a time estimation task and the ability to withhold non-rewarded responses [35–37]. Somewhat in agreement with the lesion studies, we found that the acquisition was conspicuously retarded in the A␤1–40 administrated group, compared to the control and sham groups, on the continuous reinforcement (FR1) schedule. However, the maintenance of a preoperatively acquired response was not affected after surgery (see Fig. 2). Nevertheless, it was clear that the primary instrumental response acquired before surgery would not be impaired by focal hippocampal injection of A␤1–40 . Furthermore, the hippocampus and striatum now appear to constitute the primary neural system responsible for flexible, goal-directed actions [38,39]. In nonhuman subjects such as the laboratory rat, the hippocampus is essential for encoding diverse features of the animal’s experience such as spatial locations, landmarks, visual features of the environment, goal locations, conditioned stimuli,

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

329

Fig. 4. Behavioral performance in habit learning (visual signal discrimination and extinction task) in post-training surgically manipulated animals. Subpart (A) is the number of LPs per day; subpart (B) is the number of NPs per day; subpart (C) is the number of CLPs per day; subpart (D) is the number of CNPs per day; subpart (E) is the number of ILPs per day; subpart (F) is the number of INPs per day; subpart (G) is the CLPR(CLP/LP) per day; subpart (H) is the CNPR(CNP/NP). All data are expressed as mean ± SEM. The final numbers of rats used are as follows. Control group, n = 7; sham group, n = 8; A␤1–40 group, n = 7. Significant differences *p < 0.05, **p < 0.001 compared with the control; # p < 0.05, ## p < 0.001compared with the sham.

and sequences of events [40–46]. The striatum has been reported to have a dominate in reward-related processes involving action control [47]. This region receives unilateral afferents from the hippocampus and has been proposed to be used in translating

information from the hippocampus into action [31,48,49]. Moreover, Schilman et al. found that pre-training striatal lesions had no effect on lever-pressing, whereas post-training striatal inactivation had a negative effect [50]. In addition, synaptic modification in the

330

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

striatum has been identified as the means by which lever-pressing memories are stored [51,52]. Accordingly, we inferred that the two regions work in parallel during acquisition, competing with each other. Subsequently, they become more integrated and interdependent. In any case, taking into account the impairment of contextual memory acquisition and retention [53–55], the injection of A␤1–40 into the CA1 region of the hippocampus did render the rats unable to process explicit information for guiding behavior rather than influencing the behavior per se. 4.2. Sensitivity to contingency degradation Extensive training can induce a shift in behavioral control from goal-directed actions, which are governed by A–O contingencies, to habits which are dependent on S–R relationships [56]. Goaldirected actions are controlled by their consequences, habits are formed according to antecedent stimuli. Previous studies have shown that a higher rate of response schedule generates goaldirected actions while habitual, stimulus-driven action has been characterized by lower response rates [57,58]. The long-term training programs conducted in Experiment 2 were based on the two-process theory. Cognitive flexibility in adjusting behavior to adapt to A–O contingency changes was assessed first. It is well known that animals cannot only encode a causal relationship between an action and its consequence, but can also detect changes in the consequences of their actions. After receiving a post-training intra hippocampal microinjection of A␤1–40 , gradually higher rate of response schedules were used to assess the capacity for A–O contingency renewal. We defined that rats could only earn a reward by finishing consecutive responses within 2 s with the FR2 schedule and 3 s with the FR4 schedule. The A␤1–40 group was insensitive to the contingency decline until the response ratio was raised to that of FR4. It appeared that the A␤1–40 -treated rats were unable to shift to the new A–O association. Consistent with this finding, rats with a hippocampal lesion were also insensitive to the degradation of A–O contingency [32]. They were rendered unable to distinguish the consequences after a series of consecutive lever pressings and showed a profound and enduring loss of efficiency. In addition, motivation has been defined as the mapping between outcome and its value [59]. The longer the interval between the response onset and the reward gain, the greater was the decline in the predicted reward value [60,61]. Especially during goal-driven tasks, with the declining incentive motivation, rats will stop responding if their performance is no longer paired with the instrumental outcome [62]. This explains why the A␤1–40 -treated rats exhibited an LPs reduction using FR4 schedule while the control and sham groups maintained motivation during contingency degradation. These results suggest that the A␤1–40 injection prevented the renewal of A–O contingency using the higher fixed ratio schedule but not the lower. Furthermore, a recent clinic study reported that changes in the emotional state as well as in goal-directed learning and cognition are core properties of the apathy syndrome in patients with AD [63]. Since the behavioral performance of A␤1–40 treated rats was analogous to apathy dementia in AD [63,64], we further proposed that goal-directed learning tasks could be employed as a potential means of testing for these types of dementia symptoms. 4.3. Ability to identify an S-R association (discrimination training and extinction) We further demonstrated the effects of A␤1–40 on stimulusdriven habit formation. The design of the two-color visual signal discrimination task used here was based on pre-acquired habit memory [65]. Importantly, in our new series of tasks, visual discrimination was not aided by spatial cues, thereby presenting

only the outcome prediction signals that would faster the habitual responding. Thus, the reward was no longer part of the S–R association, but merely strengthened it [66]. Moreover, a deficiency in declarative memory is the main feature of the early stages of AD [67,68]. The alternatively presented visual cue which implied a different task demand could also be classified as contextual [69,70]. Accordingly, it could be applied in testing declarative memory as a form of external cue. However, evidence has suggested that an intact hippocampus is not required for the acquisition of an S–R habit by experimental animals [71] but is only necessary in place learning [56]. In contrast, but consistent with the clinical hallmarks, our findings confirmed that A␤1–40 impaired the ability to encode a causal association between the visual cue and response. We know that the hippocampus is necessary for tasks that involve forming causal relation among cues and consequences [52] and that A␤1–40 impairs associative learning. Thus, the results suggest that disabling the processing of contextual information rendered the A␤1–40 -administrated rats unable to adjust their responses to the cue reflecting a correct association with the reward outcome. Furthermore, motivation has been argued as occupying the centre stage in the behavioral neuroscience of decision making, and specifically, instrumental action selection [59]. During the decisionmaking process, actions are chosen by comparing their relative cached values, rather than their consequent outcomes. The results of NPs (see Fig. 4B) showed that the exploration and interesting of the magazine were not impaired by A␤1–40 . Moreover, the incentive motivation of each reward was equivalent in habit-forming behavior. Although the A␤1–40 -treated rats responded at a significantly higher rate which similar to the hyperactivity seen in other lesion studies [34,72]. However, the rat with a hippocampal lesion did not display a deficit in discrimination learning and retention [65]. In addition, the spontaneous motor activity was not impaired by A␤1–40 , indicating no motor deficits in this animal model [73]. Hence the hyperactivity demonstrated that the A␤1–40 -treated rats continually attempted to obtain the reward in non-rewarded phase. Therefore, our findings proved that the failure in making a correct choice could be attributed to A␤1–40 -deposition-induced contextual information processing deficits rather than hyperactivity. The results obtained in discriminative extinction test further revealed a similar stimulus-driven memory process. Importantly, this test was carried out in the absence of the outcome to probe the nature of the memory. The rats had encoded the specific S–R associations and were able to use them to guide their performance. The extinction test is used to examine previously learned associations without contamination by new learning [59,73]. Although all subjects, including the A␤1–40 -injected, were resistant to extinction [74] and lever pressing diminished within the time course, the A␤1–40 -administrated rats exhibited a poorer memory of the relationship between the visual signal and the specific response without the interference of the reward substances. On one hand, this demonstrated that rewards could facilitate instrumental learning while on the other hand, it highlighted the role of failure in suppressing the non-rewarded response in the A␤1–40 -induced signal discrimination deficit. 4.4. Neurotransmitter changes in the hippocampus Finally, we tested neurotransmitter levels in the hippocampus after the behavioral procedures were completed. Most learning situations require combinatorial participation of many neural systems. Moreover, neurotransmitters modulate learning and memory by altering the balance of the contributions made by different neural systems. The results (see Table 4) revealed which extracellular neurotransmitters might be involved in the mediation of RDIL after A␤1–40 administration. The decline in the Ach and DA level were found in A␤1–40 -treated rats. AD is characterized by

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

331

Table 4 Alteration of neurotransmitter levels in hippocampus of post-training surgically manipulated animals after the behavior procedure. The data are expressed as mean ± SEM (␮g/g). Groups

n

DA

NE

Glu

GABA

Ach

5-HT

Control group Sham group A␤1–40 group

7 8 7

0.0816 ± 0.0105 0.1095 ± 0.0243 0.0498 ± 0.0106a , b

211.55 ± 14.77 209.89 ± 25.48 237.15 ± 10.34

577.16 ± 35.47 557.30 ± 42.98 654.95 ± 29.17

87.61 ± 5.94 90.06 ± 4.59 91.77 ± 4.78

1.5062 ± 0.122 1.3550 ± 0.077 1.0207 ± 0.083a , b

0.4903 ± 0.0176 0.5119 ± 0.0319 0.4913 ± 0.0305

DA, dopamine; NE, norepinephrine; Glu, glutamate; GABA, gamma aminobutyric acid; Ach, acetylcholine; 5-HT, 5-serotonin. a p < 0.05, compared with control group. b p < 0.05, compared with sham group.

a decreased synapse density in the hippocampus, and synapse loss is the strongest anatomical correlate of the degree of clinical impairment [75]. The dysfunction and loss of cholinergic neurons are among the pathological features seen in the pathogenesis of AD [68]. It has been reported that Ach release and synthesis are depressed in the presence of A␤1–40 [76–79]. However previous results have suggested that a decline in the Ach level would block the acquisition of competence in the RDIL task [80,81] rather than alter the expression of the FR response [82]. Moreover, extracellular Ach can promote relational processing of cue information by the hippocampus [83]. The evidences above have illustrated the important role of Ach in RDIL tasks. Furthermore, DA release has been implicated in Pavlovian and instrumental learning [22,62], but its precise contribution to incentive and instrumental learning remains a subject of intense debate. More recently, it has been suggested that DA contributes to the arousal of incentive motivation rather than aiding learning of how to obtain the reward [27,84]. Contrary to this notion, a chronic DA decline has been reported to have no affect on the sensitivity of lever pressing behavior to decrements in the reward value [66,85]. In an animal model study, it was suggested that reward-directed behavior depends on the interaction between hippocampal inputs and neurons in the ventral striatum which is modulated by DA [86]. Our observations indicated the active role of DA in regulation of A␤1–40 -induced RDIL deficits.

5. Conclusion Taken together with all the behavioral results of the present experiments, we demonstrated that focal hippocampal microinjection of A␤1–40 rendered rats unable to process new cue/context information into a causal relation, rather than acting directly on the operant action per se. Although the injected A␤1–40 did not directly influence the activity, it did prevent the information from been translated into action. We have shown here that the procedure employed in our research could be used to accurately assess cognitive deficits in RDIL induced by A␤1–40 . In addition, studies using instrument-based tasks provide greater insight into information processing and behavior adaption involving in sophisticated conditioned reflexes. Furthermore, taking into account that the protocols satisfy animal welfare requirements, we predict that this series of instrumental learning tasks has the potential to become widely used in dementia research. Moreover, the neurotransmitter results suggest that multiple neural signaling is involved in the regulation of goal-directed cognitive deficits in AD dementia. This should be taken into consideration when searching for new treatment strategies for AD.

Acknowledgements This work was supported by the National Natural Science Foundation of China (30973888) and the Ministry of Science and Technology of China (2009ZX09502-014, 2011CB711000).

References [1] American Psychiatric Association. DSM-IV-TR: Diagnostic and Statistical Manual of Mental Disorders. Arlington, VA, USA: American Psychiatric Press Inc.; 2000. [2] Cummings JL. Alzheimer’s disease. New England Journal of Medicine 2004;351:56–67. [3] Malin DH, Crothers MK, Lake JR, Goyarzu P, Plotner RE, Garcia SA, et al. Hippocampal injections of amyloid beta-peptide 1–40 impair subsequent one-trial/day reward learning. Neurobiology of Learning and Memory 2001;76:125–37. [4] Cantarella G, Uberti D, Carsana T, Lombardo G, Bernardini R, Memo M. Neutralization of TRAIL death pathway protects human neuronal cell line from beta-amyloid toxicity. Cell Death and Differentiation 2003;10:134–41. [5] Han M, Liu Y, Tan Q, Zhang B, Wang W, Liu J, et al. Therapeutic efficacy of stemazole in a beta-amyloid injection rat model of Alzheimer’s disease. European Journal of Pharmacology 2011;657:104–10. [6] Yamaguchi Y, Miyashita H, Tsunekawa H, Mouri A, Kim HC, Saito K, et al. Effects of a novel cognitive enhancer, spiro[imidazo-[1,2-a]pyridine-3,2-indan]2(3H)-one (ZSET1446), on learning impairments induced by amyloid-beta 1–40 in the rat. Journal of Pharmacology and Experimental Therapeutics 2006;317:1079–87. [7] Itkin A, Dupres V, Dufrene YF, Bechinger B, Ruysschaert JM, Raussens V. Calcium ions promote formation of amyloid beta-peptide (1–40) oligomers causally implicated in neuronal toxicity of Alzheimer’s disease. PLoS One 2011;6:e18250. [8] Stepanichev MY, Moiseeva YV, Lazareva NA, Onufriev MV, Gulyaeva NV. Single intracerebroventricular administration of amyloid-beta (25–35) peptide induces impairment in short-term rather than long-term memory in rats. Brain Research Bulletin 2003;61:197–205. [9] Benedikz E, Kloskowska E, Winblad B. The rat as an animal model of Alzheimer’s disease. Journal of Cellular and Molecular Medicine 2009;13:1034–42. [10] Zhou J, Zhou L, Hou D, Tang J, Sun J, Bondy SC. Paeonol increases levels of cortical cytochrome oxidase and vascular actin and improves behavior in a rat model of Alzheimer’s disease. Brain Research 2011;1388:141–7. [11] Ruan CJ, Li Z, Zhang L, Chen DH, Du GH, Sun L. Protective effects of trans2,4-dimethoxystibene on cognitive, impairments induced by Abeta (25–35) in, hypercholesterolemic rats. Brain Research Bulletin 2010;82:251–8. [12] Bagheri M, Joghataei MT, Mohseni S, Roghani M. Genistein ameliorates learning and memory deficits in amyloid beta((1–40)) rat model of Alzheimer’s disease. Neurobiology of Learning and Memory 2011;95:270–6. [13] Yang H, Qiao H, Tian X. Proteomic analysis of cerebral synaptosomes isolated from rat model of Alzheimer’s disease. Indian Journal of Experimental Biology 2011;49:118–24. [14] Li J, Wang YJ, Zhang M, Fang CQ, Zhou HD. Cerebral ischemia aggravates cognitive impairment in a rat model of Alzheimer’s disease. Life Sciences 2011;89:86–92. [15] Shi X, Lu X, Zhan L, Liu L, Sun M, Gong X, et al. Rat hippocampal proteomic alterations following intrahippocampal injection of amyloid beta peptide (1–40). Neuroscience Letters 2011;500:87–91. [16] Nyakas C, Granic I, Halmy LG, Banerjee P, Luiten PG. The basal forebrain cholinergic system in aging and dementia. Rescuing cholinergic neurons from neurotoxic amyloid-beta42 with memantine. Behavioural Brain Research 2011;221:594–603. [17] Tsai FS, Cheng HY, Hsieh MT, Wu CR, Lin YC, Peng WH. The ameliorating effects of luteolin on beta-amyloid-induced impairment of water maze performance and passive avoidance in rats. American Journal of Chinese Medicine 2010;38:279–91. [18] Lee I, Solivan F. The roles of the medial prefrontal cortex and hippocampus in a spatial paired-association task. Learning and Memory 2008;15:357–67. [19] Hunsaker MR, Lee B, Kesner RP. Evaluating the temporal context of episodic memory: the role of CA3 and CA1. Behavioural Brain Research 2008;188: 310–5. [20] Brembs B, Lorenzetti FD, Reyes FD, Baxter DA, Byrne JH. Operant reward learning in aplysia: neuronal correlates and mechanisms. Science 2002;296:1706–9. [21] Lelos MJ, Thomas RS, Kidd EJ, Good MA. Outcome-specific satiety reveals a deficit in context-outcome, but not stimulus- or action-outcome, associations in aged Tg2576 mice. Behavioral Neuroscience 2011;125:412–25. [22] Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998;37:407–19.

332

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333

[23] Berridge KC, Kringelbach ML. Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology 2008;199:457–80. [24] Rapanelli M, Frick LR, Zanutto BS. Learning an operant conditioning task differentially induces gliogenesis in the medial prefrontal cortex and neurogenesis in the hippocampus. PLoS One 2011;6:e14713. [25] Mura E, Preda S, Govoni S, Lanni C, Trabace L, Grilli M, et al. Specific neuromodulatory actions of amyloid-beta on dopamine release in rat nucleus accumbens and caudate putamen. Journal of Alzheimer’s Disease 2010;19:1041–53. [26] Paxinos GWC. The rat brain in stereotaxic coordinates. New York: Academic Press; 1998. [27] D’Aquila PS. Dopamine on D2-like receptors reboosts dopamine D1-like receptor-mediated behavioural activation in rats licking for sucrose. Neuropharmacology 2010;58:1085–96. [28] Albert MS. Changes in cognition. Neurobiology of Aging 2011;32(Suppl. 1):S58–63. [29] Karantzoulis S, Galvin JE. Distinguishing Alzheimer’s disease from other major forms of dementia. Expert Review of Neurotherapeutics 2011;11: 1579–91. [30] Droes RM, van der Roest HG, van Mierlo L, Meiland FJ. Memory problems in dementia: adaptation and coping strategies and psychosocial treatments. Expert Review of Neurotherapeutics 2011;11:1769–82. [31] Lansink CS, Goltstein PM, Lankelma JV, McNaughton BL, Pennartz CM. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biology 2009;7:e1000173. [32] Reichelt AC, Lin TE, Harrison JJ, Honey RC, Good MA. Differential role of the hippocampus in response-outcome and context-outcome learning: evidence from selective satiation procedures. Neurobiology of Learning and Memory 2011;96:248–53. [33] Corbit LH, Balleine BW. The role of the hippocampus in instrumental conditioning. Journal of Neuroscience 2000;20:4233–9. [34] Cheung TH, Cardinal RN. Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats. BMC Neuroscience 2005;6:36. [35] Finger S, Green L, Tarnoff ME, Mortman KD, Andersen A. Nimodipine enhances new learning after hippocampal damage. Experimental Neurology 1990;109:279–85. [36] Lee I, Kesner RP. Time-dependent relationship between the dorsal hippocampus and the prefrontal cortex in spatial memory. Journal of Neuroscience 2003;23:1517–23. [37] Tonkiss J, Morris RG, Rawlins JN. Intra-ventricular infusion of the NMDA antagonist AP5 impairs performance on a non-spatial operant DRL task in the rat. Experimental Brain Research 1988;73:181–8. [38] Mulder AB, Tabuchi E, Wiener SI. Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation. European Journal of Neuroscience 2004;19:1923–32. [39] Balleine BW, O’Doherty JP. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 2010;35:48–69. [40] Rajji T, Chapman D, Eichenbaum H, Greene R. The role of CA3 hippocampal NMDA receptors in paired associate learning. Journal of Neuroscience 2006;26:908–15. [41] Jacobson TK, Gruenbaum BF, Markus EJ. Extensive training and hippocampus or striatum lesions: effect on place and response strategies. Physiology & Behavior 2012;105:645–52. [42] Wells AM, Lasseter HC, Xie X, Cowhey KE, Reittinger AM, Fuchs RA. Interaction between the basolateral amygdala and dorsal hippocampus is critical for cocaine memory reconsolidation and subsequent drug contextinduced cocaine-seeking behavior in rats. Learning and Memory 2011;18: 693–702. [43] Eckart MT, Huelse-Matia MC, Schwarting RK. Dorsal hippocampal lesions boost performance in the rat sequential reaction time task. Hippocampus 2012;22:1202–14. [44] Davidson TL, Kanoski SE, Chan K, Clegg DJ, Benoit SC, Jarrard LE. Hippocampal lesions impair retention of discriminative responding based on energy state cues. Behavioral Neuroscience 2010;124:97–105. [45] Okatan M. Correlates of reward-predictive value in learning-related hippocampal neural activity. Hippocampus 2009;19:487–506. [46] Smith DM, Mizumori SJ. Learning-related development of context-specific neuronal responses to places and events: the hippocampal role in context processing. Journal of Neuroscience 2006;26:3154–63. [47] Balleine BW, Doya K, O’Doherty J, Reward Sakagami M. Decision making in corticobasal ganglia networks. New York: Academy of Sciences; 2007. [48] Eichenbaum H. Hippocampus: cognitive processes and neural representations that underlie declarative memory. Neuron 2004;44:109–20. [49] Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. European Journal of Neuroscience 2005;22:513–23. [50] Schilman EA, Klavir O, Winter C, Sohr R, Joel D. The role of the striatum in compulsive behavior in intact and orbitofrontal-cortex-lesioned rats: possible involvement of the serotonergic system. Neuropsychopharmacology 2010;35:1026–39. [51] Pennartz CM, Ito R, Verschure PF, Battaglia FP, Robbins TW. The hippocampalstriatal axis in learning, prediction and goal-directed behavior. Trends in Neurosciences 2011;34:548–59. [52] Lovinger DM. Neurotransmitter roles in synaptic modulation, plasticity and learning in the dorsal striatum. Neuropharmacology 2010;58:951–61.

[53] Luu TT, Pirogovsky E, Gilbert PE. Age-related changes in contextual associative learning. Neurobiology of Learning and Memory 2008;89:81–5. [54] Oler JA, Markus EJ. Age-related deficits in the ability to encode contextual change: a place cell analysis. Hippocampus 2000;10:338–50. [55] Rand-Giovannetti E, Chua EF, Driscoll AE, Schacter DL, Albert MS, Sperling RA. Hippocampal and neocortical activation during repetitive encoding in older persons. Neurobiology of Aging 2006;27:173–82. [56] Compton DM. Behavior strategy learning in rat: effects of lesions of the dorsal striatum or dorsal hippocampus. Behavioural Processes 2004;67: 335–42. [57] Dickinson A, Nicholas DJ, Adams CD. The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Journal of Experimental Psychology 1983;35:35–51. [58] Dickinson A, Balleine B. Actions and responses. The dual psychology of behaviour. In: Eilan N, McCarthy RA, editors. Spatial representation: problems in philosophy and psychology. Malden, MA, USA: Blackwell Publishers Inc.; 1993. p. 277–93. [59] Niv Y, Joel D, Dayan P. A normative perspective on motivation. Trends in Cognitive Sciences 2006;10:375–81. [60] Mazur JE. Choice, delay, probability, and conditioned reinforcement. Animal Learning and Behavior 1997;25(2):131–47. [61] Mazur JE, Biondi DR. Effects of time between trials on rats’ and pigeons’ choices with probabilistic delayed reinforcers. Journal of the Experimental Analysis of Behavior 2011;95:41–56. [62] Balleine BW. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiology & Behavior 2005;86:717–30. [63] Robert P, Onyike CU, Leentjens AF, Dujardin K, Aalten P, Starkstein S, et al. Proposed diagnostic criteria for apathy in Alzheimer’s disease and other neuropsychiatric disorders. European Psychiatry 2009;24:98–104. [64] Brown RG, Pluck G. Negative symptoms: the ‘pathology’ of motivation and goaldirected behaviour. Trends in Neurosciences 2000;23:412–7. [65] Broadbent NJ, Squire LR, Clark RE. Rats depend on habit memory for discrimination learning and retention. Learning and Memory 2007;14:145–51. [66] Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nature Reviews Neuroscience 2006;7:464–76. [67] Sabe L, Jason L, Juejati M, Leiguarda R, Starkstein SE. Dissociation between declarative and procedural learning in dementia and depression. Journal of Clinical and Experimental Neuropsychology 1995;17:841–8. [68] Auld DS, Kornecook TJ, Bastianetto S, Quirion R. Alzheimer’s disease and the basal forebrain cholinergic system: relations to beta-amyloid peptides, cognition, and treatment strategies. Progress in Neurobiology 2002;68: 209–45. [69] Kennedy PJ, Shapiro ML. Retrieving memories via internal context requires the hippocampus. Journal of Neuroscience 2004;24:6979–85. [70] White NM, McDonald RJ. Multiple parallel memory systems in the brain of the rat. Neurobiology of Learning and Memory 2002;77:125–84. [71] Gabriele A, Packard MG. Evidence of a role for multiple memory systems in behavioral extinction. Neurobiology of Learning and Memory 2006;85: 289–99. [72] Kim SM, Frank LM. Hippocampal lesions impair rapid learning of a continuous spatial alternation task. PLoS One 2009;4:e5494. [73] Nag S, Tang F, Yee BK. Chronic intracerebroventricular exposure to betaamyloid (1–40) impairs object recognition but does not affect spontaneous locomotor activity or sensorimotor gating in the rat. Experimental Brain Research 2001;136:93–100. [74] Williams JH, Gray JA, Sinden J, Buckland C, Rawlins JN. Effects of GABAergic drugs, fornicotomy, hippocampectomy and septal lesions on the extinction of a discrete-trial fixed ratio 5 lever-press response. Behavioural Brain Research 1990;41:129–50. [75] Shankar GM, Bloodgood BL, Townsend M, Walsh DM, Selkoe DJ, Sabatini BL. Natural oligomers of the Alzheimer amyloid-beta protein induce reversible synapse loss by modulating an NMDA-type glutamate receptor-dependent signaling pathway. Journal of Neuroscience 2007;27: 2866–75. [76] Stepanichev MY, Zdobnova IM, Zarubenko II, Moiseeva YV, Lazareva NA, Onufriev MV, et al. Amyloid-beta(25–35)-induced memory impairments correlate with cell loss in rat hippocampus. Physiology & Behavior 2004;80: 647–55. [77] Miguel-Hidalgo JJ, Cacabelos R. Beta-amyloid(1–40)-induced neurodegeneration in the rat hippocampal neurons of the CA1 subfield. Acta Neuropathologica 1998;95:455–65. [78] Gonzalo-Ruiz A, Sanz JM, Arevalo J, Geula C, Gonzalo P. Amyloid beta peptide-induced cholinergic fibres loss in the cerebral cortex of the rat is modified by diet high in lipids and by age. Journal of Chemical Neuroanatomy 2005;29:31–48. [79] Selkoe DJ. Alzheimer’s disease: genes, proteins, and therapy. Physiological Reviews 2001;81:741–66. [80] Izaki Y, Hori K, Nomura M. Dopamine and acetylcholine elevation on leverpress acquisition in rat prefrontal cortex. Neuroscience Letters 1998;258: 33–6. [81] Iso H, Ueki A, Shinjo H, Miwa C, Morita Y. Reinforcement enhances hippocampal acetylcholine release in rats: an in vivo microdialysis study. Behavioural Brain Research 1999;101:207–13. [82] Sharf R, Ranaldi R. Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats. Psychopharmacology 2006;184:87–94.

Z. Shi et al. / Behavioural Brain Research 234 (2012) 323–333 [83] Calandreau L, Trifilieff P, Mons N, Costes L, Marien M, Marighetto A, et al. Extracellular hippocampal acetylcholine level controls amygdala function and promotes adaptive conditioned emotional response. Journal of Neuroscience 2006;26:13556–66. [84] Berridge KC. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology 2007;191:391–431.

333

[85] Lex B, Hauber W. The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning. Neurobiology of Learning and Memory 2010;93:283–90. [86] Goto Y, Grace AA. Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nature Neuroscience 2005;8:805–12.