Delayed extinction and drive level effects: Septal self-stimulation compared with natural reward

Physiology and Behavior, Vol. 12, pp. 907-912. Brain Research Publications Inc., 1974. Printed in the U.S.A.

Delayed Extinction and Drive Level Effects: Septal Self-Stimulation Compared with Natural Reward ' R O G E R K. R. T H O M P S O N 2 A N D D O N A L D MCD W E B S T E R

Department o f Psychology, University o f Hawaii and Department o f Psychology, University o f Auckland, New Zealand

(Received 12 April 1973)

THOMPSON, R. K. R. AND D. MCD WEBSTER. Delayed extinction and drive level effects: Septal self-stimulation compared with natural reward. PHYSIOL. BEHAV. 12(6) 907-912, 1974. - Rats trained to bar press for septal intracranial reinforcement (ICR) did not show a time dependent decreasing resistance to extinction as reported for hypothalamic ICR rats [8]. This is inconsistent with the inclusive drive-decay theory interpretation of those results. In both conditions of immediate (no delay) and delayed (1 hr) extinction, bar press response rates did not differ significantly between septal ICR rats and nondeprived rats reinforced with 10% sucrose solution. However, for both immediate and delayed extinction, bar press response rates of deprived (23 hr) water reinforced rats were significantly higher (p<0.001) than bar press response rates of both septal ICR rats and 10% sucrose reinforced rats. This finding is consistent with an incentive theory of 1CR. Septal ICR rats

Resistance to extinction

Incentive theory

maintained on ad lib food (laboratory pellets) and water, and reinforced with 10% sucrose solution (ii) animals maintained on a d lib food and water, and reinforced intracranially by septal stimulation, and (iii) animals maintained on ad lib food on a 23 hr water deprivation scheduIe, reinforced with water.

E V I D E N C E for a time d e p e n d e n t decreasing resistance to e x t i n c t i o n consistent with drive-decay t h e o r y of intracranial r e i n f o r c e m e n t (ICR) [ l , 2, 3, 4] was r e p o r t e d for h y p o t h a l a m i c ICR rats by Quartermain and Webster [8]. The generality o f their results to septal ICR animals was tested in this e x p e r i m e n t using the same apparatus and procedure. Responses in e x t i n c t i o n and activity levels o f septal ICR rats were c o m p a r e d with water deprived rats b o t h when e x t i n c t i o n was i m m e d i a t e and when it was delayed for one hour. Behavioral differences b e t w e e n h y p o t h a l a m i c ICR animals and c o n v e n t i o n a l l y reinforced animals have also been explained in terms o f an incentive t h e o r y [9]. One assumption o f the incentive t h e o r y is that p e r f o r m a n c e of nondeprived (low drive) animals reinforced with a highly palatable reward delivered with minimal delay will duplicate typical ICR induced behavior [ 5 ] . This assumption was also e x a m i n e d in this experiment. Activity levels and effect o f delay b e t w e e n acquisition and e x t i n c t i o n were c o m p a r e d in 3 groups: (i) animals

METHOD

Animals Animals were 30 naive male rats 1 2 0 - 1 5 0 days old o f m i x e d albino-hooded strain from the c o l o n y maintained by the Auckland University D e p a r t m e n t of Psychology. They weighed b e t w e e n 2 7 5 - 3 5 0 g at the time of implantation. The e x p e r i m e n t e m p l o y e d 6 groups o f 5 animals (Table I).

Preparation o f A nimals Before starting the experiment, the 2 groups scheduled for water r e i n f o r c e m e n t were adapted to a 23 hr deprivation schedule for 10 days, by which t i m e their daily intake

The experiment was performed in part fulfillment of M.A. degree requirements by the first author, whilst at the University of Auckland. aReprint requests should be sent to the first author, University of Hawaii, Kewalo Marine Mammal Facility, 1129 Ala Moana Blvd., Honolulu, Hawaii 96814. 907

908

THOMPSON AND WEBSTER

TABLE 1 EXPERIMENTAL GROUPS*

Extinction Treatment Reinforcement Condition

No Delay

One Hour Delay

I.C.R.

Group !

Group II

Water

Group lII

Group IV

Sucrose

Group V

Group VI

e x t e n d e d 2.5 cm and 3.0 cm f r o m the floor. To m a k e response t o p o g r a p h y b e t w e e n ICR, w a t e r and sucrose r e i n f o r c e d groups as similar as possible, the stainless steel lever i n c o r p o r a t e d a d r i n k i n g cup (1.0 cm in dia. and 0.5 cm deep), o n its u p p e r surface. A bar press activated a c o u n t e r a n d c u m u l a t i v e r e c o r d e r a n d resulted in a s t i m u l u s of 0.5 sec in d u r a t i o n , regardless of the d u r a t i o n of bar presses. Water or sucrose s o l u t i o n was p r e s e n t e d via a solenoid valve ( c o n t r o l l e d by G r a s o n - S t a d l e r p r o g r a m m i n g e q u i p m e n t ) w h i c h delivered 0.025 ml in 0.5 sec following o p e r a t i o n of the lever. O p e r a t i o n of the same lever could alternatively deliver brain s t i m u l a t i o n for 0.5 sec to the ICR r e i n f o r c e d groups. Procedure

*For all groups N = 5 h a d stabilized. The 2 groups s c h e d u l e d for sucrose reinf o r c e m e n t ( s o l u t i o n c o n c e n t r a t i o n 10% by weight), were m a i n t a i n e d o n ad lib w a t e r a n d food ( l a b o r a t o r y pellets) t h r o u g h o u t the e x p e r i m e n t a l period. In the case o f t h e 2 groups s c h e d u l e d for septal ICR r e i n f o r c e m e n t , s t i m u l a t i n g electrodes ( m o n o p o l a r , acrylic insulated, stainless steel 1 2 1 / 6 intestinal straight suture needles, b a r e d 0.5 m m at the tip), were c h r o n i c a l l y imp l a n t e d bilaterally w i t h s t e r e o t a x i c guidance u n d e r p e n t o b a r b i t o l ( 4 0 m g / k g ) anesthesia. The septally a i m e d electrodes were i m p l a n t e d 2.0 m m a n t e r i o r to bregma, 1.5 m m lateral to the m i d l i n e b e t w e e n b r e g m a a n d l a m b d a , 10 ° 20' f r o m t h e vertical, a n d were lowered 5.6 m m i n t o the b r a i n along t h e plane o f entry. C o o r d i n a t e s were d e t e r m i n e d f r o m t h e atlas o f Pellegrino and C u s h m a n [ 1 0 ] . D e n t a l c e m e n t was used to a t t a c h t h e e l e c t r o d e a s s e m b l y to the a n i m a l ' s skull. T h e skin was s u t u r e d a r o u n d the e l e c t r o d e unit, t h e w o u n d covered with a n t i s e p t i c dusting p o w d e r (Tolid, Bayer), and the animal given 0.1 ml penicillin ( B e n a p e n , Glaxo), i n t r a m u s c u l a r l y . Animals were allowed b e t w e e n 7 to 10 days r e c o v e r y before b e i n g p r e t e s t e d . All animals were r u n s t i m u l a t e d via t h e left electrode. Each i m p l a n t e d a n i m a l was placed in t h e e x p e r i m e n t a l c h a m b e r and t h e e l e c t r o d e leads a t t a c h e d . The s t i m u l a t i o n level was a d j u s t e d u n t i l the a n i m a l s h o w e d signs o f sniffing a n d increased e x p l o r a t i o n . T h e n the a n i m a l was s h a p e d to b a r press. T h e s t i m u l u s was 0.5 second of 50 Hz sine wave c u r r e n t delivered b e t w e e n one o f the electrodes and an i n d i f f e r e n t wire loop laid o n the surface o f t h e skull. The c u r r e n t was regulated b y a m i c r o p o t e n t i o m e t e r used as a voltage divider a n d was m o n i t o r e d by an oscilloscope and an a.c. microa m p m e t e r in series with t h e animal. F l u c t u a t i o n s in b r a i n resistance were m i n i m i z e d by a 1.8 meg o h m resistor in series with t h e animal. A ppara tus

The a p p a r a t u s was the same as t h a t used in the Quartermain and Webster s t u d y [ 8 ] . An e x p e r i m e n t a l c h a m b e r with d i m e n s i o n s of 4 0 cm x 25 cm x 23 cm high was used t h r o u g h o u t the e x p e r i m e n t . T h r e e sides a n d lid were Plexiglas. The floor-grid i n c o r p o r a t e d a s t a b i l i m e t e r w i t h a p e n d u l u m t r a n s d u c e r f r o m w h i c h c o n t a c t s , via a sensitom e t e r circuit, could be r e c o r d e d o n a c o u n t e r a n d a cumulative r e c o r d e r as an activity record. The f o u r t h side or back of the c h a m b e r was stainless steel f r o m w h i c h a lever

T h e p r o c e d u r e followed t h a t used in the Q u a r t e r m a i n a n d Webster study. All animals on the 3 r e i n f o r c e m e n t c o n d i t i o n s were given 10 days of acquisition t r a i n i n g in w h i c h t h e y gave a t o t a l of 1,000 c o n t i n u o u s r e i n f o r c e d responses; 100 responses being m a d e on the first 8 days, 150 o n the n i n t h a n d 50 on the t e n t h and final day. At the end of 10 days of a c q u i s i t i o n training, animals were r a n d o m l y assigned to groups s c h e d u l e d for the 1 h r delay a n d n o delay t r e a t m e n t s . Cumulative records k e p t of response rates d e t e r m i n e d t h a t t h e r e were n o significant differences b e t w e e n m e a n s and ' s t a n d a r d deviations of response rates in a c q u i s i t i o n for the 6 groups. I m m e d i a t e l y following acquisition-trials o n the t e n t h day, n o delay G r o u p s I, Ill and V were s u b j e c t e d to a half h o u r o f e x t i n c t i o n d u r i n g w h i c h r e i n f o r c e m e n t was withheld. Bar press responses were c u m u l a t i v e l y recorded. Each a n i m a l in the r e m a i n i n g 1 hr delay groups (II, IV and VI) following the 50 a c q u i s i t i o n trials o n the t e n t h day was i m m e d i a t e l y r e t u r n e d to its individual h o m e cage w h i c h was t h e n p r o m p t l y placed on the s t a b i l i m e t e r floor of the e x p e r i m e n t a l c h a m b e r for a p e r i o d of 1 hr. During this delay p e r i o d the a n i m a l ' s activity was recorded, every 30 sec for the first 10 min, and t h e n every 5 m i n for the r e m a i n d e r of t h e 1 hr period. The animals were t h e n ret u r n e d i m m e d i a t e l y to the e x p e r i m e n t a l c h a m b e r for a 1/2 h r e x t i n c t i o n p e r i o d d u r i n g w h i c h bar press responses were r e c o r d e d as for the 3 n o delay groups (I, III and V). Histology

At the end o f t h e e x p e r i m e n t , septal e l e c t r o d e sites were verified for each i m p l a n t e d rat (Fig. 1) e x c e p t for animals 09 ( G r o u p I) a n d 02 ( G r o u p II) w h o s e brains were accidentally d a m a g e d b e y o n d salvage. Initial i n s p e c t i o n , p r i o r to t h e i r loss, however, i n d i c a t e d septally located e l e c t r o d e sites a n d since t h e i r a c q u i s i t i o n rates did n o t differ f r o m the o t h e r ICR animals t h e i r results were i n c l u d e d in the data analysis. RESULTS Table 2 shows m e a n n u m b e r o f responses m a d e by each group during e x t i n c t i o n . Animals in w a t e r r e i n f o r c e d groups (III and IV) gave significantly m o r e responses in e x t i n c t i o n t h a n t h o s e in the o t h e r 4 groups, (I a n d II; V and VI). T h e r e is n o significant difference b e t w e e n these 4 groups (I a n d II; V and VI) in the n u m b e r o f r e s p o n s e s m a d e d u r i n g e x t i n c t i o n . A difference of only 3 responses b e t w e e n the t w o c o n d i t i o n s of delay exists for the ICR r e i n f o r c e d animals ( G r o u p s I and II). However, t h e r e were

ICR IN T H E R A T

909

SUBJECT '~ OI • 03 • 06 • 16

°17

SUBJECT • 04 • 20 °21 FIG. 1. Drawings showing septal electrode sites.

910

THOMPSON AND WEBSTER greater differences in frequency of responding between sucrose (Groups V and VI) and water reinforced animals (Groups III and IV). Sucrose reinforced animals under the 1 hr delay c o n d i t i o n (Group VI) made fewer responses during e x t i n c t i o n than the sucrose reinforced animals under e x t i n c t i o n w i t h o u t delay (Group V). The water reinforced animals under 1 hr delay c o n d i t i o n (Group IV) made more responses during extinction than the water reinforced animals under n o delay extinction (Group III). Analysis of variance [ 10] indicated that the interaction between types of r e i n f o r c e m e n t (ICR vs water; ICR vs sucrose; and water vs sucrose) and delay of extinction (no delay vs one h o u r delay) was n o t significant. That is, differences in frequency of responding during extinction, among animals under the two conditions of delay, were not significant within any of the three r e i n f o r c e m e n t conditions.

TABLE 2 MEAN NUMBER OF RESPONSES DURING EXTINCTION

Reinforcement Condition

Extinction Treatment No Delay One Hour Delay

Septal ICR

(I)

72

(II)

75

Water

(III)

178

(IV)

206

Sucrose

(V)

78

(VI)

47

I

190

re-

No

DELAY

180

SEPTAL ICR ( I ) ~ WATER ('rn) ~

170 160 150 140 130 12o 11o loo 90 80

SUCROSE

(327) "-

1 HOUR

-"

DELAY

(11") O (~r) 0

O 0

('3ZI)

A

A

70 ~ 60 N 50 ~

40

"

-

I

3o 20 10

I

I

I

I

1

2

3

4

I

I

I

I

I

5 6 MINUTES

7

8

9

10

FIG. 2. Mean cumulative responses in extinction for the 6 experimental groups during the first 10 min of the half hour extinction period.

ICR IN THE RAT

911

Significant (F(2, 24) = 18, p<0.001 ) reinforcer effects were found. Tests of simple effects showed that ICR and water reinforced groups differed significantly (F = 18, p<0.001) for both no delay and 1 hr delay conditions. ICR reinforced animals (Groups I and II) made significantly fewer responses during extinction than water reinforced animals (Groups III and IV). The same significant difference held between water and sucrose reinforced groups for both no delay and one hour delay conditions. Water reinforced animals (Groups III and IV) made significantly more responses during extinction than sucrose reinforced animals (Groups V and VI). However, the difference in frequency of responding during extinction between ICR and sucrose reinforced groups was not significant for either condition of delay. Figure 2 shows the cumulative responses obtained from the 6 groups during the first ten minutes of extinction. Comparison of response rates shows that both water reinforced groups (III and IV) responded at higher rates than the other 4 groups (I, II, V and VI). Of interest is the similarity of response rates of ICR and sucrose reinforced groups, in particular Groups I and V. The slowest extinction response rate was obtained from the sucrose reinforced, I hr delay group (VI). The most rapid response rate was obtained from the water reinforced, 1 hr delay group (II). Figure 3 compares mean number of stabilimeter transducer contacts made in 5 rain blocks during the 1 hr delay before extinction in Groups II, IV and VI. It must be pointed out that comparisons of activity levels can be made only within the 3 reinforcement conditions and not between them. This is because the stabilimeter should not

have been altered following initial adjustment at the start of the experiment. However, alteration to the pendulum unfortunately occurred during the course of the experiment when equipment had to be moved into a new building. By virtue of its design it was not possible to recalibrate the pendulum precisely. Bearing this limitation in mind, activity measures over 1 hr within each group show that activity decreased to some extent for all three reinforcement conditions, but more irregularly in the ICR group. DISCUSSION Septal ICR animals in which extinction was delayed for 1 hr did not make fewer responses in extinction than septal ICR animals in which extinction was immediate. Septal ICR animals do not show the time dependent decreasing resistance to extinction demonstrated with hypothalamic ICR animals using the same apparatus and procedure. Septal ICR animals also do not show a decreased activity level immediately following stimulation reported for hypothalamic ICR animals. The results of the present study, then, do not allow generalization of Quartermain and Webster's findings with hypothalamic ICR animals to septal ICR animals. This suggests that their results should not be interpreted solely in terms of drive-decay theory. Interestingly enough Trowill, Panksepp, and Gandleman [9] in the face of such data as that obtained by Quartermain and Webster do concede that reinforcement-derived motivation does decay with time in the absence of responding. This decay, however, is believed to be obscured in most

SEPTAL ICR (Tr} I,,,Z

60

,-,-

50-

~

48-

Z

F-

.,.J

p-

WATER (TV') S UC ROSE ("a'r)

,

302010I

I

5

10

I 15

I I I I I 20 25 30 35 40 MINUTES OF DELAY PERIOD

I 45

I 50

I 55

I 60

FIG. 3. Mean number of stabilimeter transducer contacts made in 5 min blocks during the 1 hr delay before extinction in Groups If, IV and VI.

912

T H O M P S O N AND W E B S T E R

studies using natural reinforcers because deprivation conditions provide persistent motivation. Such masking of decay appears to have occurred in the deprived, water reinforced group (IV) of the present study. There is, however, a definite, though nonsignificant, trend towards decay in the sucrose reinforced group (VI) which is consistent with such a view. The present findings support the assumption of the incentive t h e o r y that a highly palatable reinforcer (sucrose solution), delivered with minimal delay, and under low drive level (non deprivation) will duplicate ICR induced behavior. In both conditions of i m m e d i a t e (no delay) and delayed (1 hr) extinction, response rates did not significantly differ between septal ICR animals and nondeprived sucrose reinforced animals (Groups I, II, V, and VI). Deprived water reinforced animals under b o t h extinction conditions made significantly more responses than either

septal ICR or sucrose reinforced animals. These results were particularly evident in the first 10 rain of extinction. Septal ICR and sucrose reinforced animals in this study, however, do not show a decrease effect similar to that of h y p o t h a l a m i c ICR animals which should be the case if negative contrast is relevant, as suggested by Panksepp and Trowill [ 6 ] , as an alternative explanation for the rapid extinction found after removal of h y p o t h a l a m i c stimulation. In general, the results of this study support an incentive theory of ICR which assumes that behavioral differences obtained between ICR and natural (food and water) reinforcers are largely procedural artifacts; in this case deprivation level. Drive level defined in terms of deprivation level is an i m p o r t a n t variable that must be controlled in comparisons between conventional r e i n f o r c e m e n t and ICR.

REFERENCES 1. Deutsch, J. A. Learning and electrical self-stimulation of the brain. J. theor. Biol. 4: 193-214, 1963. 2. Deutsch, J. A. and C. I. Howarth. Some tests of a theory of intracranial self-stimulation. Psychol. Rev. 70: 444-460, 1963. 3. Gallistel, C. R. lntracranial stimulation and natural reward: Differential effects of trial spacing. Psychon. Sci 9" 167-168, 1967. 4. Howarth, C. I. and J. A. Deutsch. Drive decay: The cause of fast 'extinction' of habits learned for brain stimulation. Science 137: 35-36, 1962. 5. Panksepp, J. and J. A. Trowill. Intraoral self-injection: II. The simulation of self-stimulation phenomenon with a conventional reward. Psychon. Sci. 9: 407-408, 1967.('o).

6. Panksepp, J. and J. A. TrowiU. Positive and negative contrast e f f e c t s with hypothalamic reward. Physiol. Behav. 4: 173-175, 1969. 7. Pellegrino, L. J. and A. J. Cushman. A Stereotaxic Atlas of the RatBrain. New York: Appleton-Century-Crofts, 1967. 8. Quartermain, D. and D. Webster. Extinction following intracranial reward: The effect of delay between acquisition and extinction. Science 159: 1259-1260, 1968. 9. Trowill, J. A., J. Panksepp and R. Gandleman. An incentive model of rewarding brain stimulation. Psychol. Rev. 76: 264-281, 1969. 10. Winer, J. B. Statistical Principles in Experimental Design. New York: McGraw-Hill, 1962.