- Email: [email protected]
Effect of fixed-ratio thermal reinforcement on thermoregulatory behavior
Physiology and Behavior. Vol. 4, pp. 23-28. Pergamon Press, 1969. Printed in Great Britain
Effect of Fixed-Ratio Thermal Reinforcement on Thermoregulatory Behavior 1 H. J. C A R L I S L E
Department of Psychology, University of California, Santa Barbara, California 93105 (Received 12 June 1968)
CARLISI~,H. J. Effect of fixed-ratio thermalreinforcement on thermoregulatorybehavior. PHYSIOL.B~tAV. 4 (1) 23-28, 1969.--Rats and squirrel monkeys were tested on fixed-ratio schedules for radiant heat reinforcement in a cold environment. Monkeys worked at a rate determined by the schedule as well as the parameters of thermal reinforcement. Rats failed to demonstrate a behavioral regulation of body temperature on ratio schedules by fairing to maintain a rate of work great enough to insure a constant rate of reinforcement per unit time. Body temperature was maintained when continuous reinforcement was available, but decreased on ratios. A low ambient temperature or low body temperature facilitated rate of responding by the rat. Repeated daily tests produced a form of cold acclimatization in rats; behavioral response rate decreased, while the usual decrease in body temperature was attenuated as daily testing progressed. Thermoregulation
Behavior
Reinforcement
Acclimatization
A CONSIDERABLEBODY of behavioral literature has shown that intermittent reinforcement schedules generate dependable rates of responding controlled by the contingencies of reinforcement. Each reinforcement, usually food or water, is quantitatively small so that it does not appreciably change the motive state of the animal. F o o d intake or water balance are therefore not regulated under these circumstances. Thermal reinforcement offers the possibility of examining behavioral responding as an aspect of temperature regulation as well as a variable controlled by the contingent reinforcer. Previous work has shown that animals in a cold environment will maintain body temperature by working at steady and dependable rates if each response provides a burst of radiant heat [1, 3, 11, 12]. The present study is an examination of the use of thermal reinforcement on fixed-ratio schedules. Two questions are of concern. First, can intermittent thermal reinforcement be used to sustain behavior in the same manner as conventional, nutritive reinforcers? Second, to what extent will an animal regulate body temperature by working for intermittent heat? The data are based primarily on rats; two squirrel monkeys are included for comparative purposes.
Hypothalamus
supplemented with oranges for monkeys. Water was freely available.
Apparatus The test cages and apparatus have been described previously [3]. These consisted essentially of hardware-cloth enclosures with Plexiglas rod flooring. Depression of a Plexiglas lever activated red-bulb infrared heat lamps mounted to the sides of the cage and focused in front of the lever. The distance from the center of the cage to the lamps was 7-10 in. Two to four lamps, rated at either 250 or 375 W, were used depending on the desired intensity. A variable-voltage transformer regulated the wattage output of the lamps and a timer controlled the duration of reinforcements. Standard reinforcements, unless noted otherwise, were 300 W for 3 see (rats) and 750 W for 3 sec (monkeys). The test cages were placed in a cold chamber, the temperature of which could be adjusted to any level between --5 and 10°C controllable to ~2°C. An ambient temperature of --5°C was typically used for rats and 0°C for monkeys. Bar-presses and reinforcements were recorded on a cumulative recorder and a digital print-out counter.
METHOD
Subjects
Procedure
Seventeen adult, male rats of the Sprague-Dawley strain and two squirrel monkeys (Saimiri sciureus) were the subjects. Body weights ranged from 430-510 g for the rats and from 750-850 g for the monkeys. The animals were housed individually in cages at a room temperature of 23-25°C. The diet was a d libitum Purina pellets for rats, and Purina biscuits
Rats were always clipped of their fur with an Oster clipper the day prior to a test. A depilatory agent ("Neet") was used in two instances. Monkeys were tested with intact pelage. The subjects were trained initially to press the lever on a continuous reinforcement (CRF) schedule. Lever-presses during the reinforcement interval did not prolong the heat or
x This investigation was supported by PHS Research Grant No. MH-12414 from National Institute of Mental Health. The assistance of Marion Dalin and Tim Denman is gratefully acknowledged. 23
24
CARLISLE
provide a subsequent reinforcement. When the animal was proficient at working for heat on CRF, a fixed-ratio (FR) program was started. At least three 5-hr tests were given at weekly intervals; the ratio was increased gradually from F R 3 through F R 10 during these preliminary tests. Bar-presses during the reinforcement interval on F R schedules counted toward the requisite number for the following reinforcement. The subsequent testing procedure depended on the adjustment of the animal to the initial F R testing sequence. The usual procedure was to maintain a constant ambient temperature, with constant heat intensity and duration, while varying the ratio of reinforcement. In some cases, the parameters of reinforcement or ambient temperature were varied as well as the schedule. A test usually lasted 4--5 hr, but was longer in some instances. Rats and monkeys were tested once a week, except as noted otherwise. Rectal temperature (Tre) was obtained from rats using a small-animal thermistor probe inserted to a constant depth of 5 cm, and a Model 46 Yellow Springs Telethermometer. The temperature of the anterior hypothalamus (Thy) was obtained from two rats. The technique of chronic thermistor implantation and recording has been described previously [3]. A Leeds-Northrup recording potentiometer was used to provide a continuous measure of Thy during a test, while Tre was measured before and after a test. The behavior of an animal was considered to be stable if it worked at a consistent rate for heat, and did not show sporadic bursts of responding followed by prolonged pauses. The effectiveness of responding was determined by comparing the rate at which an animal obtained intermittent thermal reinforcement with its rate on CRF, and the level of body temperature that prevailed under both conditions. The animals with implanted hypothalamic thermistors were perfused at the completion of testing. The brains were removed, fixed in 10 Yoformalin, and sectioned at 40 ~. Cresylviolet stained sections showed the thermistors to be located 1.5-2.0 mm lateral to the midline at a level of A7.4 in the atlas of de Groot [7]. The ventral penetration was from 1-2 mm above the base of the brain.
those experimental procedures which facilitate or retard performance. Table 1 shows the average rate of responding and of obtaining reinforcement for three rats given weekly 5-hr tests. The animals had been tested weekly for two months prior to the collection of this data. The first hr of the test was a warm-up period on CRF; the subjects then worked on CRF, F R 5, 10, and 15 in sequence for 1 hr at each ratio. Rate of responding increased on the F R 5 schedule, but not sufficiently to maintain reinforcement rate. Rate of reinforcement falls as a function of the ratio in all cases. Average rectal temperature (Tre) was 36.9°C prior to the tests, and 34.6°C at the termination. Two tests were conducted for each rat with CRF available for the entire 5-hr session. Average Tre was 36.5°C prior to these tests, and 36.4°C at the end. The rats therefore maintained a relatively constant Tre during 5-hr tests on CRF, but failed to maintain reinforcement rate and consequently Tre on F R schedules. The data presented in Table 1 represent optimal performance under the specified conditions of testing. Six additional rats were tested extensively, but all showed a high incidence of sporadic responding on F R 5 and F R 10. Thus, rates of response for these animals were lower than those noted in Table 1. Three of these rats were tested for 3 months under standard conditions, but this procedure did not preclude the appearance of sporadic responding during each test. Three rats were tested under a variety of reinforcement conditions and ambient temperatures. Variations of reinforcement duration between 1-5 sec., or of reinforcement intensity between
A. CRF
FR$
i/"
Till =36.6 ff~ Till- 354 j
FRI$
F
/
,~
/
/
RESULTS
/,/
The Rat
d
Rats worked for heat on F R schedules, but failed to maintain a constant reinforcement rate even at low ratios. The consequence was a fall in body temperature except under special circumstances noted below. The data show, first, the conditions favoring optimal rate of responding, and secondly,
FRIO
,
i
/
/
t
I
~
/
/
,!
i
:
,f
,(
,'
i':
B. c.
4.8/2.9
,.s
54/11
~t!o
_ _
2 3/24
1111=363
TABLE 1 MEAN RESPONSE AND REINFORCEMENT RATES FOR THREE RATS DURING FOUR WEEKLY TFSTS AT - - 5 ° C WITH
REINFORCEMENTI~rr~r~srrmsOF 300 W ACTINOFOR 3 SEC
Schedule CRF FR 5 FR I0 FR 15
Rate/Min. 4- SEM Responses 6.71 -4- 0.635 10.98 4- 0.768 10.59 4- 0.788 7.86 4- 1.071
Reinforcements 4.10 + 0.312 2.07 4- 0.196 1.17 4- 0.144 0.52 4- 0.072
FIG. 1. Cumulative records of 5 hr tests at -- 5°C for Rats No. 7 (A) and No. 9 (B) under standard reinforcement conditions. The proportion specified under each schedule indicates the ratio of average rate of response per rain to average rate of reinforcement per min over the designated time period. Trl and Tr2 indicate rectal temperature at the beginning and end of each test, respectively. The oblique pips on the cumulative records indicate the onset of each reinforcement on FR schedules.
BEHAVIORAL TEMPERATURE REGULATION
25
200-600 W, did not result in maintenance of reinforcement rate on any F R schedule compared to C R F performance. That is, rats that responded sporadically on F R schedules under standard reinforcement conditions continued to do so with a wide range of variation in the parameters of reinforcement. Ambient temperature was varied between --5 and 5°C; optimal performance was obtained at --5 ° while the incidence of sporadic responding became more prevalent at higher temperatures. Two rats were treated with a depilatory cream to remove all vestiges of fur, but this also failed to effect an improvement in performance. Two diverse behavioral patterns are shown in Fig. 1. The top record shows an animal that responded at a high and consistent rate on CRF. Rate of responding did not vary appreciably on any schedule for this rat, and hence the rate of reinforcement decreased as the ratio increased. The lower part of the figure shows a typical pattern of sporadic responding on F R 5 and F R 10 for a different animal. Note that rate of response on C R F was low for this rat. The six sporadic rats worked at an average rate of 5.48 responses/min on C R F under standard conditions,
39
• RmS)ONSmI o RmaNFOXC|MI~vm
cm)
i
w|,
~o sr
. .:
aa ~..
0u
.. ..-
"
after approximately 7 hr without showing any improvement in performance, while two rats showed a slight increase in rate of obtaining heat. Figure 3 shows the test performance of one surviving rat. Note that Tre was 22°C at the end of the test, in spite of the improvement in performance during the later hours. The second survivor obtained an average of 2 reinforcements/min, and had a Tre of 30.6°C at the end of the test. A low body temperature may therefore facilitate rate of responding, but this effect is not invariant. Adaptation to the cold was observed if rats were tested daily, rather than weekly, on F R 5. Rate of responding decreased over a period of days under these circumstances, while the decrease in body temperature that is normally observed during a test was attenuated with repeated exposures. Figure 4 shows the records for the first and last of 12 daily 5-hr tests at --5°C for one rat. Note that Tre was elevated at the end of the session on the last day in spite of the paucity of behavioral work. Rate of reinforcement showed a progressive decline as the daily tests were repeated. Rectal temperature was 33.935.9°C during the initial eight tests, but consistently above
.
•
~
cv
ms
Mio
"-
•
" "
o.
""
.:[~
o
FIG. 2. Hypothalamic temperature (Thy)and behavioral responding on various schedules at -- 5°C for Rat No. 43. Rate of response and reinforcement are the average for the indicated preceding 10 min. Reinforcements of 200 W acting for 2 see were given during the first 2 hr, and 350 W for 2 see thereafter.
and obtained 3.81 reinforcements/min. These rates are lower than those noted in Table 1. Figure 2 presents a record of hypothalamic temperature (Thy) during a 6-hr test for one of the implanted rats. This animal showed the highest rate of response obtained from a rat on F R schedules• Hypothalamic temperature rose at the start of the test, and then stabilized at approximately 38°C on CRF. A marked increase in response rate occurred when the schedule was changed to F R 3, but this rate was not maintained, and Thy fell to 36.6°C. A low Thy was initially maintained on F R 5 by a high, although sporadic, response rate. This session was unusual in that the apparatus was disconnected for a 20-rain period; this procedure constitutes extinction. The rate of fall of Thy during extinction provides a measure of the rate at which body heat is lost in the absence of thermal reinforcement. The rate of fall of Thy during extinction was faster than during intermittent reinforcement. A normal Thy was rapidly established on CRF, but a heatdebt was reincurred on F R 5 and F R 10. Four rats were tested for a 10-hr period on F R 5 with 400 W-3 sec reinforcements to determine whether a decreasing body temperature would eventually facilitate rate of responding. These animals had shown sporadic responding during preliminary testing on F R 5 and F R 10. Two animals died
PR5
,O.our ,.,,
.....
_ ~
,,, =~7.o
U
/
:
FIG. 3. Cumulative record of a complete 10 hr test on FR 5 at --5°C for Rat No. 12. Reinforcements of 400 W for 3 sec. The lower record is a continuation of the top record. Average rates of response and reinforcement were 5/min, and 1/min, respectively.
26
CARLISLE
36°C after the ninth test. Adaptation as a consequence of repeated daily tests was noted in t w o additional animals. Figure 5 shows Thy during two sequential tests after acclimatization for one of these rats. The subject was at a neutral ambient temperature for the first 40--50 min of the tests. Ambient temperature was then lowered to 0°C within 10 rain. Note that Thy rises when ambient temperature is lowered, and remains at an elevated level in spite of the lack of behavioral responding. Food deprivation for 16 hr prior to the second test resulted in a more variable Thy and an improvement in rate of responding. Hence, an interference with metabolic resources, as would occur with food deprivation, can facilitate rate of responding in acclimatized rats.
The Monkey One monkey was tested on various F R schedules with constant heat duration and intensity. A single session is shown in Fig. 6 for this animal. Approximately 5 reinforcements/min were obtained on CRF, and 3/min on F R 15. Response and reinforcement rates became sporadic on F R 20. A response rate of 100/min would be required on F R 20 to maintain a constant reinforcement rate of 5/min. Sustained response rates of greater than 50/min were not observed in this monkey. A second monkey was trained to work on F R 5 and F R 10 with varying intensities, durations, and ambient temperatures.
=CRF FR5
x FRI5
I Respodses [
.
FR20
/~
TRI =36.6
/// ///
4o
Tm2=34,4
!
i!
30
.~ 20 ~ 10
~
o!
Test 12 .96/.~s
FRIO
50
Test 1
99/i.9
., FR5
TRI:36.8
Time
T:2:37.4
in
hours
FIG. 6. Response and reinforcements rates for Monkey I during a single test at 0°C under standard reinforcement conditions.
FIG. 4. Cumulative records of the first and last of 12 daily 5 hr tests on FR 5 at --5°C under standard reinforcement conditions for Rat No. 10.
Monkey D 0'%
3o.t 8.w
3°l
¢: 2 0 -
~
10-
0 37
I
J $00"
,,..
20
FR 5
I0
j n 1 2 Duration in secar~ls
FR 5
~ 3
Oi, 0
~ 5 TemDeroture in °C
E I0
FR 10
400" .....
~ .....
~ .....
~ .....
a .....
~ ....
~
-~6
300-
TIME IN HOURS 200-
ioc=~ o - - -
_ f
c
~oo-
T~Mt~AeRS-mMm
~o~
I00-
~ ,,oo0 200100O-
FIG. 5. Hypothalamic temperature (Thy) during two tests for acclimatized Rat No. 28. Records C and D are cumulative records of the last 2 hr of sessions A and B, respectively. Food deprivation for 16 hr preceded test B. Reinforcements of 375 W for 2 see.
FIG. 7. Average rate of response for Monkey D when the duration of reinforcement was varied on FR 5 and FR 10 during a single test (upper left) and when ambient temperature was varied during 3 separate tests (upper right) under the specified conditions. The cumulative record shows a segment of a session when both the ratio and the duration of reinforcement were doubled (indicated by the arrow) at 0°C with standard reinforcement intensities.
BEHAVIORAL TEMPERATURE REGULATION Figure 7 shows the data for this subject. The monkey worked at a rate that was dependent on variations in the duration of reinforcement and the FR schedule when intensity and ambient temperature were constant, as shown at the left of the figure. These data were collected during a single session during which the animal worked for 30 min under each of the indicated conditions. Rate of response was also dependent on variations in ambient temperature when intensity and duration of reinforcement were constant, as shown at the right of Fig. 7. These data were obtained from 3 separate tests at weekly intervals under the specified conditions. Rate of responding by this monkey produced a reinforcement rate that was comparable to that obtained on CRF. Monkeys therefore worked at a rate determined by the schedule as well as the parameters of reinforcement up to FR 15. Sporadic responding was not observed until an exorbitant response rate was required to maintain reinforcement rate. DISCUSSION
Rats will maintain body temperature by working for radiant heat under a variety of reinforcement and ambient temperature conditions on CRF, even though this schedule is not a simple situation since heat reinforcement lasts several seconds. Responses occurring while the lamps are off turn them on, but responses occurring during a reinforcement have no effect. Thus, rate of response is higher than rate of reinforcement on CRF [3, 11, 12]. Reinforcement rate and body temperature are not maintained on FR schedules, as noted in this study. Sporadic responding is frequently observed, indicating that the schedule exerts poor control over the behavior of the rat under these circumstances. Conditions favoring stable responsing are primarily a low ambient temperature, or low body temperature, and infrequent test sessions. The specific parameters of reinforcement, the duration and intensity of heat, do not influence rate of reinforcement on FR schedules in comparison to the rate observed on CRF, at least within the range of values used in this study. That is, the rat will work at a steady and consistent rate for radiant heat on CRF, but typically fails to respond appropriately when all conditions are held constant except the response contingency. Thus, a limited behavioral control can be exerted at the lowest response contingencies, but there is little evidence for behavioral regulation of body temperature. Removal of all vestiges of fur by a depilatory cream and repeated weekly tests are ineffective in improving performance. Repeated tests in close temporal succession, however, produce an effect analogous to acclimatization. It is therefore obvious that the rat has two responses for avoiding the cold, one behavioral and the other metabolic. Repeated cold exposures in close temporal sequence would seem to be a necessary condition for activation of the metabolic response. It is not clear why the rat performs poorly on intermittent heat-reinforcement schedules. One possibility is that the removal of hair alters thermoregulatory responsiveness [2]. The immediacy with which heat acts upon the subject may be a more plausible alternative; that is, there is no delay of reinforcement and no consummatory response with heat, as there would be with nutritive reinforcers. Inadequate responding has been noted with intermittent electrical self-stimulation of the brain, a case in which reinforcement is not only immediate, but central. Schedule performance can be improved in this case by making the response situation more analogous to that occurring with food reinforcement, and by providing
27 for the occurrence of a secondary reinforcer [9]. It should be noted that response conditions in the present study are similar to those in the self-stimulation situation. The secondary reinforcement argument, however, would not readily apply here since visual and auditory cues are associated with heat reinforcement. Activation of the heat lamps produces an increase in ambient illumination and a "Whirr" sound. In addition, the notions of secondary reinforcement and immediacy of reinforcement would not explain the observed differences between rats and monkeys. A more cogent explanation would seem to require a consideration of the metabolic responsiveness of these two species. A number of studies have noted an increased resistance to cold stress in acclimatized rats. Cottle and Carlson [5] found that rats undergo a major metabolic adjustment during the first 5-7 days of continued exposure to 5°C. Metabolic heat production increased initially, and was maintained at a high level during exposure. The ability to increase heat production by non-shivering means has been shown to occur only in coldadapted rats, since these animals, in contrast with warmadapted control rats, increased heat production when muscular activity was blocked by curare [6]. Revusky [10] has noted a temporal decrease in rate of obtaining heat by hairless mice tested for 6 hr daily for 30 consecutive days. Laties and Weiss [8] also have demonstrated that behavioral thermoregulation is a function of cold acclimatization. Rats that had lived at 2°C for a month waited for an extremely long time before beginning to respond regularly for radiant heat. This effect is evident in Figs 4 and 5 of the present study even though cold exposure was limited to 5 hr per day. The present results thus show that behavioral responses are sufficient to maintain body temperature when the rat is tested infrequently on CRF, but not on FR schedules. The metabolic response of the rat appears to predominate when the animal is tested daily on ratio schedules, and body temperature can be maintained in spite of an absence of behavioral responding under these conditions. The implication is that the metabolic response of the rat would have been preempted had reinforcement rate and body temperature been maintained on FR schedules. Cold acclimatization apparently does not occur when rats are tested at weekly intervals on FR schedules. Body temperature shows a consistent decrease during these tests, which may imply that de-acclimation has occurred during the intervening week. The squirrel monkey worked for heat at a rate determined by the schedule and the parameters of reinforcement. Responding was sporadic only under conditions which demanded response rates greater than 50/rain. The behavioral response of the monkey may imply that its metabolic responsiveness to cold stress is not an effective mechanism. Squirrel monkeys show a decrease in shivering and increased muscular coordination during prolonged cold exposure, indicating a form of adaptation. Few changes in the activities of cellular enzyme systems are noted, however, so that non-shivering thermogenesis, as it applies in the case of rodents, may not apply to the monkey [4]. Weiss and Laties [11] have reported that rats will work for heat on a variable interval 12-see schedule of reinforcement. Their animals survived 16-hr tests at a temperature of 2°C. This particular schedule makes reinforcement available at the average rate of 5/min, which is the same rate at which animals in the Weiss and Laties study obtained reinforcement on CRF. A time-dependent schedule which makes reinforcement available at a high rate would thus seem to be consistent with
28
CARLISLE
stable behavior. Rate of response, as noted in this study, is inadequately sustained by the contingencies of reinforcement with response-dependent schedules for the rat. Three conclusions may be derived from this study. First, intermittent heat reinforcement does not sustain behavioral responding of rats in a manner that is comparable to responding observed with nutritive reinforcers. Second, species differ-
ences are an important source of variance determining adjustment to an intermittent thermal reinforcement schedule. Squirrel monkeys perform consistently at ratios up to F R 15, while rats typically fail to adjust to ratios as low as F R 5. Third, the intertest interval is a critical variable for rats, since adaptation to the cold occurs with repeated, brief (5 hr) daily tests.
REFERENCES
1. Baldwin, B. A. and D. L. Ingram. Behavioural thermoregulation in pigs. PhysioL Behav. 2, 15-21, 1967. 2. Bligh, J., Inhibition of thermal polypnoea in the closely shorn sheep. J. PhysioL 168: 764-781, 1963. 3. Carlisle, H. J. Heat intake and hypothalamic temperature during behavioral temperature regulation. J. comp. physiol. Psychol. 61: 388-397, 1966. 4. Chaffee, R. R. J., S. M. Horvath, R. E. Smith and R. S. Welsh. Cellular biochemistry and organ mass of cold- and heatacclimated monkeys. Fedn Proc. 25: 1177-1181, 1966. 5. Cottle, W., and L. D. Carlson. Adaptive changes in rats exposed to cold. Caloric exchange. Am. J. PhysioL 178: 305-308, 1954. 6. Cottle, W. H. and L. D. Carlson. Regulation of heat proauction in cold adapted rats. Proc. Soc. exp. BioL Med. 92: 845-849, 1956.
7. deGroot, J. The rat forebrain in stereotaxic coordinates. Verb. K. Med. Akad. Wet., B. Natuurkunde 52: 1-40, 1959. 8. Laties, V. G., and B.Weiss. Behavior in the cold after acclimatization. Science N.Y. 131: 1891-1892, 1960. 9. Pliskoff, S. S., J. E. Wright and T. D. Hawkins. Brain stimulation as a reinforcer: intermittent schedules. J. exp. Analysis Behav. 8: 75-88, 1965. 10. Revusky, S. H. Cold acclimatization in hairless mice measured by behavioral thermoregulation. Psychonom. Sci. 6: 209-210, 1966. 11. Weiss, B. and V. G. Laties. Magnitude of reinforcement as a variable in thermoregulatory behavior. J. comp. physiol. Psych. 53: 603--608, 1960. 12. Weiss, B., and V. G. Laties. Behavioral thermoregnlation. Science N.Y., 133: 1338-1344, 1961.
Effect of Fixed-Ratio Thermal Reinforcement on Thermoregulatory Behavior 1 H. J. C A R L I S L E
Department of Psychology, University of California, Santa Barbara, California 93105 (Received 12 June 1968)
CARLISI~,H. J. Effect of fixed-ratio thermalreinforcement on thermoregulatorybehavior. PHYSIOL.B~tAV. 4 (1) 23-28, 1969.--Rats and squirrel monkeys were tested on fixed-ratio schedules for radiant heat reinforcement in a cold environment. Monkeys worked at a rate determined by the schedule as well as the parameters of thermal reinforcement. Rats failed to demonstrate a behavioral regulation of body temperature on ratio schedules by fairing to maintain a rate of work great enough to insure a constant rate of reinforcement per unit time. Body temperature was maintained when continuous reinforcement was available, but decreased on ratios. A low ambient temperature or low body temperature facilitated rate of responding by the rat. Repeated daily tests produced a form of cold acclimatization in rats; behavioral response rate decreased, while the usual decrease in body temperature was attenuated as daily testing progressed. Thermoregulation
Behavior
Reinforcement
Acclimatization
A CONSIDERABLEBODY of behavioral literature has shown that intermittent reinforcement schedules generate dependable rates of responding controlled by the contingencies of reinforcement. Each reinforcement, usually food or water, is quantitatively small so that it does not appreciably change the motive state of the animal. F o o d intake or water balance are therefore not regulated under these circumstances. Thermal reinforcement offers the possibility of examining behavioral responding as an aspect of temperature regulation as well as a variable controlled by the contingent reinforcer. Previous work has shown that animals in a cold environment will maintain body temperature by working at steady and dependable rates if each response provides a burst of radiant heat [1, 3, 11, 12]. The present study is an examination of the use of thermal reinforcement on fixed-ratio schedules. Two questions are of concern. First, can intermittent thermal reinforcement be used to sustain behavior in the same manner as conventional, nutritive reinforcers? Second, to what extent will an animal regulate body temperature by working for intermittent heat? The data are based primarily on rats; two squirrel monkeys are included for comparative purposes.
Hypothalamus
supplemented with oranges for monkeys. Water was freely available.
Apparatus The test cages and apparatus have been described previously [3]. These consisted essentially of hardware-cloth enclosures with Plexiglas rod flooring. Depression of a Plexiglas lever activated red-bulb infrared heat lamps mounted to the sides of the cage and focused in front of the lever. The distance from the center of the cage to the lamps was 7-10 in. Two to four lamps, rated at either 250 or 375 W, were used depending on the desired intensity. A variable-voltage transformer regulated the wattage output of the lamps and a timer controlled the duration of reinforcements. Standard reinforcements, unless noted otherwise, were 300 W for 3 see (rats) and 750 W for 3 sec (monkeys). The test cages were placed in a cold chamber, the temperature of which could be adjusted to any level between --5 and 10°C controllable to ~2°C. An ambient temperature of --5°C was typically used for rats and 0°C for monkeys. Bar-presses and reinforcements were recorded on a cumulative recorder and a digital print-out counter.
METHOD
Subjects
Procedure
Seventeen adult, male rats of the Sprague-Dawley strain and two squirrel monkeys (Saimiri sciureus) were the subjects. Body weights ranged from 430-510 g for the rats and from 750-850 g for the monkeys. The animals were housed individually in cages at a room temperature of 23-25°C. The diet was a d libitum Purina pellets for rats, and Purina biscuits
Rats were always clipped of their fur with an Oster clipper the day prior to a test. A depilatory agent ("Neet") was used in two instances. Monkeys were tested with intact pelage. The subjects were trained initially to press the lever on a continuous reinforcement (CRF) schedule. Lever-presses during the reinforcement interval did not prolong the heat or
x This investigation was supported by PHS Research Grant No. MH-12414 from National Institute of Mental Health. The assistance of Marion Dalin and Tim Denman is gratefully acknowledged. 23
24
CARLISLE
provide a subsequent reinforcement. When the animal was proficient at working for heat on CRF, a fixed-ratio (FR) program was started. At least three 5-hr tests were given at weekly intervals; the ratio was increased gradually from F R 3 through F R 10 during these preliminary tests. Bar-presses during the reinforcement interval on F R schedules counted toward the requisite number for the following reinforcement. The subsequent testing procedure depended on the adjustment of the animal to the initial F R testing sequence. The usual procedure was to maintain a constant ambient temperature, with constant heat intensity and duration, while varying the ratio of reinforcement. In some cases, the parameters of reinforcement or ambient temperature were varied as well as the schedule. A test usually lasted 4--5 hr, but was longer in some instances. Rats and monkeys were tested once a week, except as noted otherwise. Rectal temperature (Tre) was obtained from rats using a small-animal thermistor probe inserted to a constant depth of 5 cm, and a Model 46 Yellow Springs Telethermometer. The temperature of the anterior hypothalamus (Thy) was obtained from two rats. The technique of chronic thermistor implantation and recording has been described previously [3]. A Leeds-Northrup recording potentiometer was used to provide a continuous measure of Thy during a test, while Tre was measured before and after a test. The behavior of an animal was considered to be stable if it worked at a consistent rate for heat, and did not show sporadic bursts of responding followed by prolonged pauses. The effectiveness of responding was determined by comparing the rate at which an animal obtained intermittent thermal reinforcement with its rate on CRF, and the level of body temperature that prevailed under both conditions. The animals with implanted hypothalamic thermistors were perfused at the completion of testing. The brains were removed, fixed in 10 Yoformalin, and sectioned at 40 ~. Cresylviolet stained sections showed the thermistors to be located 1.5-2.0 mm lateral to the midline at a level of A7.4 in the atlas of de Groot [7]. The ventral penetration was from 1-2 mm above the base of the brain.
those experimental procedures which facilitate or retard performance. Table 1 shows the average rate of responding and of obtaining reinforcement for three rats given weekly 5-hr tests. The animals had been tested weekly for two months prior to the collection of this data. The first hr of the test was a warm-up period on CRF; the subjects then worked on CRF, F R 5, 10, and 15 in sequence for 1 hr at each ratio. Rate of responding increased on the F R 5 schedule, but not sufficiently to maintain reinforcement rate. Rate of reinforcement falls as a function of the ratio in all cases. Average rectal temperature (Tre) was 36.9°C prior to the tests, and 34.6°C at the termination. Two tests were conducted for each rat with CRF available for the entire 5-hr session. Average Tre was 36.5°C prior to these tests, and 36.4°C at the end. The rats therefore maintained a relatively constant Tre during 5-hr tests on CRF, but failed to maintain reinforcement rate and consequently Tre on F R schedules. The data presented in Table 1 represent optimal performance under the specified conditions of testing. Six additional rats were tested extensively, but all showed a high incidence of sporadic responding on F R 5 and F R 10. Thus, rates of response for these animals were lower than those noted in Table 1. Three of these rats were tested for 3 months under standard conditions, but this procedure did not preclude the appearance of sporadic responding during each test. Three rats were tested under a variety of reinforcement conditions and ambient temperatures. Variations of reinforcement duration between 1-5 sec., or of reinforcement intensity between
A. CRF
FR$
i/"
Till =36.6 ff~ Till- 354 j
FRI$
F
/
,~
/
/
RESULTS
/,/
The Rat
d
Rats worked for heat on F R schedules, but failed to maintain a constant reinforcement rate even at low ratios. The consequence was a fall in body temperature except under special circumstances noted below. The data show, first, the conditions favoring optimal rate of responding, and secondly,
FRIO
,
i
/
/
t
I
~
/
/
,!
i
:
,f
,(
,'
i':
B. c.
4.8/2.9
,.s
54/11
~t!o
_ _
2 3/24
1111=363
TABLE 1 MEAN RESPONSE AND REINFORCEMENT RATES FOR THREE RATS DURING FOUR WEEKLY TFSTS AT - - 5 ° C WITH
REINFORCEMENTI~rr~r~srrmsOF 300 W ACTINOFOR 3 SEC
Schedule CRF FR 5 FR I0 FR 15
Rate/Min. 4- SEM Responses 6.71 -4- 0.635 10.98 4- 0.768 10.59 4- 0.788 7.86 4- 1.071
Reinforcements 4.10 + 0.312 2.07 4- 0.196 1.17 4- 0.144 0.52 4- 0.072
FIG. 1. Cumulative records of 5 hr tests at -- 5°C for Rats No. 7 (A) and No. 9 (B) under standard reinforcement conditions. The proportion specified under each schedule indicates the ratio of average rate of response per rain to average rate of reinforcement per min over the designated time period. Trl and Tr2 indicate rectal temperature at the beginning and end of each test, respectively. The oblique pips on the cumulative records indicate the onset of each reinforcement on FR schedules.
BEHAVIORAL TEMPERATURE REGULATION
25
200-600 W, did not result in maintenance of reinforcement rate on any F R schedule compared to C R F performance. That is, rats that responded sporadically on F R schedules under standard reinforcement conditions continued to do so with a wide range of variation in the parameters of reinforcement. Ambient temperature was varied between --5 and 5°C; optimal performance was obtained at --5 ° while the incidence of sporadic responding became more prevalent at higher temperatures. Two rats were treated with a depilatory cream to remove all vestiges of fur, but this also failed to effect an improvement in performance. Two diverse behavioral patterns are shown in Fig. 1. The top record shows an animal that responded at a high and consistent rate on CRF. Rate of responding did not vary appreciably on any schedule for this rat, and hence the rate of reinforcement decreased as the ratio increased. The lower part of the figure shows a typical pattern of sporadic responding on F R 5 and F R 10 for a different animal. Note that rate of response on C R F was low for this rat. The six sporadic rats worked at an average rate of 5.48 responses/min on C R F under standard conditions,
39
• RmS)ONSmI o RmaNFOXC|MI~vm
cm)
i
w|,
~o sr
. .:
aa ~..
0u
.. ..-
"
after approximately 7 hr without showing any improvement in performance, while two rats showed a slight increase in rate of obtaining heat. Figure 3 shows the test performance of one surviving rat. Note that Tre was 22°C at the end of the test, in spite of the improvement in performance during the later hours. The second survivor obtained an average of 2 reinforcements/min, and had a Tre of 30.6°C at the end of the test. A low body temperature may therefore facilitate rate of responding, but this effect is not invariant. Adaptation to the cold was observed if rats were tested daily, rather than weekly, on F R 5. Rate of responding decreased over a period of days under these circumstances, while the decrease in body temperature that is normally observed during a test was attenuated with repeated exposures. Figure 4 shows the records for the first and last of 12 daily 5-hr tests at --5°C for one rat. Note that Tre was elevated at the end of the session on the last day in spite of the paucity of behavioral work. Rate of reinforcement showed a progressive decline as the daily tests were repeated. Rectal temperature was 33.935.9°C during the initial eight tests, but consistently above
.
•
~
cv
ms
Mio
"-
•
" "
o.
""
.:[~
o
FIG. 2. Hypothalamic temperature (Thy)and behavioral responding on various schedules at -- 5°C for Rat No. 43. Rate of response and reinforcement are the average for the indicated preceding 10 min. Reinforcements of 200 W acting for 2 see were given during the first 2 hr, and 350 W for 2 see thereafter.
and obtained 3.81 reinforcements/min. These rates are lower than those noted in Table 1. Figure 2 presents a record of hypothalamic temperature (Thy) during a 6-hr test for one of the implanted rats. This animal showed the highest rate of response obtained from a rat on F R schedules• Hypothalamic temperature rose at the start of the test, and then stabilized at approximately 38°C on CRF. A marked increase in response rate occurred when the schedule was changed to F R 3, but this rate was not maintained, and Thy fell to 36.6°C. A low Thy was initially maintained on F R 5 by a high, although sporadic, response rate. This session was unusual in that the apparatus was disconnected for a 20-rain period; this procedure constitutes extinction. The rate of fall of Thy during extinction provides a measure of the rate at which body heat is lost in the absence of thermal reinforcement. The rate of fall of Thy during extinction was faster than during intermittent reinforcement. A normal Thy was rapidly established on CRF, but a heatdebt was reincurred on F R 5 and F R 10. Four rats were tested for a 10-hr period on F R 5 with 400 W-3 sec reinforcements to determine whether a decreasing body temperature would eventually facilitate rate of responding. These animals had shown sporadic responding during preliminary testing on F R 5 and F R 10. Two animals died
PR5
,O.our ,.,,
.....
_ ~
,,, =~7.o
U
/
:
FIG. 3. Cumulative record of a complete 10 hr test on FR 5 at --5°C for Rat No. 12. Reinforcements of 400 W for 3 sec. The lower record is a continuation of the top record. Average rates of response and reinforcement were 5/min, and 1/min, respectively.
26
CARLISLE
36°C after the ninth test. Adaptation as a consequence of repeated daily tests was noted in t w o additional animals. Figure 5 shows Thy during two sequential tests after acclimatization for one of these rats. The subject was at a neutral ambient temperature for the first 40--50 min of the tests. Ambient temperature was then lowered to 0°C within 10 rain. Note that Thy rises when ambient temperature is lowered, and remains at an elevated level in spite of the lack of behavioral responding. Food deprivation for 16 hr prior to the second test resulted in a more variable Thy and an improvement in rate of responding. Hence, an interference with metabolic resources, as would occur with food deprivation, can facilitate rate of responding in acclimatized rats.
The Monkey One monkey was tested on various F R schedules with constant heat duration and intensity. A single session is shown in Fig. 6 for this animal. Approximately 5 reinforcements/min were obtained on CRF, and 3/min on F R 15. Response and reinforcement rates became sporadic on F R 20. A response rate of 100/min would be required on F R 20 to maintain a constant reinforcement rate of 5/min. Sustained response rates of greater than 50/min were not observed in this monkey. A second monkey was trained to work on F R 5 and F R 10 with varying intensities, durations, and ambient temperatures.
=CRF FR5
x FRI5
I Respodses [
.
FR20
/~
TRI =36.6
/// ///
4o
Tm2=34,4
!
i!
30
.~ 20 ~ 10
~
o!
Test 12 .96/.~s
FRIO
50
Test 1
99/i.9
., FR5
TRI:36.8
Time
T:2:37.4
in
hours
FIG. 6. Response and reinforcements rates for Monkey I during a single test at 0°C under standard reinforcement conditions.
FIG. 4. Cumulative records of the first and last of 12 daily 5 hr tests on FR 5 at --5°C under standard reinforcement conditions for Rat No. 10.
Monkey D 0'%
3o.t 8.w
3°l
¢: 2 0 -
~
10-
0 37
I
J $00"
,,..
20
FR 5
I0
j n 1 2 Duration in secar~ls
FR 5
~ 3
Oi, 0
~ 5 TemDeroture in °C
E I0
FR 10
400" .....
~ .....
~ .....
~ .....
a .....
~ ....
~
-~6
300-
TIME IN HOURS 200-
ioc=~ o - - -
_ f
c
~oo-
T~Mt~AeRS-mMm
~o~
I00-
~ ,,oo0 200100O-
FIG. 5. Hypothalamic temperature (Thy) during two tests for acclimatized Rat No. 28. Records C and D are cumulative records of the last 2 hr of sessions A and B, respectively. Food deprivation for 16 hr preceded test B. Reinforcements of 375 W for 2 see.
FIG. 7. Average rate of response for Monkey D when the duration of reinforcement was varied on FR 5 and FR 10 during a single test (upper left) and when ambient temperature was varied during 3 separate tests (upper right) under the specified conditions. The cumulative record shows a segment of a session when both the ratio and the duration of reinforcement were doubled (indicated by the arrow) at 0°C with standard reinforcement intensities.
BEHAVIORAL TEMPERATURE REGULATION Figure 7 shows the data for this subject. The monkey worked at a rate that was dependent on variations in the duration of reinforcement and the FR schedule when intensity and ambient temperature were constant, as shown at the left of the figure. These data were collected during a single session during which the animal worked for 30 min under each of the indicated conditions. Rate of response was also dependent on variations in ambient temperature when intensity and duration of reinforcement were constant, as shown at the right of Fig. 7. These data were obtained from 3 separate tests at weekly intervals under the specified conditions. Rate of responding by this monkey produced a reinforcement rate that was comparable to that obtained on CRF. Monkeys therefore worked at a rate determined by the schedule as well as the parameters of reinforcement up to FR 15. Sporadic responding was not observed until an exorbitant response rate was required to maintain reinforcement rate. DISCUSSION
Rats will maintain body temperature by working for radiant heat under a variety of reinforcement and ambient temperature conditions on CRF, even though this schedule is not a simple situation since heat reinforcement lasts several seconds. Responses occurring while the lamps are off turn them on, but responses occurring during a reinforcement have no effect. Thus, rate of response is higher than rate of reinforcement on CRF [3, 11, 12]. Reinforcement rate and body temperature are not maintained on FR schedules, as noted in this study. Sporadic responding is frequently observed, indicating that the schedule exerts poor control over the behavior of the rat under these circumstances. Conditions favoring stable responsing are primarily a low ambient temperature, or low body temperature, and infrequent test sessions. The specific parameters of reinforcement, the duration and intensity of heat, do not influence rate of reinforcement on FR schedules in comparison to the rate observed on CRF, at least within the range of values used in this study. That is, the rat will work at a steady and consistent rate for radiant heat on CRF, but typically fails to respond appropriately when all conditions are held constant except the response contingency. Thus, a limited behavioral control can be exerted at the lowest response contingencies, but there is little evidence for behavioral regulation of body temperature. Removal of all vestiges of fur by a depilatory cream and repeated weekly tests are ineffective in improving performance. Repeated tests in close temporal succession, however, produce an effect analogous to acclimatization. It is therefore obvious that the rat has two responses for avoiding the cold, one behavioral and the other metabolic. Repeated cold exposures in close temporal sequence would seem to be a necessary condition for activation of the metabolic response. It is not clear why the rat performs poorly on intermittent heat-reinforcement schedules. One possibility is that the removal of hair alters thermoregulatory responsiveness [2]. The immediacy with which heat acts upon the subject may be a more plausible alternative; that is, there is no delay of reinforcement and no consummatory response with heat, as there would be with nutritive reinforcers. Inadequate responding has been noted with intermittent electrical self-stimulation of the brain, a case in which reinforcement is not only immediate, but central. Schedule performance can be improved in this case by making the response situation more analogous to that occurring with food reinforcement, and by providing
27 for the occurrence of a secondary reinforcer [9]. It should be noted that response conditions in the present study are similar to those in the self-stimulation situation. The secondary reinforcement argument, however, would not readily apply here since visual and auditory cues are associated with heat reinforcement. Activation of the heat lamps produces an increase in ambient illumination and a "Whirr" sound. In addition, the notions of secondary reinforcement and immediacy of reinforcement would not explain the observed differences between rats and monkeys. A more cogent explanation would seem to require a consideration of the metabolic responsiveness of these two species. A number of studies have noted an increased resistance to cold stress in acclimatized rats. Cottle and Carlson [5] found that rats undergo a major metabolic adjustment during the first 5-7 days of continued exposure to 5°C. Metabolic heat production increased initially, and was maintained at a high level during exposure. The ability to increase heat production by non-shivering means has been shown to occur only in coldadapted rats, since these animals, in contrast with warmadapted control rats, increased heat production when muscular activity was blocked by curare [6]. Revusky [10] has noted a temporal decrease in rate of obtaining heat by hairless mice tested for 6 hr daily for 30 consecutive days. Laties and Weiss [8] also have demonstrated that behavioral thermoregulation is a function of cold acclimatization. Rats that had lived at 2°C for a month waited for an extremely long time before beginning to respond regularly for radiant heat. This effect is evident in Figs 4 and 5 of the present study even though cold exposure was limited to 5 hr per day. The present results thus show that behavioral responses are sufficient to maintain body temperature when the rat is tested infrequently on CRF, but not on FR schedules. The metabolic response of the rat appears to predominate when the animal is tested daily on ratio schedules, and body temperature can be maintained in spite of an absence of behavioral responding under these conditions. The implication is that the metabolic response of the rat would have been preempted had reinforcement rate and body temperature been maintained on FR schedules. Cold acclimatization apparently does not occur when rats are tested at weekly intervals on FR schedules. Body temperature shows a consistent decrease during these tests, which may imply that de-acclimation has occurred during the intervening week. The squirrel monkey worked for heat at a rate determined by the schedule and the parameters of reinforcement. Responding was sporadic only under conditions which demanded response rates greater than 50/rain. The behavioral response of the monkey may imply that its metabolic responsiveness to cold stress is not an effective mechanism. Squirrel monkeys show a decrease in shivering and increased muscular coordination during prolonged cold exposure, indicating a form of adaptation. Few changes in the activities of cellular enzyme systems are noted, however, so that non-shivering thermogenesis, as it applies in the case of rodents, may not apply to the monkey [4]. Weiss and Laties [11] have reported that rats will work for heat on a variable interval 12-see schedule of reinforcement. Their animals survived 16-hr tests at a temperature of 2°C. This particular schedule makes reinforcement available at the average rate of 5/min, which is the same rate at which animals in the Weiss and Laties study obtained reinforcement on CRF. A time-dependent schedule which makes reinforcement available at a high rate would thus seem to be consistent with
28
CARLISLE
stable behavior. Rate of response, as noted in this study, is inadequately sustained by the contingencies of reinforcement with response-dependent schedules for the rat. Three conclusions may be derived from this study. First, intermittent heat reinforcement does not sustain behavioral responding of rats in a manner that is comparable to responding observed with nutritive reinforcers. Second, species differ-
ences are an important source of variance determining adjustment to an intermittent thermal reinforcement schedule. Squirrel monkeys perform consistently at ratios up to F R 15, while rats typically fail to adjust to ratios as low as F R 5. Third, the intertest interval is a critical variable for rats, since adaptation to the cold occurs with repeated, brief (5 hr) daily tests.
REFERENCES
1. Baldwin, B. A. and D. L. Ingram. Behavioural thermoregulation in pigs. PhysioL Behav. 2, 15-21, 1967. 2. Bligh, J., Inhibition of thermal polypnoea in the closely shorn sheep. J. PhysioL 168: 764-781, 1963. 3. Carlisle, H. J. Heat intake and hypothalamic temperature during behavioral temperature regulation. J. comp. physiol. Psychol. 61: 388-397, 1966. 4. Chaffee, R. R. J., S. M. Horvath, R. E. Smith and R. S. Welsh. Cellular biochemistry and organ mass of cold- and heatacclimated monkeys. Fedn Proc. 25: 1177-1181, 1966. 5. Cottle, W., and L. D. Carlson. Adaptive changes in rats exposed to cold. Caloric exchange. Am. J. PhysioL 178: 305-308, 1954. 6. Cottle, W. H. and L. D. Carlson. Regulation of heat proauction in cold adapted rats. Proc. Soc. exp. BioL Med. 92: 845-849, 1956.
7. deGroot, J. The rat forebrain in stereotaxic coordinates. Verb. K. Med. Akad. Wet., B. Natuurkunde 52: 1-40, 1959. 8. Laties, V. G., and B.Weiss. Behavior in the cold after acclimatization. Science N.Y. 131: 1891-1892, 1960. 9. Pliskoff, S. S., J. E. Wright and T. D. Hawkins. Brain stimulation as a reinforcer: intermittent schedules. J. exp. Analysis Behav. 8: 75-88, 1965. 10. Revusky, S. H. Cold acclimatization in hairless mice measured by behavioral thermoregulation. Psychonom. Sci. 6: 209-210, 1966. 11. Weiss, B. and V. G. Laties. Magnitude of reinforcement as a variable in thermoregulatory behavior. J. comp. physiol. Psych. 53: 603--608, 1960. 12. Weiss, B., and V. G. Laties. Behavioral thermoregnlation. Science N.Y., 133: 1338-1344, 1961.