- Email: [email protected]
Instrumental learning without proprioceptive feedback
Physiology and Behavior, Vol. 10, pp. lO1-107, Brain Research Publications Inc., 1973. Printed in U.S.A.
Instrumental Learning Without Proprioceptive Feedback' INESSA B. KOSLOVSKAYA 2, ROBERT P. VERTES 3 AND NEAL E. MILLER
The Rockefeller University, N e w York, N. Y. 10021 (Received 10 August 1972) KOSLOVSKAYA, I. B., R. P. VERTES, AND N: E. MILLER. Instrumental learning without proprioceptive feedback. PHYSIOL. BEHAV.10(1) I 01-107, 1973. A preliminary technique was developed for studying instrumental motor nerve conditioning in paralyzed animals. Twelve rats completely paralyzed by d-tubocuarine were trained to change the level of motor nerve activity while the integrated neurogram served as the motor output. All rats demonstrated the ability to increase nerve activity when increases were reinforced by electrical stimulation of the brain. Three out of four rats were able to adjust the level of neural firing to alternating increases and decreases in the criteria for reward. And three out of four rats showed the ability to produce more activity in the nerve of one leg than in that of the other when a difference in a specific direction was rewarded. Motor learning
Curarized animal
Proprioceptive feedback
THESE experiments were part of a series aimed at testing the learning ability of the motor system of the skeletal muscles in the complete absence of kinesthetic feedback. This problem has been studied for more than 75 years, starting with the classical experiments of Mott and Sherrington in 1895 [16]. The results of experiments on this problem are, however, quite contradictory. In complete agreement with clinical observations on the necessity of somatic sensation for the performance of voluntary movements, most of the early studies [2, 12, 16, 20] have shown that deafferentation renders animals incapable of purposeful movements. However, in the experiments with monkeys in which the healthy limb was restricted, the animals were quite capable of c o m p l i c a t e d purposeful movements with the deafferented limb [ 17]. This observation was supported by the results of behavioral conditioning experiments performed more recently [7, 9, 19, 21, 22]. In these experiments, animals given instrumental training were capable of retaining old habits and even of learning new motor responses with the deafferented limbs. In our previous experiments [I0, l l] after complete deafferentation of the forelimb, dogs were capable of learning to avoid electric shock by flexing the unseen, deafferented limb; they were not, however, able to learn a sustained avoidance reaction: to hold the limb lifted for a required period of time [ 11 ]. However, even in the case of broad deafferentation, there always remains the chance of incomplete blockade or the possibility of utilizing, during learning, information
Instrumental conditioning
coming through indirect afferent channels, such as those controlling posture and equilibrium. To eliminate afferent feedback completely, we chose a curarized preparation in which the required response was a change of activity in a peripheral nerve (sciatic) recorded from electrodes hooked around the nerve. Because of the neuromuscular block produced by the curare, this response could in no way affect the skeletal musculature and hence influence the afferent flow of information. Other investigators using similar preparations have again provided us with conflicting results. Classical conditioning was detected during paralysis caused by crushing the anterior nerve roots [ 13] and during bulbocapnotic catatonia [ 1 ]. In animals paralyzed by d-tubocurarine, training during paralysis influenced instrumental responses elicited subsequently in the nonparalyzed state [3, 4, 18]. The negative results obtained in analogous experiments [8, 15] were probably inconclusive because of possible anesthetic effects of the old type of curare that was used. Gama efferent neurons recorded at the ventral root were classically conditioned in animals under Flaxedil by Buchwald et al. [5]. These same researchers later demonstrated that animals subjected to a classical conditioning procedure under Flaxedil showed a much greater propensity to develop a conditioned flexion response in the normal state than animals that were not so pretrained [6 ]. It has yet to be shown that the instrumental learning (operant conditioning) of a motor nerve response can develop in an animal completely devoid of afferent feedback. If such learning can be demonstrated, the question
~Supported by USPHS Research Grant MH 13189. 2Exchange Fellow of the Academy of Sciences of USSR, Moscow. a Requests for reprints should be addressed to R. P. Vertes, The Rockefeller University, New York, N. Y. 10021, U.S.A. lOl
102
K O S L O V S K A Y A , V E R T E S AND M I L L E R
remains whether the neurons involved are alpha or gamma. If they are alpha m o t o r neurons, then those neurons responsible for m o v e m e n t per se can be o p e r a n t l y conditioned w i t h o u t proprioceptive feedback. If they are gamma m o t o r neurons, then those neurons that prepare and facilitate m o v e m e n t are capable of operant learning under conditions of paralysis, neurons which under similar conditions of paralysis have been classically conditioned [5]. In this paper, we present the results of experiments in which electrical stimulation of the brain (ESB) was used as a reward for changing the rate of firing of the sciatic nerves of animals paralyzed by d-tubocurarine.
GENERAL METHOD
ment Co.). In c o m p l e t e agreement with Miller and DiCara [14], we found that a constant heart rate (420--450 beats per min) and a constant high t e m p e r a t u r e (not less than 38°C) were most reliable signs of a good c o n d i t i o n in the preparation. Procedure On the day before the recording session, the rat was anesthetized with Nembutal. His hind limbs were opened and b o t h sciatic nerves and their main branches ( c o m m o n peroneal and tibial nerves) were freed from the surrounding tissues so as to make them accessible for recording (Fig. 1 ). After the surgery, the edges of the skin were clamped together.
A nimals The animals were 12 adult male albino Sprague-Dawley rats, weighing a p p r o x i m a t e l y 350 g. Each animal was stereotaxically implanted with bipolar electrodes in the posterior region of the lateral h y p o t h a l a m u s with flat skull coordinates of 2.5 m m posterior to bregma, 1.5 mm lateral to it and 9.0 m m vertical to the surface of the skull. All animals showed a stable rate of p e r f o r m a n c e (more than 300 responses in 30 min) when pretested for reward in a bar-pressing situation. Apparatus Tests for the reinforcing effects of brain stimulation were given in a clear plastic box, 12 in. x 10 in. x 16 in., with a lever elevated 1.5 in. above the floor. The reinforcing stimulus consisted of a train of 0.2 msec square wave pulses delivered at 100 pulses/sec from a Grass stimulator (Model $4) with a Grass stimulus isolation unit (Model SIU-4B) in series. The current, which ranged f r o m 2 0 - 1 0 0 uA, was adjusted for each rat to a level which produced a strong reward. It remained on as long as the rat held the bar d o w n ; in this area the initial effects of stimulation are rewarding, but prolonged ones are aversive. During sessions under d-tubocurarine, the animal's respiration was maintained artificially by means of a small animal respirator (E. & M. Instrument Co., Model V5KG) at a fixed rate of 70 inspiration-expiration breaths per min and at a pressure of 1 5 - 2 0 cc of water. The neurograms from the c o m m o n peroneal and sometimes from the tibial nerves were recorded bipolarly by platinum hook electrodes on which the nerves rested. A T e k t r o n i x low-level amplifier (Type 122) was used to amplify the signal 100 times. F r o m the preamplifier the signal was simultaneously relayed to a T e k t r o n i x 502 dual beam oscilloscope, to provide the recording of raw nerve activity, and to an integrator channel (Model 753A) of a Grass polygraph (Model 7), to display the average value of nerve activity for an integration time of 200 msec. The level of averaged nerve activity taken from the o u t p u t of the driver amplifier served as the response which triggered the reinforcement circuit composed of a Schmitt trigger and standard relay e q u i p m e n t programmed so that a set level of activity could be determined and reinforced. As long as the level of the integrated neurogram exceeded the criterion level, ESB was delivered to the animal. The EMG was recorded with needle electrodes placed into the biceps and triceps of the animal's left forelimb. Body t e m p e r a t u r e was continuously monitored by means of a T e l e t h e r m o m e t e r (Yellowsprings Instru-
Common peroneol
biol
FIG. 1. A. Lateral aspect of the left thigh dissected to show the common peroneal and tibial nerves, from which the recordings were made. Taken from The Anatomy of the Rat by E. C. Greene (1959). On the day of the recording session the animals were retested in the bar-pressing situation to determine that the previous day's operation had not had any adverse effect on posterior limb m o v e m e n t (i.e., damage to the sciatic nerve) and that the rewarding properties of the ESB remained intact. If no adverse effect was observed, the e x p e r i m e n t was begun. The subjects were injected intraperitoneally with 0.3 to 0.4 ml of a 3 m g / m l solution of d-tubocurarine chloride (Squibb), sufficient to maintain c o m p l e t e paralysis of the rat for a p p r o x i m a t e l y 1 hr. Subsequent doses of 0.2 ml were given every 30 rain. The animals were fitted with a small face mask that was connected to the respirator as described in Miller and DiCara [14]. Heart and EMG electrodes and the rectal t h e r m o m e t e r were inserted. After all measures (heart rate, respiration, temperature) indicated that the animals were in good condition, the skin and tissue around the wounds were injected with Xylocaine, and the nerves were reexposed and
MOTOR LEARNING WITHOUT PROPRIOCEPTIVE FEEDBACK placed on the platinum electrodes. Skin around the nerve was lifted up and clamped to a metal ring to form a cavity that was filled with warm mineral oil to protect the nerve. After these preparations were completed, the animals were given 3 0 - 4 0 rain of rest. At the end of the rest period a neurogram of very low amplitude (not exceeding 5 - 1 0 uV) was usually recorded with spontaneous bursts of 1 0 - 1 5 uV in amplitude, as illustrated respectively by traces A and B in Fig. 2, lasting 5 - 2 0 sec and occurring with a frequency of 1 - 4 bursts per rain. However, if rats were suddenly aroused by approach, touch, bright light, or loud noise, high amplitude activity of 3 0 - 4 0 ,~V occurred and lasted 2- 3 sec. The first part of every experiment consisted in the shaping of the response to increase the magnitude of spontaneous nerve activity. The shapin~ was accomplished by first rewarding spontaneous bursts ~,f low amplitude and then demanding bursts of ever increasing amplitude. In animals in which spontaneous bursts did ~ot occur, activity i~fitially aroused by touch or noise w:,,s rewarded until spontaneous bursts did occur. Shaping was continued until the subject was able by itself to maintain self-stimulation rates, producing bursts of the amplitude required by the criterion. During shaping, the magnitude of the bursts usually was increased by 2 . 5 - 3 times, reaching 3 5 - 4 0 uV; however, in some preparations it could be brought up to 75 uV (traces C and D of Fig. 2). After shaping, the criteria for reward were changed in a way corresponding to the conditions of Experiments 1, 2, and 3 which involved, respectively, training to a certain level, training to different levels, or training to different levels of activity in the nerves of the two sides. Student's t-test was used for all analyses of data. I
II
! i
j :'~ V ,~l ~ j
,~
/', . ¢
,j J,
e'
,
I,
~'~
l
I I0/.zv I I IO msec
FIG. 2. Neurograms of the peroneal nerves of two curarized rats at different states of the experiments. A and B show the neurograms at the beginning of training; C and D represenl activity at the middle and the end of training.
I03 EXPERIMENT
1: I N C R E A S E
IN L E V E L
In this experiment, four rats were trained to increase their level of activity. Similar training was also given to the rats in Experiments 2 and 3 during the first, or shaping, part of those experiments.
Special Procedure The procedure consisted of the following periods; 2 0 - 3 0 min of training; 1 0 - 1 5 min of extinction, during which bursts of neural activity were no longer followed by ESB; and a final 1 5 - 2 0 min of retraining with reward for high neural activity. The activity of the sciatic nerve during every fifth rain of each of the intervals mentioned above was analyztd to determine the frequency of occurrence of criterion response (causing ESB to be delivered) per min, the average length of time between t'v~) successive criterion responses, and the duration of each criterion response. When the record was not quite 20 rain long, every fourth rain wag analyzed in order to yield a total of 4 points.
Results The results of the experiment showed that all the rats, when rewarded by ESB for as long as they remained above the criterion, were able in the paralyzed state to produce rhythmical bursts of nerve activity of a relatively standard length and amplitude, as illustrated during the twenty-first min of training in Fig. 3. The frequency and duration of the bursts varied from rat to rat, just as did the frequency and duration of holding down the bar by the free moving rats. However, two main patterns of response could be distinguished in the neurograms of the paralyzed rats: a low frequency pattern of 5 - 6 bursts per min with each burst lasting 2 - 3 sec, and a high frequency pattern of 25 37 bursts per rain with each burst lasting only 0 . 2 - 0 . 8 sec. Usually these two parameters of performance were negatively correlated. During extinction periods, when ESB was omitted, intervals between bursts progressively lengthened, the amplitude of the bursts decreased, and the overall neurogram activity returned to the pretraining level, as shown in the last sequence of Fig. 3. Results of the various procedures are summarized in histogram form in Fig. 4, which shows the clear-cut differences in the amount of time the neurogram activity was above the criterion level in the different parts of the experiment. When high amplitude activity was reinforced (during the periods of initial training and of retraining after extinction), each of the animals displayed significant increases in the overall duration of activity above the criterion level (p<0.001 for rats 2.13, 2.24, 2.28 and <0.01 for rat 2.29). During the extinction period, when ESB was omitted, the duration of such activity declined below that at the end of training (p<0.001). To reinstate learning, two out of four rats needed to be reshaped by reinforcing responses evoked by touch or noise. All of them showed reliable increases (p< 0.01 ). That the foregoing results represent true learning is demonstrated by controls inherent in the following two additional experiments. EXPERIMENT
2: DIFFERENT
LEVELS
The purpose of this experiment was to study the ability
104
KOSLOVSKAYA, VERTES AND MILLER was a d m i n i s t e r e d i m m e d i a t e l y u p o n a c h i e v e m e n t of t h e criterion level of activity.
Before t r a m m g ,~ ~Integroted
neurogrom
Results B
/EKG ,,=,
. . . . . .
q
trace, noEMG from biceps and triceps muscles seen
u
.
i
C
. . . . . . . . . . . . . . . . . . .
Twenty-firs~ m i n u t e of training
.../.Meets crrlerlon and achieves reward = all=___ n ,•
•
_
III
I
II
I
I
_
J
O n e of t h e rats failed to learn. T h e results o f t h e t h r e e rats w h o did succeed in t h e d i f f e r e n t levels p r o c e d u r e are s u m m a r i z e d in Fig. 5. It can be seen t h a t in each case an increase or a decrease in t h e c r i t e r i o n level p r o d u c e d a c o r r e s p o n d i n g , significant change in neural activity. This result c o n t r o l s for a n y general activating effect of t h e ESB w h i c h could c o n c e i v a b l y a c c o u n t for an increase in neural activity d u r i n g t r a i n i n g b u t scarcely for o n e t h a t was larger w h e n the criterion for reward was m a d e m o r e difficult a n d smaller w h e n it was m a d e easier.
Seventh minute of extinchon
i
5 traces
pl
I
.
.
.
.
.
.
.
.
.
.
I
......
",.5sec~
FIG. 3. Sample records of integrated neurogram of rat No. 2.27. Three sample min illustrate different stages of learning: before training, the twenty-first min of training; and the seventh min of extinction. Trace A shows the integrated neurogram. Trace B marks the time of activity above the criteria and, hence, reward. Trace C records ECG and EMG. Notice the absence of EMG activity due to the curarization.
Low criterion
Medium criterion
High criterion
B c
5O
o
5.22 28
Criterion
D2,~ 24
4o
r ] 2.24
5
[~
e~
,~
20
I~! 5.23 6.23
7 2.28
3O E E
g~
20
m
g~ -7 o
g
-
8
0
4
~o Before Iroininq
20 min troln~n~
First 5 rnm of exhnctLon
Second 5 mm of 20 rmn re-tro~mn( exhnchon
FIG. 4. Mean durations of the activity above criteria at different periods of experiments. Column 1, 2, 3, 4 represent the activity of different rats. of t h e paralyzed rats to change t h e a m p l i t u d e of t h e i r n e u r o g r a m s a p p r o p r i a t e l y as the criterion was shifted to d i f f e r e n t low and high levels.
Special Procedure F o u r a d d i t i o n a l rats were first s h a p e d to p e r f o r m at a high level b y the p r o c e d u r e s t h a t have already b e e n described in G E N E R A L M E T H O D . T h e n t h e y were given training consisting of f o u r or m o r e a l t e r n a t i n g periods of reward for low, high, low, and high levels of activity. Each change t o a new level was p e r f o r m e d gradually a n d reward
Low criterion
Medium criterion
High criterion
FIG. 5. A. Superimposed traces of 5 successive bursts of an integrated neurogram of Rat 5.23 during low, medium, and high criterion intervals. The solid and broken lines indicate, respectively, the levels of baseline activity and of criterion. B. Mean amplitudes of integrated bursts of three successful rats during low, medium, and high criterion intervals, shown in mm of pen deflection. The tops of histograms represent peak value of bursts; the bottoms reflect changes in baseline activity. 5 mm of pen deflection correspond to 10 ~zV changes in the neurogram.
F u r t h e r analysis of t h e p e r f o r m a n c e s h o w e d t h a t t h e different paralyzed rats solved t h e task in d i f f e r e n t ways. Rat 5.23 r e s p o n d e d to changes in t h e required level b y a
MOTOR LEARNING WITHOUT PROPRIOCEPTIVE FEEDBACK shift in the absolute magnitude of each specific response, from 15 uV when exposed to a low-level criterion to almost 100 uV when a high-level criterion was introduced (p>0.001j. Rat 6.23 gave, under each condition, a rather standard size of response, changing mainly the baseline level of activity so that under the demands of a high criterion the standard sized response still could reach it (p< 0.001). Rat 5.22 changed his baseline in response to the first increase required (p<0.001), but w i t h f u r t h e r increase in criterion also changed the magnitude of response (p< 0.001). With the increase in the magnitude of the responses, their duration usually also increased and their frequency decreased. With increase only in magnitude, as in Rat 5.23, one might say that the rat had merely learned to keep increasing his level of neural activity until he achieved the criterion with the ESB terminating further increases, much as a person might pull harder and harder at a sticking door until it opened. Or perhaps the rat even had an innate tendency to perform in this way once he had learned to fire his nerve in order to secure reward. But the two cases in which the baseline changed must have involved, in addition to this, some real memory traces because the changed activity induced by the changed criterion persisted between bursts, when the ESB was not present as an immediate cue.
EXPERIMENT 3: DIFFERENTIAL RESPONSE IN TWO NERVES
105
response. The other three paralyzed rats demonstrated a highly reliable ability to learn to differentiate the activity of the two nerves when rewarded for doing so. This differential learning is an additional control for any sensitizing or generally activating effects of the brain stimulation used as a reward. Figure 6 shows the process of initial and reversal training for rat 3.28. The record displays superimposed traces of the integrated responses from each of the two nerves during each of the specified minutes of training. As illustrated on the left, during initial training the increase in differences between the two sides was achieved by this rat mainly by suppressing the activity of the nonrewarded nerve without changing the activity of the rewarded one. However, when the reversal procedure was introduced, as shown on the fight side of Fig. 6, tile behavior in the previously i~lifibited nerve increased while that in the previously rewarded one gradually decreased. TI~E ov
/"
In the preceding experiments the curarized rats showed a high synchrony of the activity of the nerves to the two legs, which suggests that a general, nonspecific activation, perhaps a struggling response, had been learned. The purpose of this experiment was to study the ability of the paralyzed animals to learn specific differential activity in the two nerves.
Special Procedure The outputs of the integrated activity from the symmetrical nerves on the two sides were recorded separately as usual but also put into a bridge circuit which, in turn, yielded an output proportional to the algebraic difference between the two activities. This differential output with appropriate sign (indicating which of the two nerves had to be the more active) was used to activate a Schmitt trigger when the difference in the appropriate direction reached a criterion magnitude. Four additional rats were used in this series. Gains in the two recording channels were adjusted to yield approximately equal signals from the two nerves. The training to differential levels of response by the two nerves started, after completion of the usual period of shaping described in G E N E R A L METHOD, with rewarding of small spontaneous differences in the direction of the nerve that had not been rewarded during this initial shaping. As the rat learned, the criterion was gradually increased, until greater differences in the correct direction were achieved. After the rats had learned to respond with greater activity in the selected nerve, reversal training was given in the same way, to show the rat's ability to learn a greater activity in the opposite nerve.
Results One of the four rats
failed to learn a differential
FIG. 6. Superimposed traces of 5 successive bursts of an integrated neurogram of the right and left peroneal nerves of the rat at the end of 4 successive intervals of training to differentiate activities of the two sides. The left side of the figure shows the initial training when the difference favoring the activity of the right nerve was rewarded. The right side of the figure demonstrates the effects of reversal training. The results on all of the rats used in this experiment are summarized in Fig. 7. As illustrated in Part A, which represents the averages of the ratio of activity of the two nerves during initial and reversal training periods, at the beginning of training the ratio of the activity of the rewarded to the nonrewarded nerve-was approximately 1 but with training it increased to approximately 3. During reversal training the ratio of the activity of the same two nerves rapidly dropped to approximately 1 and then gradually decreased to zero, as their roles were reversed. Part B of Fig. 7 shows the average activity of each nerve for each of the four rats during the five successive stages of training, the first three of which were original training and the last two reversal training. It clearly shows that in a 4 0 - 5 0 min period of initial training, each of the rats would clearly suppress the activity on the nonreinforced nerve without to a great extent changing the activity of the reinforced side, which slightly increased in the neurogram of Rat 3.27, did not change in Rat 3.2 l, and even slightly decreased in Rats 3.07 and 3.28. However, when the
106
KOSLOVSKAYA, VERTES AND MILLER
[•
Reworded nerve Non-reworded nerve 328
_o "~
• Reworded/non-reworded o Non-reworded/reworded
._c 3.0
and 3.28 gradually decreased so that there was a clear-cut difference in the activity at the end of the original training b e t w e e n rewarded and nonrewarded nerves in Rats 3.28, 3.27 and 3.21 (p<0.001 in each case), which was reversed at the end of reversal training ( p < 0 . 0 0 1 , <0.01 and n.s. respectively). Rat 3.07 always gave a larger response with the right than with the left nerve and hence failed to show clear-cut evidence of learning a differential response.
3.27
o
GENERAL DISCUSSION 3.21
2.0
•
£,J
3 07
c
0
i io
l
o
o
o
-7~ o
L
I 50
1
7LO
L. nerve
R. nerve
Minutes of training
FIG. 7. A. Mean ratio of the burst activity of the rewarded nerves versus nonrewarded during differential training as a function of training time averaged for 4 experimental rats. The initial training is represented by closed circles, the reversal by opened circles. The area covered by activity above the baseline (like that represented in 2 Figs. 5A and 6) was counted in mm per rain, indicated on the ordinate. B. Mean burst activity per rain of 4 experimental rats during differential training averaged for 5 min of each of the represented periods of initial and reversal training. Rewarded nerves are indicated by white, non rewarded by black. Heights of histograms show the value of the burst activity counted~ as in the previous figure, in mm 2 per rain.
reversal procedure was tested, the activity of the previously suppressed nerve increased, reaching the pretraining level in Rat 3.27 and even exceeding it in Rats 3.21 and 3.28 The activity of the nonrewarded nerves in Rats 3,21, 3.27
The results of our experiments showed that the c o m p l e t e block of kinesthetic feedback does not eliminate the ability for instrumental learning by m o t o r neurons. Twelve rats trained while paralyzed by d-tubocurarine increased the amplitude of their neurograms when increases were rewarded by electrical stimulation of the brain. In experiments involving additional training, three out of four rats were able to adjust the level of neural firing to alternating increases and decreases in the level o f the criterion for reward. Finally, three out of four rats showed the ability to learn to produce more activity in one nerve than in the o t h e r when a difference in a specific direction was reinforced. So our results agree with those of previous investigators who, using instrumental procedures, have demonstrated learning ability under the somewhat less rigorous conditions of removing the kinesthetic feedback by deafferentation [2, 11, 19, 21, 22]. Using a classical conditioning procedure, Buchwald e t aL [5, 6] secured conditioning of gamma neurons in deeply paralyzed animals. It is possible that our i n s t r u m e n t a l learning involved fibers from such neurons. F u r t h e r work will be needed to d e t e r m i n e which types of fibers were involved in our case. In experiments soon to be reported, we found that we could produce only transient classical conditioning, which might be interpreted as no stable learning at all, at the deep level of paralysis that we used in the present study, but that as soon as the level of paralysis was reduced to the point where EMG activity began to appear, consistent classical conditioning could be secured. This result raises the possibility that our t e c h n i q u e was recording the activity of alpha fibers which can be trained by instrumental but not by classical conditioning techniques.
REFERENCES
1. Beck, E. C. and R. W. Doty. Conditioned reflexes acquired during combined catelepsy and deafferentation. J. comp. physiol. Psychol. 5 0 : 2 1 1 - 2 1 6 , 1957. 2. Biekel, A. Untersuchungen iiber den Mechanismus der nervosen Bewegung Regulation. Pfl[tgers A rch. 67: 299- 344, 1897. 3. Black, A. H. The extinction of avoidance responses under curare. J. comp. physiol. Psychol. 51 : 519-- 524, 1958. 4. Black, A. H. Transfer following operant conditioning in the curarized dog. Science 155 : 201 - 203, 1967. 5. Buchwald, J. and E. Eldred. Conditioned responses in the gamma efferent system. J. herr. ment, Dis. 132: 146-152, 1961. 6. Buchwald, J. S., M. Standish, E. Eldred and E. S. Halas. Contribution of muscle spindle circuits to learning as suggested by training under Flaxedil. Electroenceph. clin. Neurophysiol. 16: 582-594, 1964.
7. Gorska, T. and E. Yankowska. The effect of deafferentation on instrumental (type II) conditioned reflexes in dogs. Acta Biol. Exp. 21: 219-234, 1961. 8. Harlow, H. F. and R. Stagner. Effect of complete striate muscle paralysis upon the learning process. J. exp. Psychol. 16: 283-294, 1933. 9. Knapp, H. D., E. Taub and A. J. Berman. Movements in monkeys with deafferented forelimbs. Expl. Neurol. 7: 303-315, 1963. 10. Koslovskaya, I. B., R. L. Gasanova and N. G. Ivanova. The functional organization of complex forms of avoidance reflexes. Proc. XVIII Int. Congr. Psvchol. Moscow, 1966 , pp. 118-122. 11. Koslovskaya, 1. B., A. V. Ovsjannikoff and R. L. Gasanova. Avoidance conditioning after deafferentation of the operant limb. Proc. X X I Meeting on the Problem o f Higher Nervous Actirity, Leningrad, 1966, pp. 149-150.
MOTOR LEARNING WITHOUT PROPRIOCEPTIVE FEEDBACK 12. Lassek, A. M. Inactivation of voluntary motor function following rhizotomy. J. Neuropath. exp. Neurol. 12: 8 3 - 8 7 , 1953. 13. Light, L S. and W. H. Gantt. Essential part of reflex arc for establishment of conditioned reflex. J. comp. Psychol. 21: 1 9 - 3 6 , 1936. 14. Miller, N. E. and L. V. DiCara. Instrumental learning of heart rate changes in curarized rats. J. comp. physiol. Psychol. 63: 1 2 - 1 9 , 1967. 15. Morgan, C. T. The psychophysiology of learning. In: Handbook o f Experimental Psychology, edited by S. S. Stevens. New York: Wiley, 1951, pp. 7 7 0 - 7 7 2 . 16. Mott, F. W. and C. S. Sherrington. Experiments upon influence of sensory nerves upon movement and nutrition of the limbs. Proc. R. So~ Lond., 57: 4 8 1 - 4 8 8 , 1895. 17. Munk, H. Ober die Funktionen yon Him und R~chenmark. Berlin, 1909.
I O7
18. Solomon, R. L. and L. H. Turner. Discriminative classical conditioning in dogs paralyzed by curare can later control discriminative avoidance response in the normal state. Psychol. Rev. 69: 2 0 2 - 2 1 9 , 1962. 19. Taub, E., R. C. Bacon and A. J. Berman. Acquisition of a trace-conditioned avoidance response after deafferentation of the responding limb. J. eomp. physiol. Psychol. 5 9 : 2 7 5 - 2 7 9 , 1965. 20. Twitchell, T. E. Sensory factor in purposive movements. J. Neurophysiol. 17: 239-254, 1954. 21. Yankowska, E. Ruchowe instrumentalne odruchy warunkowe deafferentowaney konezyny u kotow. Acta physiol. Pol. (ICarszawa) 8: 360-366, 1957. 22. Yankowska, E. Instrumental scratch reflex of the deafferented limb in cats and rats. Acta Biol. exp. (Warszawa) 19: 2 3 3 - 2 4 2 , 1959.
Instrumental Learning Without Proprioceptive Feedback' INESSA B. KOSLOVSKAYA 2, ROBERT P. VERTES 3 AND NEAL E. MILLER
The Rockefeller University, N e w York, N. Y. 10021 (Received 10 August 1972) KOSLOVSKAYA, I. B., R. P. VERTES, AND N: E. MILLER. Instrumental learning without proprioceptive feedback. PHYSIOL. BEHAV.10(1) I 01-107, 1973. A preliminary technique was developed for studying instrumental motor nerve conditioning in paralyzed animals. Twelve rats completely paralyzed by d-tubocuarine were trained to change the level of motor nerve activity while the integrated neurogram served as the motor output. All rats demonstrated the ability to increase nerve activity when increases were reinforced by electrical stimulation of the brain. Three out of four rats were able to adjust the level of neural firing to alternating increases and decreases in the criteria for reward. And three out of four rats showed the ability to produce more activity in the nerve of one leg than in that of the other when a difference in a specific direction was rewarded. Motor learning
Curarized animal
Proprioceptive feedback
THESE experiments were part of a series aimed at testing the learning ability of the motor system of the skeletal muscles in the complete absence of kinesthetic feedback. This problem has been studied for more than 75 years, starting with the classical experiments of Mott and Sherrington in 1895 [16]. The results of experiments on this problem are, however, quite contradictory. In complete agreement with clinical observations on the necessity of somatic sensation for the performance of voluntary movements, most of the early studies [2, 12, 16, 20] have shown that deafferentation renders animals incapable of purposeful movements. However, in the experiments with monkeys in which the healthy limb was restricted, the animals were quite capable of c o m p l i c a t e d purposeful movements with the deafferented limb [ 17]. This observation was supported by the results of behavioral conditioning experiments performed more recently [7, 9, 19, 21, 22]. In these experiments, animals given instrumental training were capable of retaining old habits and even of learning new motor responses with the deafferented limbs. In our previous experiments [I0, l l] after complete deafferentation of the forelimb, dogs were capable of learning to avoid electric shock by flexing the unseen, deafferented limb; they were not, however, able to learn a sustained avoidance reaction: to hold the limb lifted for a required period of time [ 11 ]. However, even in the case of broad deafferentation, there always remains the chance of incomplete blockade or the possibility of utilizing, during learning, information
Instrumental conditioning
coming through indirect afferent channels, such as those controlling posture and equilibrium. To eliminate afferent feedback completely, we chose a curarized preparation in which the required response was a change of activity in a peripheral nerve (sciatic) recorded from electrodes hooked around the nerve. Because of the neuromuscular block produced by the curare, this response could in no way affect the skeletal musculature and hence influence the afferent flow of information. Other investigators using similar preparations have again provided us with conflicting results. Classical conditioning was detected during paralysis caused by crushing the anterior nerve roots [ 13] and during bulbocapnotic catatonia [ 1 ]. In animals paralyzed by d-tubocurarine, training during paralysis influenced instrumental responses elicited subsequently in the nonparalyzed state [3, 4, 18]. The negative results obtained in analogous experiments [8, 15] were probably inconclusive because of possible anesthetic effects of the old type of curare that was used. Gama efferent neurons recorded at the ventral root were classically conditioned in animals under Flaxedil by Buchwald et al. [5]. These same researchers later demonstrated that animals subjected to a classical conditioning procedure under Flaxedil showed a much greater propensity to develop a conditioned flexion response in the normal state than animals that were not so pretrained [6 ]. It has yet to be shown that the instrumental learning (operant conditioning) of a motor nerve response can develop in an animal completely devoid of afferent feedback. If such learning can be demonstrated, the question
~Supported by USPHS Research Grant MH 13189. 2Exchange Fellow of the Academy of Sciences of USSR, Moscow. a Requests for reprints should be addressed to R. P. Vertes, The Rockefeller University, New York, N. Y. 10021, U.S.A. lOl
102
K O S L O V S K A Y A , V E R T E S AND M I L L E R
remains whether the neurons involved are alpha or gamma. If they are alpha m o t o r neurons, then those neurons responsible for m o v e m e n t per se can be o p e r a n t l y conditioned w i t h o u t proprioceptive feedback. If they are gamma m o t o r neurons, then those neurons that prepare and facilitate m o v e m e n t are capable of operant learning under conditions of paralysis, neurons which under similar conditions of paralysis have been classically conditioned [5]. In this paper, we present the results of experiments in which electrical stimulation of the brain (ESB) was used as a reward for changing the rate of firing of the sciatic nerves of animals paralyzed by d-tubocurarine.
GENERAL METHOD
ment Co.). In c o m p l e t e agreement with Miller and DiCara [14], we found that a constant heart rate (420--450 beats per min) and a constant high t e m p e r a t u r e (not less than 38°C) were most reliable signs of a good c o n d i t i o n in the preparation. Procedure On the day before the recording session, the rat was anesthetized with Nembutal. His hind limbs were opened and b o t h sciatic nerves and their main branches ( c o m m o n peroneal and tibial nerves) were freed from the surrounding tissues so as to make them accessible for recording (Fig. 1 ). After the surgery, the edges of the skin were clamped together.
A nimals The animals were 12 adult male albino Sprague-Dawley rats, weighing a p p r o x i m a t e l y 350 g. Each animal was stereotaxically implanted with bipolar electrodes in the posterior region of the lateral h y p o t h a l a m u s with flat skull coordinates of 2.5 m m posterior to bregma, 1.5 mm lateral to it and 9.0 m m vertical to the surface of the skull. All animals showed a stable rate of p e r f o r m a n c e (more than 300 responses in 30 min) when pretested for reward in a bar-pressing situation. Apparatus Tests for the reinforcing effects of brain stimulation were given in a clear plastic box, 12 in. x 10 in. x 16 in., with a lever elevated 1.5 in. above the floor. The reinforcing stimulus consisted of a train of 0.2 msec square wave pulses delivered at 100 pulses/sec from a Grass stimulator (Model $4) with a Grass stimulus isolation unit (Model SIU-4B) in series. The current, which ranged f r o m 2 0 - 1 0 0 uA, was adjusted for each rat to a level which produced a strong reward. It remained on as long as the rat held the bar d o w n ; in this area the initial effects of stimulation are rewarding, but prolonged ones are aversive. During sessions under d-tubocurarine, the animal's respiration was maintained artificially by means of a small animal respirator (E. & M. Instrument Co., Model V5KG) at a fixed rate of 70 inspiration-expiration breaths per min and at a pressure of 1 5 - 2 0 cc of water. The neurograms from the c o m m o n peroneal and sometimes from the tibial nerves were recorded bipolarly by platinum hook electrodes on which the nerves rested. A T e k t r o n i x low-level amplifier (Type 122) was used to amplify the signal 100 times. F r o m the preamplifier the signal was simultaneously relayed to a T e k t r o n i x 502 dual beam oscilloscope, to provide the recording of raw nerve activity, and to an integrator channel (Model 753A) of a Grass polygraph (Model 7), to display the average value of nerve activity for an integration time of 200 msec. The level of averaged nerve activity taken from the o u t p u t of the driver amplifier served as the response which triggered the reinforcement circuit composed of a Schmitt trigger and standard relay e q u i p m e n t programmed so that a set level of activity could be determined and reinforced. As long as the level of the integrated neurogram exceeded the criterion level, ESB was delivered to the animal. The EMG was recorded with needle electrodes placed into the biceps and triceps of the animal's left forelimb. Body t e m p e r a t u r e was continuously monitored by means of a T e l e t h e r m o m e t e r (Yellowsprings Instru-
Common peroneol
biol
FIG. 1. A. Lateral aspect of the left thigh dissected to show the common peroneal and tibial nerves, from which the recordings were made. Taken from The Anatomy of the Rat by E. C. Greene (1959). On the day of the recording session the animals were retested in the bar-pressing situation to determine that the previous day's operation had not had any adverse effect on posterior limb m o v e m e n t (i.e., damage to the sciatic nerve) and that the rewarding properties of the ESB remained intact. If no adverse effect was observed, the e x p e r i m e n t was begun. The subjects were injected intraperitoneally with 0.3 to 0.4 ml of a 3 m g / m l solution of d-tubocurarine chloride (Squibb), sufficient to maintain c o m p l e t e paralysis of the rat for a p p r o x i m a t e l y 1 hr. Subsequent doses of 0.2 ml were given every 30 rain. The animals were fitted with a small face mask that was connected to the respirator as described in Miller and DiCara [14]. Heart and EMG electrodes and the rectal t h e r m o m e t e r were inserted. After all measures (heart rate, respiration, temperature) indicated that the animals were in good condition, the skin and tissue around the wounds were injected with Xylocaine, and the nerves were reexposed and
MOTOR LEARNING WITHOUT PROPRIOCEPTIVE FEEDBACK placed on the platinum electrodes. Skin around the nerve was lifted up and clamped to a metal ring to form a cavity that was filled with warm mineral oil to protect the nerve. After these preparations were completed, the animals were given 3 0 - 4 0 rain of rest. At the end of the rest period a neurogram of very low amplitude (not exceeding 5 - 1 0 uV) was usually recorded with spontaneous bursts of 1 0 - 1 5 uV in amplitude, as illustrated respectively by traces A and B in Fig. 2, lasting 5 - 2 0 sec and occurring with a frequency of 1 - 4 bursts per rain. However, if rats were suddenly aroused by approach, touch, bright light, or loud noise, high amplitude activity of 3 0 - 4 0 ,~V occurred and lasted 2- 3 sec. The first part of every experiment consisted in the shaping of the response to increase the magnitude of spontaneous nerve activity. The shapin~ was accomplished by first rewarding spontaneous bursts ~,f low amplitude and then demanding bursts of ever increasing amplitude. In animals in which spontaneous bursts did ~ot occur, activity i~fitially aroused by touch or noise w:,,s rewarded until spontaneous bursts did occur. Shaping was continued until the subject was able by itself to maintain self-stimulation rates, producing bursts of the amplitude required by the criterion. During shaping, the magnitude of the bursts usually was increased by 2 . 5 - 3 times, reaching 3 5 - 4 0 uV; however, in some preparations it could be brought up to 75 uV (traces C and D of Fig. 2). After shaping, the criteria for reward were changed in a way corresponding to the conditions of Experiments 1, 2, and 3 which involved, respectively, training to a certain level, training to different levels, or training to different levels of activity in the nerves of the two sides. Student's t-test was used for all analyses of data. I
II
! i
j :'~ V ,~l ~ j
,~
/', . ¢
,j J,
e'
,
I,
~'~
l
I I0/.zv I I IO msec
FIG. 2. Neurograms of the peroneal nerves of two curarized rats at different states of the experiments. A and B show the neurograms at the beginning of training; C and D represenl activity at the middle and the end of training.
I03 EXPERIMENT
1: I N C R E A S E
IN L E V E L
In this experiment, four rats were trained to increase their level of activity. Similar training was also given to the rats in Experiments 2 and 3 during the first, or shaping, part of those experiments.
Special Procedure The procedure consisted of the following periods; 2 0 - 3 0 min of training; 1 0 - 1 5 min of extinction, during which bursts of neural activity were no longer followed by ESB; and a final 1 5 - 2 0 min of retraining with reward for high neural activity. The activity of the sciatic nerve during every fifth rain of each of the intervals mentioned above was analyztd to determine the frequency of occurrence of criterion response (causing ESB to be delivered) per min, the average length of time between t'v~) successive criterion responses, and the duration of each criterion response. When the record was not quite 20 rain long, every fourth rain wag analyzed in order to yield a total of 4 points.
Results The results of the experiment showed that all the rats, when rewarded by ESB for as long as they remained above the criterion, were able in the paralyzed state to produce rhythmical bursts of nerve activity of a relatively standard length and amplitude, as illustrated during the twenty-first min of training in Fig. 3. The frequency and duration of the bursts varied from rat to rat, just as did the frequency and duration of holding down the bar by the free moving rats. However, two main patterns of response could be distinguished in the neurograms of the paralyzed rats: a low frequency pattern of 5 - 6 bursts per min with each burst lasting 2 - 3 sec, and a high frequency pattern of 25 37 bursts per rain with each burst lasting only 0 . 2 - 0 . 8 sec. Usually these two parameters of performance were negatively correlated. During extinction periods, when ESB was omitted, intervals between bursts progressively lengthened, the amplitude of the bursts decreased, and the overall neurogram activity returned to the pretraining level, as shown in the last sequence of Fig. 3. Results of the various procedures are summarized in histogram form in Fig. 4, which shows the clear-cut differences in the amount of time the neurogram activity was above the criterion level in the different parts of the experiment. When high amplitude activity was reinforced (during the periods of initial training and of retraining after extinction), each of the animals displayed significant increases in the overall duration of activity above the criterion level (p<0.001 for rats 2.13, 2.24, 2.28 and <0.01 for rat 2.29). During the extinction period, when ESB was omitted, the duration of such activity declined below that at the end of training (p<0.001). To reinstate learning, two out of four rats needed to be reshaped by reinforcing responses evoked by touch or noise. All of them showed reliable increases (p< 0.01 ). That the foregoing results represent true learning is demonstrated by controls inherent in the following two additional experiments. EXPERIMENT
2: DIFFERENT
LEVELS
The purpose of this experiment was to study the ability
104
KOSLOVSKAYA, VERTES AND MILLER was a d m i n i s t e r e d i m m e d i a t e l y u p o n a c h i e v e m e n t of t h e criterion level of activity.
Before t r a m m g ,~ ~Integroted
neurogrom
Results B
/EKG ,,=,
. . . . . .
q
trace, noEMG from biceps and triceps muscles seen
u
.
i
C
. . . . . . . . . . . . . . . . . . .
Twenty-firs~ m i n u t e of training
.../.Meets crrlerlon and achieves reward = all=___ n ,•
•
_
III
I
II
I
I
_
J
O n e of t h e rats failed to learn. T h e results o f t h e t h r e e rats w h o did succeed in t h e d i f f e r e n t levels p r o c e d u r e are s u m m a r i z e d in Fig. 5. It can be seen t h a t in each case an increase or a decrease in t h e c r i t e r i o n level p r o d u c e d a c o r r e s p o n d i n g , significant change in neural activity. This result c o n t r o l s for a n y general activating effect of t h e ESB w h i c h could c o n c e i v a b l y a c c o u n t for an increase in neural activity d u r i n g t r a i n i n g b u t scarcely for o n e t h a t was larger w h e n the criterion for reward was m a d e m o r e difficult a n d smaller w h e n it was m a d e easier.
Seventh minute of extinchon
i
5 traces
pl
I
.
.
.
.
.
.
.
.
.
.
I
......
",.5sec~
FIG. 3. Sample records of integrated neurogram of rat No. 2.27. Three sample min illustrate different stages of learning: before training, the twenty-first min of training; and the seventh min of extinction. Trace A shows the integrated neurogram. Trace B marks the time of activity above the criteria and, hence, reward. Trace C records ECG and EMG. Notice the absence of EMG activity due to the curarization.
Low criterion
Medium criterion
High criterion
B c
5O
o
5.22 28
Criterion
D2,~ 24
4o
r ] 2.24
5
[~
e~
,~
20
I~! 5.23 6.23
7 2.28
3O E E
g~
20
m
g~ -7 o
g
-
8
0
4
~o Before Iroininq
20 min troln~n~
First 5 rnm of exhnctLon
Second 5 mm of 20 rmn re-tro~mn( exhnchon
FIG. 4. Mean durations of the activity above criteria at different periods of experiments. Column 1, 2, 3, 4 represent the activity of different rats. of t h e paralyzed rats to change t h e a m p l i t u d e of t h e i r n e u r o g r a m s a p p r o p r i a t e l y as the criterion was shifted to d i f f e r e n t low and high levels.
Special Procedure F o u r a d d i t i o n a l rats were first s h a p e d to p e r f o r m at a high level b y the p r o c e d u r e s t h a t have already b e e n described in G E N E R A L M E T H O D . T h e n t h e y were given training consisting of f o u r or m o r e a l t e r n a t i n g periods of reward for low, high, low, and high levels of activity. Each change t o a new level was p e r f o r m e d gradually a n d reward
Low criterion
Medium criterion
High criterion
FIG. 5. A. Superimposed traces of 5 successive bursts of an integrated neurogram of Rat 5.23 during low, medium, and high criterion intervals. The solid and broken lines indicate, respectively, the levels of baseline activity and of criterion. B. Mean amplitudes of integrated bursts of three successful rats during low, medium, and high criterion intervals, shown in mm of pen deflection. The tops of histograms represent peak value of bursts; the bottoms reflect changes in baseline activity. 5 mm of pen deflection correspond to 10 ~zV changes in the neurogram.
F u r t h e r analysis of t h e p e r f o r m a n c e s h o w e d t h a t t h e different paralyzed rats solved t h e task in d i f f e r e n t ways. Rat 5.23 r e s p o n d e d to changes in t h e required level b y a
MOTOR LEARNING WITHOUT PROPRIOCEPTIVE FEEDBACK shift in the absolute magnitude of each specific response, from 15 uV when exposed to a low-level criterion to almost 100 uV when a high-level criterion was introduced (p>0.001j. Rat 6.23 gave, under each condition, a rather standard size of response, changing mainly the baseline level of activity so that under the demands of a high criterion the standard sized response still could reach it (p< 0.001). Rat 5.22 changed his baseline in response to the first increase required (p<0.001), but w i t h f u r t h e r increase in criterion also changed the magnitude of response (p< 0.001). With the increase in the magnitude of the responses, their duration usually also increased and their frequency decreased. With increase only in magnitude, as in Rat 5.23, one might say that the rat had merely learned to keep increasing his level of neural activity until he achieved the criterion with the ESB terminating further increases, much as a person might pull harder and harder at a sticking door until it opened. Or perhaps the rat even had an innate tendency to perform in this way once he had learned to fire his nerve in order to secure reward. But the two cases in which the baseline changed must have involved, in addition to this, some real memory traces because the changed activity induced by the changed criterion persisted between bursts, when the ESB was not present as an immediate cue.
EXPERIMENT 3: DIFFERENTIAL RESPONSE IN TWO NERVES
105
response. The other three paralyzed rats demonstrated a highly reliable ability to learn to differentiate the activity of the two nerves when rewarded for doing so. This differential learning is an additional control for any sensitizing or generally activating effects of the brain stimulation used as a reward. Figure 6 shows the process of initial and reversal training for rat 3.28. The record displays superimposed traces of the integrated responses from each of the two nerves during each of the specified minutes of training. As illustrated on the left, during initial training the increase in differences between the two sides was achieved by this rat mainly by suppressing the activity of the nonrewarded nerve without changing the activity of the rewarded one. However, when the reversal procedure was introduced, as shown on the fight side of Fig. 6, tile behavior in the previously i~lifibited nerve increased while that in the previously rewarded one gradually decreased. TI~E ov
/"
In the preceding experiments the curarized rats showed a high synchrony of the activity of the nerves to the two legs, which suggests that a general, nonspecific activation, perhaps a struggling response, had been learned. The purpose of this experiment was to study the ability of the paralyzed animals to learn specific differential activity in the two nerves.
Special Procedure The outputs of the integrated activity from the symmetrical nerves on the two sides were recorded separately as usual but also put into a bridge circuit which, in turn, yielded an output proportional to the algebraic difference between the two activities. This differential output with appropriate sign (indicating which of the two nerves had to be the more active) was used to activate a Schmitt trigger when the difference in the appropriate direction reached a criterion magnitude. Four additional rats were used in this series. Gains in the two recording channels were adjusted to yield approximately equal signals from the two nerves. The training to differential levels of response by the two nerves started, after completion of the usual period of shaping described in G E N E R A L METHOD, with rewarding of small spontaneous differences in the direction of the nerve that had not been rewarded during this initial shaping. As the rat learned, the criterion was gradually increased, until greater differences in the correct direction were achieved. After the rats had learned to respond with greater activity in the selected nerve, reversal training was given in the same way, to show the rat's ability to learn a greater activity in the opposite nerve.
Results One of the four rats
failed to learn a differential
FIG. 6. Superimposed traces of 5 successive bursts of an integrated neurogram of the right and left peroneal nerves of the rat at the end of 4 successive intervals of training to differentiate activities of the two sides. The left side of the figure shows the initial training when the difference favoring the activity of the right nerve was rewarded. The right side of the figure demonstrates the effects of reversal training. The results on all of the rats used in this experiment are summarized in Fig. 7. As illustrated in Part A, which represents the averages of the ratio of activity of the two nerves during initial and reversal training periods, at the beginning of training the ratio of the activity of the rewarded to the nonrewarded nerve-was approximately 1 but with training it increased to approximately 3. During reversal training the ratio of the activity of the same two nerves rapidly dropped to approximately 1 and then gradually decreased to zero, as their roles were reversed. Part B of Fig. 7 shows the average activity of each nerve for each of the four rats during the five successive stages of training, the first three of which were original training and the last two reversal training. It clearly shows that in a 4 0 - 5 0 min period of initial training, each of the rats would clearly suppress the activity on the nonreinforced nerve without to a great extent changing the activity of the reinforced side, which slightly increased in the neurogram of Rat 3.27, did not change in Rat 3.2 l, and even slightly decreased in Rats 3.07 and 3.28. However, when the
106
KOSLOVSKAYA, VERTES AND MILLER
[•
Reworded nerve Non-reworded nerve 328
_o "~
• Reworded/non-reworded o Non-reworded/reworded
._c 3.0
and 3.28 gradually decreased so that there was a clear-cut difference in the activity at the end of the original training b e t w e e n rewarded and nonrewarded nerves in Rats 3.28, 3.27 and 3.21 (p<0.001 in each case), which was reversed at the end of reversal training ( p < 0 . 0 0 1 , <0.01 and n.s. respectively). Rat 3.07 always gave a larger response with the right than with the left nerve and hence failed to show clear-cut evidence of learning a differential response.
3.27
o
GENERAL DISCUSSION 3.21
2.0
•
£,J
3 07
c
0
i io
l
o
o
o
-7~ o
L
I 50
1
7LO
L. nerve
R. nerve
Minutes of training
FIG. 7. A. Mean ratio of the burst activity of the rewarded nerves versus nonrewarded during differential training as a function of training time averaged for 4 experimental rats. The initial training is represented by closed circles, the reversal by opened circles. The area covered by activity above the baseline (like that represented in 2 Figs. 5A and 6) was counted in mm per rain, indicated on the ordinate. B. Mean burst activity per rain of 4 experimental rats during differential training averaged for 5 min of each of the represented periods of initial and reversal training. Rewarded nerves are indicated by white, non rewarded by black. Heights of histograms show the value of the burst activity counted~ as in the previous figure, in mm 2 per rain.
reversal procedure was tested, the activity of the previously suppressed nerve increased, reaching the pretraining level in Rat 3.27 and even exceeding it in Rats 3.21 and 3.28 The activity of the nonrewarded nerves in Rats 3,21, 3.27
The results of our experiments showed that the c o m p l e t e block of kinesthetic feedback does not eliminate the ability for instrumental learning by m o t o r neurons. Twelve rats trained while paralyzed by d-tubocurarine increased the amplitude of their neurograms when increases were rewarded by electrical stimulation of the brain. In experiments involving additional training, three out of four rats were able to adjust the level of neural firing to alternating increases and decreases in the level o f the criterion for reward. Finally, three out of four rats showed the ability to learn to produce more activity in one nerve than in the o t h e r when a difference in a specific direction was reinforced. So our results agree with those of previous investigators who, using instrumental procedures, have demonstrated learning ability under the somewhat less rigorous conditions of removing the kinesthetic feedback by deafferentation [2, 11, 19, 21, 22]. Using a classical conditioning procedure, Buchwald e t aL [5, 6] secured conditioning of gamma neurons in deeply paralyzed animals. It is possible that our i n s t r u m e n t a l learning involved fibers from such neurons. F u r t h e r work will be needed to d e t e r m i n e which types of fibers were involved in our case. In experiments soon to be reported, we found that we could produce only transient classical conditioning, which might be interpreted as no stable learning at all, at the deep level of paralysis that we used in the present study, but that as soon as the level of paralysis was reduced to the point where EMG activity began to appear, consistent classical conditioning could be secured. This result raises the possibility that our t e c h n i q u e was recording the activity of alpha fibers which can be trained by instrumental but not by classical conditioning techniques.
REFERENCES
1. Beck, E. C. and R. W. Doty. Conditioned reflexes acquired during combined catelepsy and deafferentation. J. comp. physiol. Psychol. 5 0 : 2 1 1 - 2 1 6 , 1957. 2. Biekel, A. Untersuchungen iiber den Mechanismus der nervosen Bewegung Regulation. Pfl[tgers A rch. 67: 299- 344, 1897. 3. Black, A. H. The extinction of avoidance responses under curare. J. comp. physiol. Psychol. 51 : 519-- 524, 1958. 4. Black, A. H. Transfer following operant conditioning in the curarized dog. Science 155 : 201 - 203, 1967. 5. Buchwald, J. and E. Eldred. Conditioned responses in the gamma efferent system. J. herr. ment, Dis. 132: 146-152, 1961. 6. Buchwald, J. S., M. Standish, E. Eldred and E. S. Halas. Contribution of muscle spindle circuits to learning as suggested by training under Flaxedil. Electroenceph. clin. Neurophysiol. 16: 582-594, 1964.
7. Gorska, T. and E. Yankowska. The effect of deafferentation on instrumental (type II) conditioned reflexes in dogs. Acta Biol. Exp. 21: 219-234, 1961. 8. Harlow, H. F. and R. Stagner. Effect of complete striate muscle paralysis upon the learning process. J. exp. Psychol. 16: 283-294, 1933. 9. Knapp, H. D., E. Taub and A. J. Berman. Movements in monkeys with deafferented forelimbs. Expl. Neurol. 7: 303-315, 1963. 10. Koslovskaya, I. B., R. L. Gasanova and N. G. Ivanova. The functional organization of complex forms of avoidance reflexes. Proc. XVIII Int. Congr. Psvchol. Moscow, 1966 , pp. 118-122. 11. Koslovskaya, 1. B., A. V. Ovsjannikoff and R. L. Gasanova. Avoidance conditioning after deafferentation of the operant limb. Proc. X X I Meeting on the Problem o f Higher Nervous Actirity, Leningrad, 1966, pp. 149-150.
MOTOR LEARNING WITHOUT PROPRIOCEPTIVE FEEDBACK 12. Lassek, A. M. Inactivation of voluntary motor function following rhizotomy. J. Neuropath. exp. Neurol. 12: 8 3 - 8 7 , 1953. 13. Light, L S. and W. H. Gantt. Essential part of reflex arc for establishment of conditioned reflex. J. comp. Psychol. 21: 1 9 - 3 6 , 1936. 14. Miller, N. E. and L. V. DiCara. Instrumental learning of heart rate changes in curarized rats. J. comp. physiol. Psychol. 63: 1 2 - 1 9 , 1967. 15. Morgan, C. T. The psychophysiology of learning. In: Handbook o f Experimental Psychology, edited by S. S. Stevens. New York: Wiley, 1951, pp. 7 7 0 - 7 7 2 . 16. Mott, F. W. and C. S. Sherrington. Experiments upon influence of sensory nerves upon movement and nutrition of the limbs. Proc. R. So~ Lond., 57: 4 8 1 - 4 8 8 , 1895. 17. Munk, H. Ober die Funktionen yon Him und R~chenmark. Berlin, 1909.
I O7
18. Solomon, R. L. and L. H. Turner. Discriminative classical conditioning in dogs paralyzed by curare can later control discriminative avoidance response in the normal state. Psychol. Rev. 69: 2 0 2 - 2 1 9 , 1962. 19. Taub, E., R. C. Bacon and A. J. Berman. Acquisition of a trace-conditioned avoidance response after deafferentation of the responding limb. J. eomp. physiol. Psychol. 5 9 : 2 7 5 - 2 7 9 , 1965. 20. Twitchell, T. E. Sensory factor in purposive movements. J. Neurophysiol. 17: 239-254, 1954. 21. Yankowska, E. Ruchowe instrumentalne odruchy warunkowe deafferentowaney konezyny u kotow. Acta physiol. Pol. (ICarszawa) 8: 360-366, 1957. 22. Yankowska, E. Instrumental scratch reflex of the deafferented limb in cats and rats. Acta Biol. exp. (Warszawa) 19: 2 3 3 - 2 4 2 , 1959.