A comparative review of motivational systems using classical control theory

Anim . Behav., 1978,26, 3 68 -380

A COMPARATIVE REVIEW OF MOTIVATIONAL SYSTEMS USING CLASSICAL CONTROL THEORY BY

F. M . TOATES & J . ARCHER

Division of Psychology, Preston Polytechnic, Preston, Lancashire

Abstract. We consider how classical control theory principles can be applied not only to feeding and drinking but also to sexual behaviour, aggression, and fear. A comparison is made of the design features in each motivational system in relation to the function each system has evolved to fulfil . Negative feedback is incorporated as an overall control principle, for feeding, drinking, fear and aggression ; it is also involved in sexual motivation, as a means of ending behaviour . Such general features as the reference value, discrepancy and proportionality are considered . Behavioural stability and the use of positive feedback as a means of achieving stability, and feedforward as a means of minimizing error, are also discussed . functions as a form of negative feedback to restore the discrepancy . In this paper we also address the problem of how classical control theory principles, such as negative feedback and feedforward, can represent different types of motivational systems . However, unlike Lorenz's general model, we suggest that there is a variety of different ways in which control theory principles can be incorporated into motivational systems . In particular, we consider how classical control theory principles can be applied not only to feeding and drinking (the main emphasis of recent work) but also to sexual behaviour, aggression and fear. The main purpose of the review is to compare the design features of each motivational system, and, where possible, these are related to the function each system has evolved to fulfil. Thus we argue that different types of mechanism are necessary for different functional end-points . For example, if the endpoint of attack is, say, to remove a territorial intruder, it would be logical for such behaviour to occur only when a territorial intruder is present. The existence of an aggressive drive resulting from deprivation (as Lorenz suggested) would seem counterproductive in this context. We begin by describing some of the models which have been suggested to account for feeding, drinking, fighting, fear behaviour and copulation, and also discussing their common properties and differences . We then discuss a number of general features of these models under the following headings : negative feedback as an overall control principle and as a way of ending a behavioural act, properties of negative feedback systems, discrepancy and proportionality, feedforward, positive feedback

The general notion that motivational systems are constructed in such a way as to maintain an animal's internal stability is one which has a fairly long history . But it is only in more recent years that mathematically precise models describing this process in terms such as negative feedback and reference values have been developed for the motivational systems underlying eating and drinking (McFarland 1971 ; Toates 1975). Used in a less precise way, such principles were also incorporated into the general model of motivation suggested by Lorenz. This involved the idea that a departure from equilibrium would automatically occur in the absence of performing a particular type of behaviour, and hence would lead to a build-up of a specific drive . Such a general model was applied by Lorenz to eating and drinking, and also to social behaviour such as courtship, copulation and aggression . Thus, he stated that `after a long passivity of an instinctive behaviour pattern, in this case courtship, the threshold value of its eliciting stimuli sink' (Lorenz 1966, p . 49) . Similarly, Storr (1968), although admitting that it is difficult to portray the source of sexual and aggressive behaviour in the physiological terms which may more readily be used to describe hunger and thirst, nevertheless describes sexual motivation as an `internal driving force which has to be satisfied' (p . 33) and refers to `aggressive tension' (p . 33), suggesting that both sex and aggression may operate in essentially the same manner. In these writings, the implication is that several very different types of behaviour can be represented by a single model, which involves a relatively fixed reference point from which the animal will automatically drift in the absence of performing the behaviour : the behaviour 368

TOATES & ARCHER: CONTROL SYSTEMS AND MOTIVATION and behavioural stability. Finally we discuss some of the strengths and weaknesses of the classical control theory approach to motivation . The models we describe vary widely in the data base used for their construction . Those representing hunger and thirst are taken primarily from physiological experiments on feeding and drinking in the rat. The model described for aggressive behaviour is derived from behavioural evidence from a variety of vertebrate species, and that for sexual behaviour is based on the male rat . It should be understood, therefore, that these models provide only illustrative examples of the design features involved in the different systems as deduced from the particular type of evidence used . Nevertheless, we believe that a preliminary attempt to compare the principles involved in the different systems is worthwhile, since we are concerned with their overall design features rather than the location of these features in specific anatomical structures . Thirst There is convincing evidence that if a rat is allowed access to water, it accurately compensates for a deficit of water in either the cellular or extra-cellular compartments . For example, hypertonic saline injections cause the animal to drink a quantity of water which will restore extra-cellular concentration to its pre-injection value and consequently allow cellular fluid volume to return to normal (Fitzsimons 1961) . Similarly, removal of extra-cellular fluid by a variety of procedures results in an accurate compensatory response (Fitzsimons 1972) . The evidence fits a system which operates to keep body fluid volume at an internally defined reference value, thirst being provoked by a deficit with respect to this value . According to such a formulation the only reason drinking appears is that fluid losses periodically reach a threshold value. As is discussed later, the animal may also anticipate future losses in association with feeding (Fitzsimons & Le Magnen 1969 ; Oatley & Toates 1969) . Some authors have concluded that the circadian rhythm of drinking may be viewed in terms of a set-point mechanism, an internal biological clock moving the set point up and down (Nicolaidis & Rowland 1974 ; Oatley 1974 ; Toates 1977) . What might appear as a failure of set-point regulation in the form of a voluntary deficit following a period of water or water and food deprivation, is explicable in terms of breakdown of cell contents and maintenance by the

369

set-point of a constant solid-to-water ratio in the cellular compartment (Toates 1977) . A number of results leads to the conclusion that a sufficient condition for satiation of thirst is a restoration of fluid volume to normal by whatever route. Thus, following a period of water deprivation, the amount a rat drinks is depressed by an intravenous infusion of water just prior to drinking, whereas isotonic saline infusion is ineffective (Novin et al . 1966). Corbit (1965) found that if water is injected intravenously at the same time that a rat bar-presses for water to drink, the injected water reduces drinking by 1 ml for every 1 ml injected . Such results are incompatible with an interpretation in terms of motivational energy or `charge' which arises spontaneously without reference to the state of body fluids or from a deficit signal and which can only be dissipated by drinking . They fit a control model in which there is only one integrator present (the body fluid pool) and where thirst is corrected by returning error in the fluid compartment to zero. Recently a series of experiments which does not easily fit any simple formulation has been published. Nicolaidis & Rowland (1975) showed that if massive continuous intragastric or intravenous infusions of water were given to rats they still drink a significant quantity of water. This of course does not fit the simple control model . Nicolaidis & Rowland (1975) suggested that the `oral and gastric compartments may have their own intrinsic needs over and above the systemic needs' . A model of reward has been proposed by Rolls (1975) in order to explain electrical selfstimulation but its relevance is far wider than this, and can be extended to thirst . According to the model, messages generated from water receptors in the mouth would make contact with water reward neurones in the hypothalamus, but before they can influence firing of these neurones, a `gate function' must be passed, and this is only opened if a state of thirst exists . In other words, body fluid receptors would open the gate when a deficit exists and close it when there is a fluid surplus . This model indicates a change of emphasis in motivated behaviour from the animal being driven by a deficit to the deficit permitting drinking to be rewarding . According to the model, there is no reason to expect that water entering the body via intravenous infusion would be equivalent to water taken by mouth . In fact, Rolls's model in no way predicts that satisfactory regulation would



3 70

ANIMAL BEHAVIOUR, 26, 2

occur as a result of intravenous infusion . But the model is still compatible with the fact that injected water is satiating when subsequent drinking is measured, since injected water would serve to close the `gate function' . Rolls's theory may also explain why intravenous infusions are unable to eliminate drinking . It may be that the `gate-function' simply opens at intervals if drinking is not otherwise taking place, even though a surplus of water exists in the body. In summary, then, it is clear that under a variety of circumstances, such as the response to salt injection, blood loss, or water deprivation, the intact rat with access to water behaves as a set-point controlled system. When food is withdrawn, water intake declines slowly to match the lowered need for water (Toates 1971) and when the diet is changed to one rich in protein there is an immediate increase in water intake to compensate for the increased renal loss (Fitzsimons & Le Magnen 1969) . Supply of water directly to the blood (thus by-passing the mouth and stomach) satiates the rat, except when it occurs over a period of days, and conversely sham drinking, where the consummatory act is normal but the body fluids obtain no benefit from the water drunk, leads to no long term satiety, despite the fact that the animal drinks an excessive amount (see Fitzsimons 1972, for review) . The only evidence which does not fit a body-fluid set-point model is that in the long term even if the rat's water needs are satisfied, the animal nevertheless drinks a certain amount . This finding could be taken as indicating some form of motivational energy (as suggested in Lorenz's general model, described in the introduction) which also serves as a `fail-safe' mechanism and could in addition supply the intrinsic need of the oral and gastric compartments. The evidence is, as yet, inconclusive . Hunger A set-point has long formed part of the theorizing of workers in the area of feeding research . By this they mean something very close to the engineer's meaning . Lipostatic, thermostatic and glucostatic models (see Grossman 1968) have all been proposed, while the most recent and perhaps the most bizarre suggestion was that the organism controls body weight by means of monitoring the pressure on the soles of the feet and that feeding is part of this control loop (Johnsen 1973) . Later we critically examine the whole set-point concept . The concept of a set-

point of quantity, such as total energy content or state of the fat deposits, as the factor which is responsible for long-term energy regulation, has come under criticism recently (Booth et al . 1976) . It was argued that rate of energy supply is perhaps the crucial variable involved in initiating and terminating meals, and a computer stimulation of such a system was shown to give a good fit to experimental data (Toates & Booth 1974) . By `energy supply' is meant the rate at which energy is being supplied from gut, i .e . absorption minus the rate of lipogenesis . In the case of lipolysis this forms an additional source of energy supply which enables feeding to be reduced . Either by examining energy rate (as Toates & Booth propose) or energy quantity in some form or other, the animal is able to maintain the energy available to the body by feeding when the metabolic need arises . Collier et al . (1972) emphasized the importance of ecological factors in determining meal patterns, arguing that a simple control theory model is inadequate. However, their point that `For some species it appears that there is a minimum stomach content, not zero, and that animals eat in such a pattern that the gut acts as a reservoir . This suggests that input across the intestinal lumen, fed from this reservoir, must be relatively constant', is entirely compatible with the model of Toates & Booth (1974) where rate of energy supply from the gut is an important variable . In the laboratory where the animal has a standard diet available, feeding is probably very closely determined by metabolic factors, whereas in the natural state availability of food and (where applicable) relationships with other animals in the same group might dominate over metabolic considerations in the short-term timing of meals . In the long term, metabolic factors must, of course, predominate. If one proposes that the animal eats in order to maintain the rate of energy supply constant, then it is largely terminological whether this is then called a set-point, the important feature being that departure from some determined point promotes feeding (see also Geertsema & Reddingius 1974). The circadian rhythm of feeding in rats may be seen to fit such a formulation since during the light phase energy is available from lipolysis whereas during the dark phase a component of the energy taken by feeding is diverted into the fat stores by the process called lipogenesis (Le Magnen et al . 1973) and this will result in an uneven supply of calories and hence a circadian rhythm of

TOATES & ARCHER : CONTROL SYSTEMS AND MOTIVATION

feeding. Also in keeping with such a modified homeostatic system is the observation that feeding may be inhibited by infusions of nutrients, though apparently it is difficult to bring feeding to zero (Nicolaidis 1971) . This particular observation suggests that feeding, like drinking, arises when some physiological quantity departs from a setting and is largely prevented from occurring by artificially maintaining the supply of calories or water respectively . That feeding and drinking cannot be completely inhibited suggests, in addition, a non-homeostatic factor . Aggression and Fear We have described how control theory models of drinking and eating involve monitoring the level of body fluid or rate of energy supply for a discrepancy from a reference point . In the case of aggression, Archer (1976) suggested that a different form of discrepancy model is involved, and this consists of comparing an internal representation based on past experience of environmental events ('neuronal models') with aspects of the present environment . In this way, inputs are compared with the neuronal models, and any large discrepancy activates one of two systems concerned with reducing the discrepancy : one of these systems activates aggression, which results in removal of the discrepancy itself, and the other system activates various forms of fear and avoidance responses which generally result in removal of the animal from the source of the discrepancy . Archer's (1976) model of aggression involves the animal comparing, in different situations, different aspects of the environment : broadly speaking, these may be associated with either pain or novelty or frustration. Thus, one form of aggression (e .g. pain induced) may involve different immediate external causal factors than another (e.g. frustration-induced) in a similar manner to drinking arising from, in one instance, the prior intake of hypertonic saline and in another from fluid loss . When adopting a control-theory approach to these types of behaviour, it is the similarity in the end-point which is crucial, rather than differences in the eliciting stimuli . This viewpoint of motivational systems is the opposite to the approach adopted by Moyer (1968) in devising what is by now a widely-quoted classification of the different types of aggression (see Archer 1976, for a further discussion of Moyer's classification) . Evidence in support of the argument, favoured here, that different external antecedents of

371

aggression do act through common mechanisms at some stage, is derived from the apparent summation of different types of antecedents to increase the probability of aggression (as, e.g . with Galef 1970a, b ; Legrand & Fielder 1973). The model also concerns the various factors which might bias the control mechanism towards either attack or fear behaviour, including the degree of discrepancy and whether or not the animal is expecting a reward, and also hormonal, early experiental factors and past reinforcement . The final part of the model concerns those (mainly external) factors which may determine the final behavioural outcome : these include target properties and whether escape is possible . One point not included in the original formulation of the model concerns the relationshipbetween the size of the discrepancy and the intensity and duration of the behaviour . As we have already pointed out, one essential difference between Lorenz's view of aggression and the control theory model proposed here is that in the former aggression is `driven' (by the level of action specific energy, which is a consequence of deprivation), whereas in our model its occurrence is a response to a discrepancy ; in other words, the intensity of aggression may be graded to the demands of the particular external situation ; this introduces the means whereby once aggressive behaviour has been initiated, its continued occurrence can be controlled not only by the magnitude of initial discrepancy but also by the moment to moment results of the aggressive action . Thus aggressive behaviour can be graded by positive or negative feedback to fit the specific demands of the situation. It is interesting to contrast this with the hypothetical situation involved in the case of fear behaviour. The usual responses are either flight or immobility, which function to remove the animal from the source of the discrepancy or to minimize the noxious consequences of such a discrepancy (Archer 1976) . If we again introduce the notion that intensity and duration of behaviour may be related to the degree of discrepancy (discussed below under the heading of proportionality) there seems less possibility than in the case of aggression that it can be graded to the changing consequences of the behaviour in such a finely-controlled manner . It is therefore suggested that in the cast of fear, the discrepancy acts more as a driving force, in a manner similar to the degree of action



372

ANIMAL BEHAVIOUR, 26, 2

degree of discrepancy determining the strength of the tendency to flee . In order that the fleeing can come to an end, this tendency must become less as time goes on. One way of achieving this would be to incorporate an integrator with a negative feedback loop . We therefore suggest that an initial signal (f) corresponding to the degree of discrepancy (and other variables which affect the strength of the tendency to show fear behaviour) is integrated to give a value F which dissipates over time (see Fig . 1) . Sexual Motivation Sexual motivation shares some of the characteristics of hunger and thirst . In the male, ejaculation, or a series of ejaculations, leads to satiety, and copulatory behaviour can only be rearoused after a recovery period has elapsed, suggesting that the system slowly drifts from a set-point in the intervening period . Historically, theories of sexual motivation have involved control by internal physiological events, e .g. the seminal vesicle theory associated the sex drive with the tension produced by seminal fluids and Hull's theory implied that drive reduction is accompanied by a fall in the level of a sex hormone (Hull 1951). Despite such claims, no obvious physiological correlate of sexual motivation and post-copulation satiation has been found, and this has led some investigators to emphasize external stimulation. The finding that sexual behaviour can be more easily reactivated by a novel than a familiar partner soon after a

previous occurrence suggests a parallel with the model of aggression described in the previous section : both involve responding to novelty or discrepancy in some way . Although we suggest that this is crucial for aggression, in the case of sexual motivation it must operate in conjunction with some internal variables dependent upon the time since the last performance of sexual behaviour . These features have been incorporated into a recent model of sexual behaviour in the male rat (Brown et al . 1974) which has attempted to integrate external stimulation from a partner with the animal's internal state as determinants of behaviour. Toates & O'Rourke (1977) have adapted the model of Brown et al . (1974) and we shall use this modification to discuss sexual motivation in the context of homeostasis . The model shown in Fig. 2 is derived from experimental evidence on the sexual behaviour

level of neural store (G)

f neural message from genital friction

1 Intromission rate

internal arousal (F)

E o e

T

T T arousal from external stimulation (A .)

fdt TZ arousal( A) -4

f Fig. 1 . Diagrammatic representation of the process whereby fear tendency decreased over time . f, strength of initial fear-eliciting stimulus ; F, signal indicating strength of fear reactions ; d, rate of dissipation ; K, constant .

Fig. 2. Block diagram model of sexual motivation in the male rat, based on Brown et al . (1974). A is arousal after gain Ki has had its effect ; Tl is copulatory threshold ; K2 relates intromission rate and the neural signal generrated ; G is a neural store which dissipates according to gain K4 ; K3 determines internal arousal which adds to external arousal stimulus Ao ; T2 is the ejaculatory threshold .

TOATES & ARCHER : CONTROL SYSTEMS AND MOTIVATION

of the male rat . A o represents the potential arousal from an external stimulus, usually the partner. Actual arousal depends upon the sensitivity or arousability of the nervous system represented by term K 1 . If K 1 is high then actual arousal (A) is also high for an adequate stimulus A o. If A exceeds the threshold T1 then it can be seen that arousal will lead to intromission . Gain K2 determines the relationship between intromission rate and the neural signal which this sets up as a result of genital friction . A neural store termed G is a representation of the integration of the neural messages derived from frictional contact. The term K3 relates the value of the neural store G to the level of arousal which it generates (F), and this summates with the arousal from external stimulation (A0) which is derived from the partner through the various sensory channels . Thus the model involves a positive feedback loop so that arousal causes intromission which in turn increases arousal. The model also involves two (effectively) negative feedback loops operating through a parameter change. When A exceeds a threshold T2, ejaculation occurs, and this reduces the arousability or sensitivity of the system by reducing K 1 and K2. This represents a lowering of the ability of a potentially arousing stimulus to arouse the system (K1) and a lowering of the neural message from genital friction which is set up for any particular intromission rate (K2). Following a series of ejaculations, these gains would be very low and would slowly increase with time. In addition to these loops producing a cessation of copulation following ejaculation or a series of ejaculations, a negative feedback pathway via K4 is involved . The contents of the integrator have a natural tendency to dissipate, so that if stimulation through intromission stops, the level of arousal returns to that decided by A o acting on its own . The rate of dissipation of the integrator contents is dictated by K4. The gains would be such that there is net positive feedback until ejaculation occurs under normal conditions of copulation . In our model (Toates & O'Rourke 1977) sexual `exhaustion' is due to loss of arousability, in other words, reduction in gain . In the model of Brown et al . (1974) it is due to inhibition . Sexual behaviour is a good example of a system where a set-point may be involved ; indeed, the system does behave in a way similar to a set-point actuated system. Yet it could be misleading to use this expression. A sexually arousable nervous system

373

leads to behaviour which then lowers arousability. However, few would want to call low arousability a set-point value (see McFarland & Nunez 1977, for a very similar argument) . The model allows for the effect of novelty described earlier in that if K 1 is fairly low the threshold T1 may still be exceeded if A0 is relatively high. We assume that the arousal value of a novel partner corresponds to a high input to the system . Given the required task, it is difficult to see how the motivational system could be constructed in a way very much different from that represented in Fig. 2. Initial arousal stimulates intromission, which then increases the level of arousal : in other words, the positive feedback loop operates to sustain the behaviour once it has started . A time-dependent function (the integrator) is necessary for the following reason . If there were merely simple proportionality in the positive feedback loop, it could not lead to a build up of neural activity with time, since for any initial value of A o the value A would immediately take its maximum value . If this value of A were insufficient to exceed the ejaculatory threshold, no further amount of intromission would be effective . If on the other hand it were sufficient, ejaculation would be immediate . It is clear that neither of these possibilities occurs, since a certain amount of frictional movement is required, and the function of this is likely to be to prevent ejaculation occurring at irrelevant times (although involuntary emission still occurs under some circumstances). An integrator is therefore essential . The negative feedback pathway around the integrator is probably also indispensible. If for some reason copulation were to be interrupted and this feedback pathway were not present, the animal would remain in a state of high sexual arousal which would possibly offer competition with more vital activities . Loss of sexual motivation could be brought about by discharge of the contents of the integrator through K4. The presence of K4 means that copulation is in the nature of an uphill struggle to reach the ejaculatory threshold . In the model described above the parameters do not correspond to any particular physiological quantity (e.g . drive does not correspond to the volume of seminal fluids) ; instead they are representations of the degree of potential arousal by external stimulation and from intromission . Recovery of motivation may occur in parallel with physiological events such as



3 74

ANIMAL BEHAVIOUR, 26, 2

increased seminal fluid volume but is not dependent upon any such event . Sexual motivation is not homeostatic in the conventional sense of the word, and yet it exhibits some of the characteristics of such a system, including a negative feedback effect (McFarland & Nunez 1977) ; arousal leads to ejaculation which then lowers arousability for a period of time . The result is a relaxation oscillation . Thus the animal appears sated after copulation; the adaptive significance of this satiation is probably derived from the time required for accumulation of seminal fluids and from the relatively small return in terms of probability of fertilization which would be derived from further sexual activity at this time . The model represented in Fig. 2 could possibly be adapted to cover the female mammal . This would probably involve much faster recovery (if in fact the evidence justifies postcopulation desensitization) . It would also require a powerful hormonal influence to represent the oestrous cycle or the breeding season . Negative Feedback Each of the motivational models outlined above incorporates the principle of negative feedback, which is of course an essential feature of conventional homeostatic systems . Thus, in the case of hunger and thirst a deficit produces a discrepancy from a reference value which leads to consummatory behaviour, and the substance ingested as a result of this behaviour corrects the discrepancy and ultimately the deficit . In addition, there may be negative feedback pathways running from the stomach to the brain to signal the presence of food or water . In the case of aggression and fear, it is suggested that the principle of negative feedback involves behaviour being instigated to correct a discrepancy based on neural representations of expected environmental events . The model describing sexual behaviour is of a rather different nature, in that negative feedback is not the major controlling feature of the system ; instead there are two effectively negative feedback loops incorporated into a system which contains other features (principally positive feedback) in its overall design . Thus, in the first three motivational systems, negative feedback is incorporated to reduce a discrepancy from a reference value, and this process results in the attainment of the functional end-point of the system. In the case of the fourth system, sexual behaviour, the only

way in which the functional end-point could be achieved through a comparable process, reduction of a discrepancy by negative feedback, would involve some form of reflex discharge of semen in response to pressure receptors . Such a mechanism would be severely limited as regards the types of situations in which it would be able to achieve a satisfactory functional end-point. (For successful mammalian copulation entailing as it does internal fertilization, such a mechanism could only work if the female were to make rapid advances to a male about to ejaculate!) Clearly a different type of mechanism has evolved for mammalian copulation ; this has to ensure that ejaculation tends to occur more in those situations where it is likely to result in fertilization . To achieve this, the mechanism incorporates positive feedback resulting from the stimulation produced by copulation itself, so that ejaculation occurs after a build-up of the resulting stimulation to a level unlikely to be attained outside the copulatory situation . In the case of hunger, thirst and aggression, negative feedback serves not only as the main controlling feature of the behaviour (see above) but also to end the particular behaviour when the discrepancy is corrected . In the case of sexual motivation, this second function of negative feedback has been retained, although the overall control of the system is different (see above) . Thus negative feedback is still required to produce cessation of copulation after ejaculation has occurred and this seems to operate by producing a sudden decrease in sensitivity at some point in the system (see section on sexual motivation) . The role of negative feedback in terminating behaviour may involve one of several precise mechanisms ; thus in the case of sex it appears to operate by a change in responsiveness with performance, whereas with aggression it may be the attainment of a match to a neuronal model. It should be noted (see Hinde 1970) that there are other ways in which behaviour may be brought to an end, which do not involve negative feedback, e .g . cessation of positive feedback (see below), alteration of the reference value so that a discrepancy is no longer perceived (habituation), or a change in eliciting stimuli . Properties of Negative Feedback Systems Negative feedback systems show a number of general features, which it will be useful to discuss in turn . Such systems may include a reference value and detection of a discrepancy,



TOATES & ARCHER : CONTROL SYSTEMS AND MOTIVATION

although in some cases the reference value may be purely hypothetical (Toates 1975) . In some cases even a hypothetical reference value seems unnecessary and misleading, and yet still a negative feedback effect can be identified (e .g. sex) . The principle of proportionality (which relates response strength to stimulus magnitude) can be investigated whether or not a set-point is involved or even imagined. (a) Reference Values Reference value, input, desired value and setpoint are expressions meaning much the same thing and which have been borrowed from engineering, where their use is often metaphorical, involving terms more appropriate to human behaviour . It would appear that in the biological sciences these expressions sometimes cause on the one hand, unnecessarily complex neurophysiological speculation and, on the other, unjustified simplification. The engineer would speak of room temperature being compared with a desired value and action being taken in response to a discrepancy. By analogy the experimental psychologist would speak of body fluid volume being controlled at a setpoint value. This often leads to the conclusion that a neural signal proportional to actual body fluid volume is compared with the neural signal representing the set-point and the brain takes action when the difference between these exceeds a threshold . The system may possibly work in this way, but a simpler method would be for a cell to generate a neural signal only when its volume falls below normal : the signal would then represent a deficit . The volume of the cell when it is not causing neural firing then assumes the role of reference value . Mathematically the two systems are identical . In the latter case the operation appears to have more in common with a pressure or temperature detector at the skin . The conclusions of a recent conference on feeding control (Silverstone 1976) are very relevant to this argument : `The set point is a variable which acts as a source of comparison . . . . , ' A system with a set point will account for regulatory phenomena but it is not necessary for such a system . Moreover, any mathematically defined control system containing a set point can be transformed into a system without a set point yet exhibiting the same behaviour .' In the case of aggression and fear, it is clear that any hypothetical reference value would bear little resemblance to the kind of set-point that

37 5

has been postulated in simple physiological regulation . Presumably essential features are checked against an internal store or model and at some point there is an integration of disparity messages. The result of this integration determines whether action in the form of fleeing or fighting results . (b) Proportional Control The proportional mode of control has probably been the one to attract most attention from biological control system model-builders, if only because the mathematics of the system is relatively simple . In this system action is proportional to the discrepancy and so, as this is corrected, the action becomes less intense . For example, an animal might slow up its rate of drinking as the deficit is reduced . Some species do behave in this way (e.g. the rat and the Barbary dove : McFarland & McFarland 1968) whereas others drink at the same rate until the deficit is corrected (e .g. dogs) . In some species there may exist a rate-limitation due to the environment, e .g . the depth of water. Whether we are dealing, in the former case, with a linear proportional control system or not, the results give this appearance. It could alternatively be the non-linearity of a threshold plus inhibitory loops which gives an appearance of proportionality . Pauses in drinking could be caused by inhibition from the stomach and mouth temporarily cancelling out the deficit signal (Toates 1975) . Alternatively, `time-out', a switching off of drinking, (McFarland 1971) could be responsible with no particular reference to stomach inhibition or competition from other motivations . Either way, the pauses, which become more frequent as satiety approaches, give an almost exponential response . In the case of animals which drink to satiety without a break, this corresponds to on-off control which is familiar to the engineer . It may reflect a mechanism for increasing persistence or the relative lack of inhibitory effects . The strength of the hunger and thirst drive may well be a function of a disparity signal, possibly the proportional model provides a reasonable first approximation . In the case of fear and aggression it is proposed that not only the strength but also the choice of behaviour is dependent upon the size of a disparity signal . Thus Archer's (1976) model incorporated a disparity detector, small disparity causing aggression and large disparity fear. Such a concept was first advanced several years ago by Hebb (1966, p . 257) who

3 76

ANIMAL BEHAVIOUR, 26, 2

stated : `It is also possible that milder degrees of disruption innately tend to lead to aggression, stronger ones to avoidance .' An obvious problem would be the tendency of an animal to switch between fear and aggression when the disparity is in the region of the cross-over point . For example, the animal may start to flee, which reduces disparity and which consequently might cause a change to attack . Conversely, attack might initially increase the disparity to that associated with fear . Clearly inertial mechanisms are called for in order to stop dithering and these are described below . Positive Feedback We have described above how negative feedback may be the main controlling feature for hunger, thirst and aggression but plays a role only in the cessation of sexual behaviour . The principle of positive feedback is also employed, to differing extents, in the four systems. We suggest that a major function may be to promote maintenance of a particular type of behaviour in the face of competing causal factors (referred to as 'behavioural dominance,' e .g. McFarland (1974)) . Thus positive feedback prevents uneconomical 'dithering' which would otherwise occur when the causal factors for two types of behaviour are relatively equal . Consider the case of an animal which has a dominant thirst motivation and a secondary hunger motivation . It comes across both food and water, and it begins to drink . As a consequence of this action, the water reduces the thirst motivation by means of negative feedback, and hunger motivation is disinhibited . As a result, hunger would become the dominant motivation and the animal would switch from drinking to feeding ; the result of this action would be to decrease hunger relative to thirst and to again disinhibit drinking . This would clearly be uneconomical for both the reasons mentioned above : it would involve extra energy expenditure and the increased activity might attract predator attention. In fact, the state of affairs described does not occur ; thus, it appears that the immediate effect of feeding is not to decrease feeding activity but to potentiate it (Wiepkema 1971) ; this result implies the existence of positive feedback mechanisms which promote behavioural dominance . Similarly, in their computer simulation and experimentation with drinking in the Barbary dove, McFarland & McFarland (1968) indicated that positive feedback from the mouth serves to give momentum to drinking. Toates (1974, 1975)

argued that negative rather than positive feedback is associated with the mouth but there is no reason why both should not co-exist, an immediate positive feedback effect to keep the animal going at drinking and a delayed negative feedback to contribute to satiety in the longer term. Such positive feedback would be one factor which would determine how strongly the animal would `lock-on' to drinking when food was also present (McFarland 1971) . This would, however, also be determined by external factors such as the degree of proximity of food and water (McFarland 1971, p .248) . Thus, positive feedback mechanisms involved will have to be sufficiently flexible to respond not only to stimuli indicating the consequence of performing a particular behaviour activity such as eating or drinking, but also to estimates of the relative `cost' of changing to another type of activity (see McFarland 1971, p . 248) . In the model of sexual motivation (Toates & O'Rourke 1977 : see above), behaviour is maintained by positive feedback derived from genital contact, and this is conceptualized in terms of `sexual arousal' (Fig . 2) . Implicit in this concept is the idea that behaviour, once started, gains momentum from its consequences . In this particular case, the animal derives positive feedback from the partner (Fig . 2), both through genital stimulation and through the initial arousing effect of stimuli through other sensory channels . Similarly, in the case of aggressive motivation, positive feedback may be derived from the partner. Wiepkema (personal communication) has recently proposed a model of territorial aggression which is very similar in principle to the more general model proposed by Archer (1976), but he has also included an immediate but short-lasting positive feedback effect when the discrepancy between observed and expected stimulation becomes smaller as a result of an intruder beginning to flee. A similar short-lived positive feedback effect could be included in Archer's model, to account for continued attack despite the intruder fleeing . In addition to positive feedback there are other possible reasons why behaviour may gain momentum once it has begun. For example, an initial stimulus may produce a long-lasting effect which then keeps the behaviour going irrespective of its consequences . In this case, a stored representation of the effects of the stimulus would gradually dissipate over time, as we have suggested for fear . Such a mechanism might also operate, but to a lesser degree, in the



TOATES & ARCHER : CONTROL SYSTEMS AND MOTIVATION

cases of sex and aggression, e .g . when the initiating stimulus for sexual or aggressive motivation are suddenly removed the animal does not show an immediate loss of sexual arousal or rage but a more gradual cessation of the emotional state . This observation is, however, more consistent with a gradual dissipation of motivational tendency occurring unless this is maintained by positive feedback than with the alternative possibility of delayed positive feedback (which we would expect to be associated with a more sudden cessation of the emotional state) . Thus, sexual and aggressive behaviour may be maintained by their own momentum (represented by the appropriate emotional arousal) and also by positive feedback from the results of the behaviour (i .e. from the partner's activities) . Functionally, these two processes are probably incorporated into the two motivational systems because, compared with feeding and drinking, sexual and aggressive behaviour may require greater continuity of activity (because a partner and a discreet end-point are involved). Behavioural dominance might also be aided by other mechanisms. For example, research on chicks and mice has indicated that testosterone produces greater persistence of food searching (Andrew & Rogers 1972 ; Andrew 1972a ; Rogers 1974 ; Archer 1974, 1977) . It has also been suggested by Andrew (1972a) that this persistence serves the function of maintaining an activity such as aggressive or sexual behaviour once it has begun . Andrew (1972b) has described the effect of testosterone on persistence as entailing increased availability of search specifications once these have become activated (e .g . by stimuli previously associated with food). The precise mechanism involved has been discussed in detail by Andrew (1972b, 1976) ; the relevance of the phenomenon to the present discussion is that it may provide a mechanism for increasing behavioural dominance . The `persistence effect' provides a form of selective positive feedback, in that on-going behaviour is more likely to be maintained provided that the external stimuli match the animal's search specifications : if a testosteronetreated chick is searching for a particular colour of food grain, it will go on feeding from this colour for longer and be less likely to change to another type of grain than a control-injected chick would (Andrew & Rogers 1972) . However, once there is a mismatch between expected and observed stimuli, the animal will still be able to change to an alternative type of behaviour :

377

hence there is no general effect on resistance to extinction and therefore any increase in behavioural dominance will not occur irrespective of the environmental consequences of the behaviour. (One can speculate that the latter would have produced an inflexible animal unable to cease responding when it is advantageous to do so .) Feedforward The behaviour of a control system serves to minimize the disparity between some reference value and an actual value and it achieves this by responding to the difference between the values in such a way as to eliminate the difference . It may, however, be possible to take anticipatory action and so avoid an error even appearing . McFarland (1971, p . 102) drew attention to the feedforward mechanisms involved in temperature regulation, which serve this purpose . The body normally responds to a departure of deep body temperature from a set-point, but if the ambient temperature changes, skin temperature receptors send a message to the central controller so that it can take anticipatory heating or cooling action . In the case of the model for fear behaviour suggested by Archer (1976) feedforward could involve the effects of learning : if an animal responds to a conditioned stimulus by escaping, it would thus be able to prevent the discrepancy situation which originally evoked the unconditioned escape . Feedforward also appears to be valuable in the case of the interaction between feeding and drinking. Oatley & Toates (1969) showed that for rats denied access to water each gram of food eaten pulls 1 to 1 .5 ml of water from the blood into the gut . If the system were to be purely set-point actuated then following feeding a certain time would elapse during which water would be pulled into the gut and the animal would then drink in response to this loss from the blood . Two times the deficit, in other words the deficit plus the amount drunk (a total of 2 to 3 ml/g), would inevitably find itself in the gut and would then return to the blood . A much more efficient operation would be for the animal to anticipate the deficit and therefore drink to avoid it . The rapidity with which drinking follows feeding indicates that this is indeed what happens (Oatley & Toates 1969) . The gut is therefore not overloaded with water and no deficit occurs . Evidence of feedforward was also obtained by Fitzsimons & Le Magnen (1969) who found that increased drinking in response to high protein diets appeared to anticipate rather than follow renal loss.



3 78

ANIMAL BEHAVIOUR, 26, 2

In the case of feeding and drinking, feedforward is needed in order to by-pass the long delays in the feedback system . For aggression and fear, there is no delay to be avoided but rather the inevitable uncertainty connected with a choice mechanism to be resolved . Feedforward consists of biasing the choice in the light of past experience . However, in both cases, if the consequences are such as to be in conflict with those which would have resulted from negative feedback control then the feedforward mechanism is altered through relearning . For example, in the case of aggression, if the situation changes such that an attack strategy, which the animal has learnt as being successful through past experience, is no longer successful, its effect will be such that an increase rather than a diminution in discrepancy occurs . This will lead to the choice mechanism being biased towards fear rather than aggression on future occasions . Similarly, one imagines that if the kidney were to change its excretion characteristics so as to lose much less urine, the animal would recalibrate its meal-associated drinking for any given diet . Certainly if the diet is changed back from high to low protein content, the animal recalibrates to its previous value of the amount of water drunk for a given quantity of food eaten (Fitzsimons & Le Magnen 1969) . Summary and Conclusions We began our consideration of the application of control theory to motivation by referring to Lorenz's general model of motivation . This involved a building-up of a drive in the absence of performance and hence it stressed the internal spontaneous nature of causal factors for behaviour . More detailed examination of the factors influencing each particular motivational system showed that this general model was based on only a crude analogy of feeding, drinking, and sexual behaviour, and was inappropriate for aggression and fear behaviour . Instead, we discussed more complex models which incorporated classical control theory principles such as negative and positive feedback, and feedforward, but in different ways for the different types of motivation . The control theory approach to motivation was introduced into ethological studies by McFarland . His own work (e .g. McFarland 1970, 1971, 1974) and that of his associates (e.g . Houston & McFarland 1976 ; McFarland & Sibly 1972) has concentrated primarily on the analysis of motivational interactions, using

feeding and drinking in the Barbary dove as the main example. Problems central to ethological theory, such as the causation of displacement activities (McFarland 1966a, b), behavioural stability (McFarland 1969, 1971, 1974), motivational energy (McFarland 1970) and unitary drives (McFarland & Sibly 1972), have all been considered on an experimental and theoretical level . Our intention in this paper has been to take a more general view of control theory approaches to motivation, first by describing how models incorporating classical control theory principles can be applied not only to feeding and drinking but also to social behaviour such as aggression and sex . Having introduced this wider range of motivational systems, it was then possible to compare their overall design features and to relate these to some extent to the functional constraints imposed during evolution . In the course of this discussion we suggested that hunger, thirst, fear and aggression can all be accounted for in terms of a model that is disparity-activated, the object of behaviour being to restore optimum conditions for the organism ; hence the principle of negative feedback is incorporated as the main design feature . Following feeding and drinking, the organism displays satiety and decreased probability of responding but that is only because the natural disturbing influence is a gradual one . An insulin injection or a hypertonic saline injection would immediately reactivate hunger or thirst respectively, just as the appearance of a new unexpected stimulus would reactivate either fear or aggression soon after its cessation . In none of these cases does the actual neural mechanism exhibit a significant satiety, fatigue or inhibition as a result of responding. Since the function of sexual behaviour is not to restore optimum conditions, its design features will be different . The system is not essentially disparity-actuated, but it incorporates both negative and positive feedback . There is an absolute refractory period following an ejaculatory series in the male rat and other species during which sexual arousal is never exhibited. A relative refractory period follows during which arousal may occur, this being dependent upon the novelty of the partner amongst other things. In other words, arousal and satiety are more dependent upon changes in the neural mechanism in the case of sexual behaviour. In contrast to feeding and drinking, given a particular animal and opportunity for consummatory behaviour, there is little the experi-

TOATES & ARCHER : CONTROL SYSTEMS AND MOTIVATION

menter can do to reactivate sexual behaviour (apart from electrical stimulation) . Although a set-point cannot be identified in the case of sex, and it is difficult even to conceptualize, under normal conditions sexual behaviour appears to involve negative feedback in the way that hunger and thirst do . That is to say, a response decreases the probability of a second response in the near future, whereas deprivation increases the probability of a response . This is a reflection of the importance of internal factors, which is not apparent in the case of aggression and fear . In relation to hunger and thirst, we adopted Rolls' argument that the control of behaviour is mediated via activation of hypothalamic reward neurones, and this gives a joint responsibility to both internal and external factors in determining appetitive and consummatory behaviour. The sight of food, for instance, activates these neurones in the hungry animal . A change in diet causes a reactivation of an apparently sated hunger (Le Magnen 1971) and in a similar way sexual behaviour can be reactivated by a change of partner . Traditionally it has been assumed (e .g . Grossman 1968) that we can group motivations into those which are aroused and sated by factors within the organism (i .e. hunger and thirst) and those determined by external factors (i .e . sex) . This classification is perhaps deceptive in its simplicity since as we have pointed out external factors are also important for feeding, and internal factors for sexual behaviour . The above discussion has shown that at least for some purposes it may be more useful to adopt other classification criteria which are also appropriate for fear and aggression. Such a classification could be either in terms of a set-point control which includes thirst, hunger, fear and aggression but excludes sex ; or neural satiety which puts sex in a class of its own ; or the ability under normal circumstances to reactivate immediately and in full strength following completion of a response, which puts fear and aggression in a class of their own . This last classification serves to indicate the similarities under normal conditions of hunger, thirst and sex . Acknowledgments We thank Drs Peter Clifton, Robert Drewett and Peter Slater for their helpful comments in preparing this paper . REFERENCES Andrew, R. J . 1972a . Changes in search behaviour in male and female chicks, following different doses of testosterone . Anim . Behav ., 20, 741-750 .

379

Andrew, R . J . 1972b. Recognition processes and behavior with special reference to effects of testosterone on persistence. In : Advances in the Study of Behavior . Vol . 4 (Ed. by D . S . Lehrman, R. A . Hinde & E. Shaw), pp. 175-208 . New York and London : Academic Press . Andrew, R. J. 1976. Attentional process and animal behaviour. In : Growing Points in Ethology (Ed . by R . A . Hinde & P . P. G. Bateson) . Cambridge University Press. Andrew, R. J . & Rogers, L. 1972 . Testosterone, search behaviour and persistence . Nature, Lond., 237, 343-346 . Archer, J. 1974. The effects of testosterone on the distractability of chicks by irrelevant and relevant novel stimuli. Anim. Behav., 22, 397-404. Archer, J. 1976. The organization of aggression and fear in vertebrates . In : Perspectives in Ethology. Vol. 2 (Ed . by P. P. G . Bateson & P . Klopfer), pp. 231-298 . New York : Plenum Press . Archer, J . 1977. Testosterone and persistence in mice . Anim . Behav., 25, 479-488 . Booth, D. A., Toates, F. M. & Platt, S . V. 1976 . Control system for hunger and its implications in animals and man . In : Hunger: Basic Mechanisms and Clinical Implications (Ed. by D . Novin, W . Wyrwicka & G . Bray), pp . 127-143. New York Raven Press . Brown, R ., Freeman, S . & McFarland, D . J. 1974. Towards a model for the copulatory behaviour of a male rat . Motivational Control Systems Analysis (Ed. by D . J . McFarland), pp. 461-510. London and New York : Academic Press. Collier, G ., Hirsch, E. & Hamlin, P . H . 1972. The ecological determinants of reinforcement in the rat. Physiol. Behav., 9, 705-716. Corbit, J . D. 1965 . Effects of intravenous sodium chloride on drinking in the rat . J. comp . physiol. Psycho!., 60, 397-406 . Fitzsimons, J. T . 1961 . Drinking by nephrectomized rats injected with various substances . J. Physiol., Lond., 155, 563-579. Fitzsimons, J. T. 1972. Thirst. Physiol. Rev ., 52, 468-561 . Fitzsimons, J. T . & Le Magnen, J . 1969. Eating as a regulatory control of drinking in the rat . J. comp. physiol. Psycho!., 67,273-283 . Galef, B . G . 1970a. Familiarity of target location as a factor in the shock-associated aggression of wild rats . Psychon . Sci., 19, 299-300. Galef, B . G. 1970b . Stimulus novelty as a factor in the intraspecific pain-associated aggression of domestic rats . Psychon. Sci., 18, 21 . Geertsema, S. & Reddingius, H . 1974. Preliminary considerations in the simulation of behaviour . In : Motivational Control Systems Analysis (Ed. by D . J. McFarland), pp. 335-405 . London : Academic Press . Grossman, S . P . 1968. A Textbook of Physiological Psychology. New York and London : Wiley . Hebb, D . O. 1966 . Organization of Behaviour . New York : Wiley. Hinde, R. A . 1970 . Animal Behaviour . New York : McGraw-Hill . Houston, A . & McFarland, D . 1976. On the measurement of motivational variables . Anim. Behav ., 24, 459-475 .



380

ANIMAL BEHAVIOUR, 26, 2

Hull, C. L. 1951 . Essentials of Behaviour . New Haven, Conn ., Yale University Press . Johnsen, S . G. 1973. Nogle tanker om fedme. Ugerskrift far Laeger, 135, 2751-2754. Legrand, R. & Fielder, R. 1973 . Role of dominancesubmission relationships in shock-induced fighting of mice . J. comp. physiol. Psycho!., 82, 501-506. Le Magnen, J . 1971 . Advances in studies on the physiological control and regulation of food intake . In : Progress in Physiological Psychology . Vol. 4 (Ed. by E. Stellar & J. M . Sprague). London and New York : Academic Press . Le Magnen, J., Devos, M ., Gaudilliere, J. P ., LouisSylvestre, J . & Tallon, S . 1973 . Role of a lipostatic mechanism in regulation by feeding of energy balance in rats . J. comp. physio!. Psycho! ., 84,1-23 . Lorenz, K. 1966. On Aggression . New York : Harcourt, Brace & World . McFarland, D . J. 1966a . On the causal and functional significance of displacement activities. Z. Tierpsychol., 23, 217-235 . McFarland, D . J. 1966b. The role of attention in the disinhibition of displacement activities . Q . Jl. exp. Psycho!., 18, 19-30 . McFarland, D . J. 1969 . Mechanisms of behavioural disinhibition. Anim. Behav., 17, 238-242. McFarland, D . J. 1970 . Behavioural aspects of homeostasis. In : Advances in the Study of Behaviour . Vol. 3 (Ed. by D . S . Lehrman, R . A. Hinde & E. Shaw), pp. 1-26 . New York : Academic Press . McFarland, D . J . 1971 . Feedback Mechanisms in Animal Behaviour. London : Academic Press. McFarland, D. J. 1974. Time-sharing as a behavioural phenomenon . In : Advances in the Study of Behaviour, Vol . 5 (Ed . by D . S. Lehrman, J . S . Rosenblatt, R . A. Hinde & E . Shaw), pp . 201225. New York : Academic Press . McFarland, D . J . & McFarland, J. 1968 . Dynamic analysis of an avian drinking response . Med. Biol. Engng., 6, 659-668 . McFarland, D . J. & Nunez, A . T. 1977. Systems analysis and sexual behaviour. In : Biological Determinants of Sexual Behaviour (Ed. by J. Hutchison) . London : Wiley. McFarland, D. J. & Sibly, R. 1972 . Unitary drives revisited. Anim . Behav ., 20, 548-563. Moyer, K. E. 1968. Kinds of aggression and their physiological basis . Comm . Behav . Biol., A, 2, 65-87. Nicolaidis, S . 1971 . Parenteral alimentation and regulation of energy. Physiol. & Behav., 7, 939-940.

Nicolaidis, S . & Rowland, N . 1974. Long-term self intravenous drinking in the rat. J. comp . physiol. Psycho!., 87, 1-15 . Nicolaidis, S. & Rowland, N. 1975. Systemic versus oral and gastro-intestinal metering of fluid intake. In : Control Mechanisms of Drinking (Ed. by G . Peters, J. T . Fitzsimons & L. Peters-Haefeli), pp. 14-21 . Berlin : Springer . Novin, D ., Hutchins, C . & Mundy, P. 1966. Effects of intravenous injections of water on the consummatory behaviour of rats . J. comp . physio!. Psycho!., 61, 473-474. Oatley, K. 1974. Circadian rhythms and representations of the environment in motivational systems . In : Motivational Control Systems Analysis (Ed . by D. J . McFarland), pp. 427-459 . London : Academic Press . Oatley, K . & Toates, F. M. 1969. The passage of food through the gut of rats and its uptake of fluid . Psychon . Sci., 16, 225-226 . Rogers, L . 1974. Persistence and search influenced by natural levels of androgens in young and adult chickens. Physiol. & Behav., 12, 197-204. Rolls, E . T. 1975. The Brain and Reward. Oxford : Pergamon . Silverstone, T. 1976. Appetite and Food Intake. (Dahlem Konferenzen) . Berlin : Abakon Verlagsgesellschaft . Storr, A. 1968. Human Aggression . Harmondsworth, Middx . : Allen Lane . Toates, F . M . 1971 . Thirst and Body Fluid Regulation in the Rat. D.Phil . thesis (University of Sussex). Toates, F. M. 1974 . Computer simultation and the homeostatic control of behaviour. In : Motivational Control Systems Analysis (Ed . by D . J. McFarland), pp . 407-426. London : Academic Press. Toates, F . M. 1975 . Control Theory in Biology and Experimental Psychology . London : Hutchinson Educational Ltd. Toates, F. M . 1977 . A physiological control theory of the hunger-thirst interaction . In : Hunger Models : Quantitative Theory of Feeding Control (Ed . by D . A . Booth), London : Academic, in Press . Toates, F . M . & Booth, D. A. 1974. Energy control of feeding . Nature, Lond., 251, 710-711 . Toates, F. M . & O'Rourke, C. 1977 . Computer simulation of male rat sexual behaviour . Med. Bio!. Engng., In Press . Wiepkema, P. R . 1971 . Positive feedbacks at work during feeding . Behaviour, 39, 266-273 . (Received 20 December 1976 ; revised 27 May 1977 ; MS. number : 1595)