Motor Intelligence in a Simple Distributed Control System: Walking Machines and Stick Insects

MOTOR INTELLIGENCE IN A SIMPLE DISTRIBUTED CONTROL SYSTEM: WALKING MACHINES AND STICK INSECTS Holk Cruse and Jeffrey Dean Abteilung f'tir Biokybemetik und theoretische Biologie, Universit~t Bielefeld, Postfach 100131, D-33501 Bielefeld (GERMANY)

Abstract Walking in insects and machines is considered as an example of a difficult motor control problem involving many degrees of freedom and strong constraints by the environment. A control system based on experimental results from stick insects and other arthropods is proposed; results from simulations and initial tests in a walking machine are presented. The control system relies on decentralized control at three levels ranging from individual joints to interleg coordination, and including intraleg step pattern generation and interleg coordination of step timing and leg forces. This decentralized control immensely reduces the demands on central supervisory systems. Control modules and mechanisms are formulated in neural terms and optimized using connectionist learning rules and genetic algorithms. Several approximate algorithms suggested by biological results are used in place of classical, explicit solutions. The control system also relies heavily on the loop through the environment to simplify calculations. In particular, positive feedback at the level of individual joints is proposed as a way of exploiting the physical constraints to simplify the control problem.

240

H. Cruse and J. Dean

1 Introduction

For many years, the study of complex motor systems attracted fewer experimental biologists than the study of sensory systems. This disfavor was based on several features of motor systems. One is the essential role of the interaction of an organism and its environment--the loop through the real world--in most natural behaviors. Partly this importance arises by necessity because animals must adapt to complex, often unpredictable environments. Partly, as we will see below, it is a matter of convenience and efficiency, as when the dynamics of the interaction can be used to simplify and accelerate control. In either case the importance of the interaction with the environment requires that it be included in any analysis of complex motor systems. In the past, this additional complication has often been avoided by investigating motor behaviors where this loop plays a minor role, e.g., reflexes or simple ballistic movements or stereotypic, rhythmic activity. However, by focusing on simplified environmental situations, one may overlook essential properties of the system and improvements in computational and modeling techniques have provided better tools for dealing with the added complexity. The second feature is that complex motor systems often show a high degree of autonomy both in the selection and in the performance of motor behaviors. In many biological systems this autonomy represents an efficient strategy for handling predictable features of the environment and the interaction. However, it means that the experimenter does not have direct control of some important inputs to the motor system and therefore has less quantitative control of its state. Third, in most biological systems, the number of degrees of freedom which must be controlled is normally higher than that necessary to perform the task, i.e., the task alone does not specify a single movement. Therefore, systematic motor control involves selection among a host of alternative movements. This selection can be used to optimize additional parameters. In most biological systems the potential optimization criteria are numerous and often context-dependent. Thus, it is not easy to quantitatively formulate these additional constraints and, even where it is possible, the resulting optimization problem often cannot be solved quickly enough to enable real time control in applications to artificial systems. Fourth, to handle the complexity inherent in combining numerous environmental influences with autonomous activity and optimizing

Motor bztelligence in a Sbnple Distributed Control @stem

241

numerous and changing criteria, biological motor systems have evolved a complex structure of diffuse modularity. Anatomical modules can be identified on the basis of structure and patterns of connectivity among the constituent neurons but these anatomical modules rarely correspond in a clear, 1:1 manner to identifiable functional tasks. Thus, the whole system cannot easily be decomposed into self-contained submodules performing clear functions; it does not facilitate analysis the way a well-designed, artificial system, like a structured program, does. Indeed, the diffuse modularity often adds further complexity because the interaction among different modules can lead to emergent properties not attributable to the properties of the constituent units themselves. Despite these experimental and theoretical difficulties, the complexity makes the study of motor mechanisms especially challenging, particularly because they illustrate to a high degree the task of integrating influences from the environment, mediated through peripheral, sensory systems, with central processes reflecting the state and needs of the organism. Furthermore, it is exactly the autonomy of animals, expressed in their ability to act adaptively in complex environments without external control and to select among alternative actions in a seemingly efficient way, that in recent years has become a focus of interest in robotics and artificial life as possible models for artificial systems.

2 Walking as an example of a complex behavior

In the present chapter, we shall discuss a motor system for a complex behavior in which hard physical interactions play an essential role. We will describe how the system generates adaptable motor rhythms using a decentralized control system, i.e., the control is distributed among different subsystems, which can be considered to be agents in the sense of Minsky (1985). Furthermore, we will present work illustrating how the loop through the world is exploited to simplify computation drastically and to solve otherwise intractable computational problems. The results are based on experimental investigation of the behavior of the biological control system underlying the locomotion of six-legged arthropods in general and the stick insect in particular. Much of the information on the structure of the control system was derived from qualitative and quantitative analysis of the reactions to external disturbances.

242

H. Cruse and J. Dean

Such experiments usually provide much more insight into the computational and algorithmic structure of the system than does mere observation of the undisturbed behavior. Experimental analysis of the neural elements has provided additional insights and confirmed some of the hypotheses concerning control mechanisms, but many major questions arc still unanswered. In some of the behavioral studies it has been possible to identify and describe individual functional components within the complex behavior of walking. This leads to the important question of how these individual components cooperate to produce sensible behavior by the complete system. For even moderately complex systems, this question generally requires computer simulations, because interactions among even a few elements can easily lead to complicated, unintuitivc system properties. First of all, constructing a model forces a rigorous, quantitative specification of hypothesized mechanisms which can be tested for self-consistency in the set of separate hypotheses. Second, it can reveal and help to understand emergent properties of the system. Third, in cases where the attempt to construct a complete model of a complex system reveals qualitative discrepancies between the behavior of model and the observed behavior, it can point to the necessity of additional system properties and suggest further experiments. This feature is even more apparent when a software simulation is extended to the construction of a real robot. An final advantage of biologically motivated models, of course, is that they can be exploited by engineers and used in constructing animats autonomous, animal-like robots. Although walking, the behavior we discuss here, is sometimes regarded as simple and uninteresting, it involves a very strong and complex interaction with the physical environmcnL Most forms of locomotion share a reliance on sensory inputs to govern general features, such as direction and speed, as well as details such as activation and force of individual muscles, but, compared to other modes of locomotion like flying or swimming which occur in relatively homogeneous mediums, walking probably has the strongest dependence on sensory inputs. At the same time, some features of walking are regular and lend themselves to feed-forward control based on autonomous activity as a way of overcoming neural limitations in conduction and integration speeds. Thus, selection for speed and efficiency as well as for adaptability has favored a combination of autonomous elements together with simple reflexes and more complex sensory-driven modulations of central activity. The combination makes the walking system independent of particular stimulus inputs but at the same time enables the walking system to adapt to changes in the environment. Selection has also adjusted the balance between autonomous and sensory controlled mechanisms to match the standard walking conditions encountered by the animal.

Motor Intelligence in a Sbnple Distributed Control System

243

Thus, a study of walking provides a particularly good system for examining mechanisms for integrating autonomous activity with multimodal information from multiple sources including proprioceptors and exteroceptors. A study of the walking system might therefore be of interest not only in itself but also with respect to other, more complicated behaviors where these two types of activity must be integrated. From a biological point of view, walking is a behavior simple enough that one can hope to gain a complete understanding of the basic mechanisms. In fact, much information has been collected in recent years on how animals control leg movement during walking (review B~sler 1983, Cruse 1990, Graham 1985, Pearson 1993), although progress in elucidating neural mechanisms has been slow.

3 Structure of the control system

Rapid adaptability and flexibility in the face of substrate variation is an important aspect of walking. Behavioral and physiological studies indicate that this flexibility arises in part because control is distributed among several autonomous centers. In the case of the stick insect and other arthropods, it appears that the movement of each leg is controlled by a network in the respective hemi-ganglion. In this way, conduction delays to a master controller as well as the necessarily longer time for computation in such a controller are eliminated. These local networks are responsible for producing the two mutually exclusive movements making up a step--the power stroke and return stroke. These two movements correspond to stance and swing, the two states of the leg defined according to whether the leg has contact with the substrate. On a higher level the control systems for the individual legs have to cooperate to produce a suitable behavior of the overall system, propelling the body while maintaining postural stability. This decentralized control somewhat simplifies the analysis of the system and the problem of constructing controllers for artificial walking systems. The decentralization means that several functional problems which have to be solved can be addressed separately. One problem concerns the way the movement of the individual leg is controlled. The second refers to the coordination between legs. From biological experiments the following general answers can be given. First, each leg has its own control system which generates rhythmic step movements (B~sler 1977, v. Hoist 1943, Wendler 1964, reviewed by Graham 1985). The behavior of this control system

244

H. Cruse and J. Dean

corresponds to that of a relaxation oscillator in which the change of state, the transition between stance and swing, is determined by thresholds based on leg position. Second, several coordinating mechanisms couple the movement of the legs to produce a proper gait. The next sections consider each problem in turn.

3.1 Control of the step rhythm of the individual leg The geometry of the leg of a stick insect is shown schematically in Figure l a. The leg contains three joints, the subeoxal joint (or), the coxa-trochanter joint (13), and the femur-tibia joint (),). The axes of rotation for the latter two joints are parallel; thus all three leg segments--coxa, trochanter-femur, and tibialie in a plane. The rotational axis of the subcoxal joint is not aligned with the vertical axis of the body and is determined by the angles tp and ~. The step cycle of the walking leg can be divided into two functional states. During the stance or power stroke, the leg is on the ground, supports the body and, in the forward walking animal, moves backwards with respect to the body. During the swing or return stroke, the leg is lifted off the ground and moved in the direction of walking to where it can begin a new stance. Following B~sler (1972), the anterior transition point, i.e., the transition from swing to stance in the forward walking animal, is called the anterior extreme position (AEP) and the posterior transition point is called the posterior extreme position (PEP). In the pattern generator for stepping in the stick insect, unlike those for many rhythmic behaviors, intxinsie, autonomous activity is weak or nonexistent (B~ssler & Wegner 1983). Complete step rhythms have not been demonstrated although certain elements of the step cycle, involving particular patterns of changes in the activity of some leg muscles, have been found in the absence of sensory input (Biisehges et al. 1995). Therefore, the criterion for changing state has been defined in terms of configurations of sensory input reflecting the state of the leg. Either as a hypothesis or on the basis of experimental results several authors (Wendler 1968, B~issler 1977, Graham 1972, Cruse 1985) proposed the idea that the transition from one state to the other occurs when the leg reaches a given threshold position and that the step generator can thus be considered a relaxation oscillator. The role of the load on the leg in modifying this threshold position will be neglected here (see, however, B~sler 1977, Cruse 1983, Dean 1991a, Pearson 1972, Schmitz 1993).

Motor Intelligence in a Simple Distributed Control System

245

Diverse propdoceptive sense organs in the leg provide measures of joint angles and leg loading. The angle measures can be used for the direct kinematic transformation to obtain tarsus position, although it is not known whether this parameter is explicitly represented in the neural activity (see below). In walking straight and in most other situations, the movement of the basal leg joint, the subcoxal joint (see Hg. l a), is unidirectional like the movement of the tarsus, so threshold positions of the latter can be approximated by corresponding angles for the subcoxal joint. Differences in the constraints acting during the two states and in the conditions for their termination suggest that the leg controller may consist of separate control networks (although this separation may only be justified on a logical level and may not reflect a morphological separation in the nervous system). Thus, the relaxation oscillator making up the step pattern generator is assumed to consist of two agents-a swing network controlling the return stroke, and a stance network controlling the power stroke. The transition between swing and stance is controlled by a third agent, the selector network (Hg. l b). The swing network and the stance network are always active, but the selector network determines which of the two agents controls the motor output. To match the experimental results indicating the absence of a robust central pattern generator producing strong intrinsic rhythms, the selector net acts on the basis of sensory input. The selector net was developed by applying learning algorithms to train artificial neural nets. We started with a fully recurrent network consisting of four units: two motor command units, RS and PS, active during swing and stance, respectively, and two sensory units, a PEP unit monitoring whether the PEP threshold position has been passed and a C~ unit signaling ground contact. Using a simple training procedure, the delta rule, the selector net shown in Fig. lb was obtained. It consists of a sensory and a motor layer with recurrent, positive feedback connections in the motor layer. These positivefeedback connections serve to stabilize the ongoing activity, namely stance or swing. Similar circuits with internal positive feedback have been discussed for the motor system of vertebrates (Houk et al. 1993). A comparison with the Brown half-center oscillator based on reciprocal inhibition (Brown 1911, see also Biissler 1986a), which is the classical solution to this problem of producing alternating motor output, showed that the selector net with positive feedback is three times more stabile in the face of extemaUy applied noise (Cruse et al. 1993a).

H. Cruse and J. Dean

246

rostral thorax

L

~(Z

, caudal

z

.y

7

,-u,

a)

r P, u

~ ~,",/

target net ;~~ :

swing net

: : i

r-x

i

:

I--------~ m.d.

-

|i

at

e

"

o o

P" ..............

GC

:

,

|

|

,---------4

, ~ 1

: l

e

oOOe

i

o o o ~o

selector

o m o.o

o o qum0. o ~ e o

: o o o

o

coorclinating influences

:

:

o

net

qlm0o o ~ ~ ~ o o o o .

: o |

e

*

RS

1

i

:

"

j

: L :..............

,

1

~

,

: .,'

.:

! o ,

,

:

o

:

o

~

alulo InDIN

I

,+

l e

e

o

o

PS

~

: e

o

t+~l!

- - - . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,

direct kinematic

b)

ui

:

i stanc e : net

o

o

a

7

Figure 1: Summary of leg ~ometry and the control networks for the model. (a) Schematic drawing of a mechanical m ( ~ l of a stick insect leg showing the an'an~rr, ent of the joints and their axes of rotation. (b) The leg controller consists of three parts: the swing net which specifies the leg movement for a return stroke, the stance net which specifies leg movement for a power stroke, and the selector net which determines

Motor Intelligence in a Sbnple Distributed Control ,,~

247

whether the swing or the stance net control the motor output, i.e., the velocity of the three joints ~ 13, and y. The selector net contains four units: the PEP unit signaling posterior extreme position, the GC unit signaling ground contact, the RS unit controlling the return stroke, and the PS unit controlling the power stroke. The target net transforms information on the configuration of the anterior target leg, given by the angles oq, 131,and Yl, into angular values for the next caudal leg (~, 13t,Yt) which place the two tarsi close together. These desired final values and the current values of the leg angles, ~ 13, and y, are input to the swing net together with bias inputs and a sensory input (m.d.) which is activated by an obstruction blocking the swing and thereby initiates an avoidance movement. NL is the nonlinear compensation to make the simulated velocity profiles more realistic.

1. Return stroke inhibits start of return stroke I

1,;

3

t

2. Start of power stroke excites start of return stroke 3. Caudal positions excite start of return stroke 4. Position influences position at end of return stroke ('targeting") 5. Increased resistance increases force ("coactivation')

6. Treading-on-tarsus reflex

Figure 2: Sununmy of the coordination mechanisms operating between the legs of a stick insect. The leg controllers are labeled R and L for right and left legs and numbered from 1 to 3 for front, middle, and rear legs.

3.2 Interleg coordination of stepping Before discussing the control of the actual leg movements, we will describe how the beginning and end of the swing movement are controlled. In all, six different coupling mechanisms have been found in behavioral experiments with the stick insect (reviewed by Cruse 1990, Mtiller-Wilm et al. 1992). These are summarized in Figure 2. One mechanism serves to correct errors in leg placement; another has to do with distributing propulsive force among the legs. These will not be considered here. The other four mechanisms were

248

H. Cruse and J. Dean

successfully implemented in an earlier model (Dean 1991bc, 1992ab) which formed the basis for the coordination module in the present model. The beginning of a swing movement, and therefore the end-point of a stance movement (PEP), is modulated by three mechanisms arising from ipsilateral legs: (1) a rostraUy directed inhibition during the swing movement of the next caudal leg, (2) a rostraUy directed excitation when the next caudal leg begins active retraction, and (3) a caudally directed influence depending upon the position of the next rostral leg. Influences (2) and (3) are also active between contralateral legs. These interleg influences are mediated in parallel by neuraUy and mechanically mechanisms. In the former, the neural step pattern generator of one leg is influence~ by the activity in that of a neighboring leg via neural excitation and inhibition transmitted directly through the central nervous system. These neural connections convey specific information about the state of the sender. They apparently act only between step pattern generators of legs that are adjacent either along or across the body. In the mechanically mediated influences, each step pattern generator simply responds to the load on the leg it controls, but in the free-walking animal this load varies with the actions of the other legs. The load changes on one leg do not provide specific information on the state of another leg-as the centrally mediated influences do, but presumably they most strongly reflect the state and position of the adjacent legs. For example, when the right rear leg swings, the load on the other legs and on the right middle leg, in particular, will generally increase. Because increased load on a leg inhibits or delays the start of its swing, the effect on the right middle leg is similar to the rostrally directed inhibition (mechanism 1) transmitted directly through the nervous system. In an analogous manner, load changes sensed locally can produce effects similar to those of the other centrally mediated mechanisms. This combination of mechanisms adds redundancy and robustness to the control system of the stick insect (Dean & Cruse 1995). In the network model, influences (1), (2), and (3) act on the PEP unit of the selector net (Fig. lb), as explained in detail by Cruse et al. (1995a). I.z~d effects have not yet been implemented in the model or in its extension to the control system for the TUM (Technical University of Munich) walking machine mentioned in the Discussion. The end of the swing movement in the animal is modulated by a single, caudally directed influence (4) depending on the position of the next rostral leg; this mechanism is responsible for the targeting behavior-the placement of the tarsus at the end of a swing close to the tarsus of the adjacent mstral leg. In the model, this influence reaches the leg controller as the input to the swing net (Fig. l b), which will be explained in detail below. It was shown earlier by

Motor Intelligence in a Sbnple Distributed Control ~ystem

249

Dean (1990) that a simple feextforward net with three hidden units and logistic activation functions (Fig. lb, "target net") can transform the values of the joint angles of a leg to joint angles for the next caudal leg such that the tarsi of the two legs are at the same position. This is possible with an exactness corresponding to that found in biological experiments (Cruse 1979, Dean & Wendler 1983), which shows that an approximation can provide adequate control and replace an explicit solution of the coordinate transformations involved in the direct and inverse kinematic problems. Physiological recordings from local and intersegmental intemeurons (Brunn & Dean 1994) support the hypothesis that a similar approximate algorithm is implemented in the nervous system of the stick insect. For the present model, a target net was trained for each of the four ipsilateral leg pairs using classical back-propagation and a training set corresponding to leg positions in three dimensions. Positions of the rostral leg outside the reach of the caudal leg were also included in the training set. As the rostral leg moved farther out of reach, the target for the caudal leg shifted smoothly from a position near its rostral boundary toward a default value slightly forward of its mean AEP (see also Dean 199 lc). Figure 2 gives a schematic overview of this control structure which, together with the leg controllers, forms an asymmetric recurrent network. Counting only the four interleg coordination mechanisms implemented in the simulation, the system has a total of 34 connections between the different pairs of neighboring legs.

4 Control of leg movement

Physiological studies have not yet clarified the structure and function of the neural networks controlling leg movement. Individual premotor intemeurons have been found which elicit coordinated patterns of activity in motoneuron pools for several different muscles, leading to coordinated multijoint movements (Burrows 1992). In some cases, patterns of sensory inputs to these interneurons have been describext. A general scheme for the organization of motor and premotor networks has been proposed (Burrows 1992), but it is still far from clear how the pattern of activity in the whole network is created and modulated in making a natural leg movement. Results for still simpler movements in leeches indicate that the patterns of activity and their changes are not simply represented in terms of the activity of individual neurons

250

H. Cruse and J. Dean

(Kristan et al. 1992). Even connections of intemeurons involved in simple single joint reflexes are complex and often counterintuitive (Biischges & Schmitz 1991), providing ample substrate for evolutionary change but greatly complicating the analysis of network function. In the absence of detailed physiological information, the approach followed here is to construct control models for leg movement that are consistent with available information on the stick insect. The goal is to develop guidelines for interpreting physiological findings and control systems for walking machines. 4.1 Control of the swing movement The task of finding a network that produces a swing movement seems to be easier than finding a network to control the stance movement because a leg in swing is mechanically uncoupled from the environment and therefore, due to its small mass, essentially uncoupled from the movement of the other legs. The geometry of the leg is shown in Fig. la. The coxa-trochanter and femur-tibia joints, the two distal joints, are simple hinge joints with one degree of freedom corresponding to elevation-depression and to extension-flexion of the leg, respectively. The subcoxal joint is more complex, but its major movement is in a rostrocaudal direction around the nearly vertical axis. The additional degree of freedom allowing changes in the alignment of this axis is little used in normal walking (Cruse & Bartling 1995), so the leg can be considered as a manipulator wi'th three degrees of freedom for movement in three dimensions. Thus, the control network must have at least three output channels, one for each leg joint. We again used a fuUy connected recurrent network and linear activation functions. Initially, the net included two additional hidden units. However, it turned out that these hidden units are not necessary, so no hidden units appear in the final version of the swing net (Fig. l b, "swing net'S. In the simulation, the three outputs of this net, are interpreted as the angular velocities: A~ AI3, and AT. They are passed to an integrator (not shown in Fig. l b) to obtain the joint angles ( ~ 13, y) at each time interval; the current joint angles are measured and fed back into the net. The integrator in the simulation together with its input and output corresponds to the leg of the animal together with activation representing a kind of velocity control and the resulting leg position. Thus, the feedback loop does not correspond to an internal neuronal channel, but passes via the periphery, i.e., the leg and its position which are represented by the integrator. Besides the input to the swing net provided by three sensors measuring the current angles of the three leg joints, three other input units (o,, 13,,yd represent the target of the swing movement, i.e., the leg

Motor Intelligence in a Sbnple Distributed Control System

251

configuration that should be achieved at the end of the return stroke. These values might correspond either to a fixed AEP, or to a variable AEP depending on the information coming from the next anterior leg via the target net. In order to find a net which produces a swing movement similar to that found in the biological experiments (Cruse & Bartling 1995), the net was trained using a random search procedure and an appropriate evaluation function. The training procedure included steps of about 30 different forms, chosen to allow for varying stride lengths and, in particular, for the different AEPs and PEPs occurring during curve walking, as described by Dean and Rixe (in preparation). Because the geometries of front, middle, and rear legs differ, separate swing nets were trained for each leg controller. Using such a linear system, the form of the leg trajectory closely approximated the biological data, but the velocity profile did not. The velocity was highest at the start and then decreased monotonically. Therefore, a nonlinear compensation (Fig. l b, NL) was introduced; the output of all three motor units was multiplied by a common factor depending on the actual leg position relative to the target position. Fig. 3 shows trajectories of the tarsus during a retum stroke projected into the x-z plane (side view) and the x-y plane (top view). As can be seen, the movement continues past the target position used for training. This means that the target is not like the equilibrium point in some models of movement control (Bizzi et al. 1992); it suggests that movement velocity in a given direction is also maintained to some extent. Movement past the end-point is sensible because the movement should continue until ground contact is signaled. This behavior represents one useful feature of the network approach compared to an algorithmic computation of a movement to a predicted end-point. When the training procedure is finished, each net has 21 fixed weights. Nevertheless, this simple system is able to generalize over a considerable range of untrained situations, demonstrating a further advantage of the network approach. Figure 3 shows several examples including swing trajectories necessary for curve walking. Furthermore, the swing net is remarkably tolerant with respect to external disturbances. Fig. 3e shows an example in which all three angles were suddenly changed. Nevertheless, the leg completed the swing movement with only a small error compared to the unperturbed movement. The unperturbed trajectory represents a kind of attractor to which the disturbed trajectory returns, similar to the properties of the force fields produced by local stimulation of the frog spinal cord (Giszter et al. 1995). This compensation for disturbances occurs because the system does not compute explicit trajectories, but simply exploits the physical properties of the world. Errors resulting from such disturbances, of course, will be greater, the larger the disturbance and the later it occurs within the return stroke. However, such

252

H. Cruse and J. Dean

a)l~ fr~ 0

g

!,0

9.'i'. .....

"OlOQQQO

I

0

10

.i

":.':!.',

J 3O

x[mm]

0

04104) 0 0 0

-10 0

-g N

. ,ioOO~

0 0"~

i

i'

..i~.

d) front leg

!

e"

oee,,

_

~o

I ..:

~)

-20

-10

0

10

10

2O

x[mm]

i

9 9 oo-eooeeoe

I

o""" . . . .

ooOo~o. ~ ~ 9ooeool

9

9i t

i

,

i!

"~

J

t

I

-10

0

"'"~

,

~"~-~"!

~

l1

?

0

-20

4"

'

-10

20

-30

,o

-~

x[~l

-10

O"

I

-20

~"'"

o

3O

c) middle leg

i! o'

i i

~" ,o

20

~~176 ~

-10

-20

10

9

/

"/" [

0

OO 9

'Dee

J,

>,,,

10

8jOOe ~ 1 7 6

"" i " " ..... :"':~

,o

1

I

,

20

i .......i

2O

b)'~ hind leg

front I.g

9

.)i

-30--

_

e)

1

"******* *

!

i

-20

-10

0

10

0

i

1

! 3O

l~gure 3: Side (z-x) and top (y-x) view of the tarsus trajectory during swin~ The starting positions are shown by open squares, the end positions corresponding to o~, 13,, y, (see Fig. l b) are shown by closed squares. All legs are left legs. The origins of the coordinate systems correspond to the positions of the subcoxal joints at the base of each leg. Anterior is to the ri~t. (a) The movement of a front leg for different starting positions. Co,c,d) The movements of left rear, middle, and front legs during curve walking (rio~t turn). (e) A trajectory of the front leg including a response to a disturbance--a sudden chan~ described by Aa =-12 ~ A[3 = 7 ~ Ay= 14~

Motor Intelligence in a Sbnple Distributed Control b'ystem

253

errors in leg placement do not strongly influence the overall behavior of the walking system as long as the leg successfully attains ground contact. This ability to compensate for external disturbances permits a simple extension of the swing net in order to simulate an avoidance behavior observed in insects (e.g., Dean & Wendler 1982). When a leg strikes an obstacle during its swing, it initially attempts to avoid it by making a short retraction and elevation and then renewing its swing forward from this new position. In the augmented swing net, an additional input similar to a tactile or force sensor signals such mechanical disturbances (m.d. in Fig. lb). This unit is connected by fixed weights to the three motor units in such a way as to produce a brief retraction and elevation, simulating the avoidance reflex.

5 Modeling results

The following examples show the ability of the system described so far to perform stable and coordinated walks. In these initial simulations, the stance movement is determined by explicitly solving the inverse kinematics to find the joint configurations to move the tarsus along a straight line parallel to the long axis of the body (alternative solutions to the control of stance movement are discussed below). The model shows a proper coordination of the legs when these are walking at different speeds on a horizontal plane. Steps of ipsilateral legs are organized in triplets forming "metachronal waves" which proceed from back to front, whereas the contralateral legs on each segment step approximately in alternation. With increasing walking speed, the typical change in coordination from the tetrapod to a tripod-like gait (Graham 1972, Wendler 1964) is found. For slow and medium velocities (Fig. 4ab) the walking pattern corresponds to the tetrapod gait with four or more legs on the ground at any time and diagonal pairs of legs stepping approximately together; for higher velocities (Fig. 4c) the gait approaches the tripod pattern with front and rear legs on each side stepping together with the contralateral middle leg. The coordination pattern is very stable. For example, when the movement of the fight middle leg is interrupted briefly during the power stroke, the normal coordination is regained immediately at the end of the perturbation (Fig. 4d). The results of simulations are similar regardless of whether the coordination mechanisms are formulated algorithmically in terms of the functions affecting the threshold positions (Dean 1991bc, 1992ab) or in terms of networks of neuron-like units as shown here. The form of the interactions,

H. Cruse and J. Dean

254

a)

r

c)

,

L1

L2

1.2

L3

L3

R1

R1

R2

R2

R3

R3

b)

d)

L1

L1

L2

L2

L3

L3

R1

R1

R2

R~

R3

R~ time

time

Figure 4: Movement of the legs in simulations of unperturbed walking at different speeds and a response to a perturbation. The abscissa is time in relative units. The ordinate is tarsus position along the x-axis. The six traces refer to left (L) or right (R) legs. The numbers indicate front (1), middle (2), and rear legs (3). (a,b, and c). Slow, medium, and high walking speeds. The walking pattern gradually chants from tetrapod to a tripod-like gait. (d) Stability of the coordination pattern when the power stroke of the fight middle leg is interrupted for a short time. in which neural inhibition and excitation, for example, correspond to caudal and rostral shifts of the position threshold for beginning swing, lends itself to the network formulation. A particularly difficult problem for the walking system (MiiUer-Wilm et al. 1992) is to begin a walk from different starting configurations, i.e., positions of the six legs relative to the body. In earlier versiom of the network model (Cruse et al. 1993a), many starting configurations led to unstable positions during the first steps. These are defined as configurations in which the center of gravity was not over the polygon formed by the tarsi of the legs in stance. For the insect, this may not be a serious problem because the supporting legs can attach to most natural substrates and pull as well as push; for the TUM walking machine, ground contact is passive, so it is important to maintain an adequate arrangement of supporting legs. To improve the situation, the

Motor bffelligence in a Sbnple Distributed Control System

255

weights describing the strength of the different coordinating mechanisms were optimize~ using genetic algorithms. With the best set of weights found to date, 92% of the simulated sequences contain no instabilities and less than 1% include instabilities within one step cycle amounting to more than 40% of the mean swing duration. These values hold for walks at medium velocities (the ratio of swing duration to stance duration was 0.4 : 1) started from randomly selected leg configurations. At lower walking speeds (the ratio of swing duration to stance duration was 0.2 : 1), no walks contain instabilities. At higher walking speeds, the percentage of stable walks decreases. In summary, adaptability of the control system can be shown not only for disturbances during a walk (Fig. 4d), but also for variations in starting configurations.

6 Control of movement and interleg coordination during stance

The task of controlling leg movement during stance involves several problems. It is not enough simply to specify a movement for each leg on its own: the mechanical coupling through the substrate means that efficient locomotion requires coordinated movement of all the joints of all the legs in contact with the substrate. When aU six legs are on the ground, the movements of 18 joints have to be coordinatext. However, the number and combination of mechanically Coupled joints varies from one moment to the next, depending on which legs are lifted. The task is quite nonlinear, particularly when the rotational axes of the joints are not orthogonal, as is often the case in insects, particularly for the basal leg joint. A further complication occurs when the animal negotiates a curve because then the different legs move at different speeds. In machines, these problems can be solved using traditional, though computationaUy costly, methods which consider the ground reaction forces of all legs in stance and seek to optimize some additional criteria, such as minimizing the tension or compression exerted by the legs on the substrate. Examples together with several optimization criteria discussed in the context of analyzing the ground reaction forces of the stick insect can be found in Pfeiffer et al. (1991, 1994; see also Eltze 1995). Due to the nature of the mechanical interactions and the search for a globally optimal control strategy, such algorithms require a single, central controller; they do not lend themselves to distributed processing. This makes real-time control difficult, even for the relatively simple case of walking on a rigid substrate.

256

H. Cruse and J. Dean

Further complexities arise in more complex, natural walking situations,

making solution difficulteven with high computational power. These occur, for example, when an animal or a machine walks on a slippery surface or on a

compliant substrate, such as the leaves and twigs encountered by stick insects. Any flexibility in the suspension of the joints further increases the degrees of freedom that must be considered and the complexity of the computation. Further problems for an exact, analytical solution occur when the length of leg segments changes during growth or their shape changes through injury. In such cases, knowledge of the geometrical situation is incomplete, making an explicit calculation difficult, if not impossible. Despite the evident complexity of these tasks, they are mastered even by insects with their ,,simple" nervous systems. Therefore, there has to be a solution that is fast enough that on-line computation is possible even using slow, biological neurons. How can this be done? Several authors (e.g., Gibson 1966 for perception, Brooks 1991 for controlling action) have pointed out that some relevant parameters do not need to be explicitly calculated by the nervous system because they are already available in the interaction with the environment. This means that, instead of relying on an abstract calculation, the system can directly exploit the dynamics of the interaction and thereby avoid a slow, computationally exact algorithm. To solve the particular problem at hand, we propose to replace the central controller with distributed control in the form of local positive feextback (Cruse et al. 1995b). Compared to earlier versions (Cruse et al. 1993a, 1995a), this change permits the stance net to be radically simplified. The positive feedback occurs at the level of single joints: the position signal of each joint is fed back to control the motor output of the same joint (Fig. l b, stance net). How does this system work? Let us assume that any one of the joints is moved actively. Then, because of the mechanical connections, all other joints begin to move passively, but in exactly the proper way. The movement direction and speed of each joint do not have to be computed because this information is akeady provided by the physics. The positive feedback then transforms this passive movement into an active movement. There are, however, several problems to be solved. The first is that positive feedback using the raw position signal would lead to unpredictable changes in movement speed, not the nearly constant walking speed which is usually desired. This problem can be solved by introducing a kind of bandpass filter into the feedback loop. The effect is to make the feedback proportional to the angular velocity of joint movement, not the angular position. In the simulation, this is done by feeding back a signal proportional to the angular chalage over the preceding time interval, i.e., the current angle minus the angle at the previous time step.

Motor b:telligence in a Sbnple Distributed Control System

257

The second problem is that using positive feedback for all three leg joints leads to unpredictable changes in body height, even in a computer simulation neglecting gravity. A physical system, of course, would be pulled downward by gravity and the positive feedback would accelerate this movement. In summary, a system with positive feedback at all joints does not maintain a constant body height even in the absence of gravity; it is even less able to do so under the influence of gravity. Such control, of course, is an essential characteristic of a capable walking system. During standing, the stick insect as a whole behaves like a proportional controller to compensate for changes in load (BEssler 1965, Wendler 1964). In the standing or inactive animal, this spring-like characteristic is also true of each leg separately (B~sler 1983, Cruse 1976, Cruse et al. 1989, 1992, 1993b, Wendler 1964); it results from resistance reflexes providing negative feextback at each of the leg joints. Similarly, during walking, body height is controlled by a distributed system in which each leg acts like an independent, proportional controller (Cruse 1976, Cruse et al. 1993b). Thus, no master height controller is necessary; the only central information is the invariant reference value for each leg. For an individual leg, however, maintaining a given height via negative feedback appears at odds with the proposed local positive feexlback for forward movement. How can both functions be fulfilled at the same time? To solve this problem we assume that during walking positive feedback is provided for the ot joints and the y joints, but not for the 13joints (Fig. 1). The 13 joint is the major determinant of the separation between leg insertion and substrate, i.e., body height. The action of the ),joint in extending the leg is less important. Thus, in the control scheme proposed here, the negative feedback at the 13joint present in the standing animal is continued during walking. In this way, it is possible to solve all the problems mentioned above in an easy and computationaUy simple way. Two tasks remain for a central controller. One is how to initiate walking. For this purpose, a 'starting impulse' (not shown in Fig. lb) has to be applied to at least some joints. Figure 5 shows a 3D view of a six-legged system performing a stance with all six legs on the ground after the two front legs (a), or one front leg (b), are stimulated by a brief starting impulse. A second task is to ensure proper speed and direction. As an extreme example of the problem inherent in using positive feeAback, let us assume, for example, that a standing insect is held at the abdomen and pulled slightly to the rear by an experimenter or by gravity. Under positive feedback control as described above, the insect should then continue to walk backward. This has never been observed. Therefore, we assume that a supervisory system exists which is not only responsible for switching on and off the entire walking

258

H. Cruse andJ. Dean

system (Schmitz & Hagfeld 1989), but also determines walking speed and roughly specifies walking direction.

Figure 5: Movement of a six-legged system subject to gravity when negative feedback is applied to all six 13joints and positive feezlback applied to all a and y joints. The movement direction is from left to fight. Only the three fight legs are shown in top view (upper panels) and from the side (lower panels). (a) The starting impulse of-3 degrees is applied to the ~ joints of both front legs. All legs perform a stance movement. (b) The starting impulse of-7 is applied only to the a joint of the right front leg. The system shows the beginning of a left turn.

6.1 Evidence for positive feedback in stick insects during walking It is not clear how the hypothesized supervisory system works within the biological system, but there are some hints which will guide further experiments. The idea of using positive feedback to support active leg movement goes back to experimental findings of B~sler (1976). He showed that, in an active animal, elongation of the femoral chordotonal organ of the femur-tibia joint, which occurs during joint flexion, leads to an inhibition of activity in the extensor muscle, which thereby facilitates rather than opposes continued joint flexion. Further investigations (B~sler 1986ab, 1988, 1993) have made it very probable that this inhibition of the extensor muscle and the concomitant activation of the antagonistic flexor muscle result from positive feedback. Negative feedback is seen in a number of other experimental situations, however, so the interpretation is difficult (for discussion see Cruse et al. 1995b). One resolution of this contradiction would be to interpret the results as "phenomenological" positive feedback generate~ in a negative feedback system. By this we mean that the movement might induce a shift in the reference value for a negative feedback system which causes it to assist

Motor h~telligence in a Simple Distributed Control System

259

rather than oppose the change in position. This interpretation, however, was ruled out by an experiment (Schmitz et al. 1995) which directly demonstrateA the existence of logical positive feedback. How can we explain the f'mdings showing negative feedback? It was already shown by B~sler (1986b, 1988) that negative feeAback responses do occur in the active animal, as in the inactive animal, for very low or very high stimulus velocities. One may speculate that negative and positive feedback channels exist in parallel and furthermore that the positive feedback channel is switched on and off according to the state of the animal and that it is filtered so as to limit the range of movement velocities produced. If the deviations in velocity produced by external influences are too large, the negative-feexiback channel becomes dominant. This already would provide a kind of check that movement is appropriate which would look like supervision by higher centers. Using positive feedback, as already mentioned, solves a number of difficult problems during the control of a mechanically complicated system; it obviates the neeA for a master controller to seek a global solution, leaving it to perform a merely supervisory role. This supervision would include checking that position values are within suitable ranges, but this comparison can presumably be done with an approximation requiring less computation than if the master system had to control all details of movement at all the joints.

7 Discussion

Walking is a typical example o f a behavior which is not easily analyzed in terms of formal rules. Although the step cycle can be reduced to two states, swing and stance, which suggests that coordination patterns can be analyzed using simple logical rules, the actual criteria for transitions between these states are, as we have seen, many-valued and the factors influencing leg motion are still more complex. Thus, any biological system which successfully controls walking can be said to illustrate a high degree of ,,motor intelligence." Here we have discussed how a system based on approximations and decentralized control in place of exact solutions can be constructed to govern the movement of a six-legged system walking in an unknown environment without the aid of vision or other exteroceptive information for preprogramming movement. As far as possible, the construction of this system relies on biological data obtained from experiments with insects.

260

H. Cruse and Y. Dean

Specifically, the results show that the information obtained from biological experiments can be incorporated into a 6-legged model which is able to walk at different speeds over irregular surfaces. The model shows a stable gait even when the movement of the legs is disturbed. The success of the simulations indicates that the control system described here can be applied to a real walking machine. The earlier, algorithmic version described by MUUer-Wilm et al. (1992) has been implemented in the TUM hexapod walking machine (Pfeiffer et al. 1994) and found to provide good leg coordination. The network model has not yet been tested in the machine. However, we are quite confident that the ANN version will be able to control the walking machine successfully, because the simulation results of both the algorithmic and the new ANN model are very similar. 7.1 Control based on recurrent, redundant and distributed networks

In order to produce an active behavior, i.e., a time-varying motor output, a system needs recurrent connections at some level. A pacemaker neuron incorporates these recurrent influences within a single neuron, one which can produce simple rhythms on its own. More interesting are n e t w o r k s with or without pacemakers - where the recurrent connections involve nonlinear interactions among two or more units. In such systems, recurrent connections may occur in two ways. They may occur as internal connections, as in the various types of artificial recurrent networks (e.g., Hopfield nets or Elman nets). Alternatively, if a simple system with a feedforward structure is situated in a real or simulated environment, information flow via the environment serves as an "external" recurrent connection because any action by the system changes its sensory input, and thereby closes the loop. We have described a system for hexapod walking which contains recurrent connections of both types. The system is constructed of elements representing simplified artificial neurons with nonlinear activation functions. This control is distributed on each of the three levels of organization considered here. The first (uppermost) level concerns the control system of the six legs. There is no central controller but each leg has its own, independent control system which is able to generate rhythmic stepping movements. These six systems are coupled by a small number of connections forming a recurrent network. Together with the ever-present external recurrent connections--during walking the legs are mechanically coupled via the substrate--this recurrent system produces a proper spatiotemporal patterns corresponding to gait patterns found in insects.

Motor bztelligence in a Simple Distributed Control System

261

On the intermediate level, each leg controller by itself consists of three subsystems, or agents, one for the swing movement, and one for the stance movement. A third subsystem gates the outputs of the swing and the stance system such that only one is able to influence the motor output at a given time. The lowest level considered here lumps individual muscles, i.e., all agonists or antagonists, as well as the motor units within a muscle into a single functional unit moving a joint to a new position. At this level, the swing net consists of an extremely simple feedforward net which generates swing movements in time by exploiting the recurrent loop via the sensorimotor system. This loop provides information on the state of the system as it is influenced by the motor commands and external factors. The system does not compute explicit trajectories; instead, it computes changes in time based on the actual and the desired final joint angles. Thus, the control system exploits the physical properties of the leg. This organization permits a very simple network and it responds in an adaptive way to external disturbances. In a similar manner, the use of positive feedback at the level of individual joints in during stance permits a still more extreme distributed control based on exploiting the constraints present in the system and its interaction with the surroundings. The proposed decentralized control scheme still requires some central commands from a superior level, as noted in Section 6. These global commands are necessary to determine the beginning and ending of walking as well as its speed and direction. However, these commands do not have to be adjusted to the details of the configurations and tasks of all the legs because an approximation suffices. With the use of positive feedback to control stance, for example, it may not even be necessary for all legs to receive a command for turning. Initial simulations related to this and other questions are considered in more detail in Cruse et al. (1996).

7.2 The advantages of positive feedback for control during stance The stance system, the second subsystem in the lowest control level considered here, has to solve the most difficult problem. In order that each leg contributes efficiently to support and propulsion and that the legs do not work at cross-purposes during walking, all legs contacting the ground at a given time have to move in such a way that no unwanted forces arise across the body. To this end, the movements of many joints (9-18 in an insect) have to be controlled, making a centralized controller to mediate the cooperation seem unavoidable at first sight. First, the task is complicated

262

H. Cruse and J. Dean

because it is rather nonlinear. Second, the number and combination of legs on the ground varies from one moment to another. Third, the joint axes are usually not orthogonal. Fourth, during curve walking, different legs move at different speeds. These problems could be solved by a central controller at the cost of great computational effort, but additional problems encountered in walking over natural substrates make a computational solution still more difficult if not impossible. For example, fifth, when the system walks on soft ground, each leg may move at a different speed in an unpredictable way. Sixth, the suspension of the legs may be not completely rigid, and therefore the geometry of the system may vary under different load conditions. Seventh, changes in geometry may also occur in living systems through growth, or in both living and artificial systems through injury or external damage. Solving this complex task, like the control of walking as a whole, appears to require quite a high level of "motor intelligence" and to involve complex global criteria. However, this performance does not necessarily imply the presence of a complicated, or even a centralized, control system. In contrast, our simulations have shown that an extremely decentralized control structure copes with all these problems and, at the same time, allows a very simple structure for the local controllers. Local control at the level of the individual joint, which essentially ignores interaction effects, is evident in the height control of the standing animal: the spring-like behavior of the whole leg essentially reflects the sum of the individual resistance reflexes at each of the three joints. In the extension to walking, joints continue to be governed by classical negative feedback systems providing proportional control, but some joints are also affected by a positive feedback loop via a band-pass filter. No neuronal connection between the joint controllers, even among those of the same leg, are necessary during stance. The coupling is simply provided by the physical connections of the joints. Thus, the system controlling the joint movements of the legs during stance is not only "intelligent" in terms of its behavioral properties, but also in terms of simplicity of construction: the most difficult problems are solved by the most decentralized control structure. Thus, the use of positive feeAback eliminates the need for extensive computation with multiple coordinate transformations. This simplification is possible because the physical properties of the system and its interaction with the world are exploited to replace an abstract, explicit computation. Thus, "the world" is used as "its own best model" (Brooks 1991). Due to its extremely decentralized organization and the simple type of calculation required, this method allows very fast computation which can be further accelerated by implementation on a parallel computer.

Motor bztelligence in a Sbnple Distributed Control System

263

Beyond this simplicity, the main advantage of a positive feedback controller is its robusmess with respect to all kinds of geometrical changes within the mechanical system. Changes in segment length or bending of a segment could occur by accident. Furthermore, a change in the orientation of a joint axis can occur during walking, which effectively introduces an additional joint into the equations. In the walking stick insect, for the example, the axis of the basal leg joint shows systematical changes in orientation during the step (Cruse & Bartling 1995). Whether these changes are passive or active is uncertain. In other animals, however, functional recruitment of additional joints has been observed. In crayfish and many other crustaceans, the leg has six joints but two (the ischiopodite-meropodite joint and the carpopoditepropodite joint) are usually kept immobile during walking. Only under special circumstances, such as in passing through a narrow gap, are these two joints actively moved. In such a case, a classical, algorithmic would require either a single controller which would always have to compute solutions for all six joints although only four joints are used most of the time, or, alternatively, one controller handling the four joints used in normal walking and another, more complicated one for using all six joints in emergencies. In the system based on local positive feedback, the same simple structure can control an arbitrary number of joints, so it suffices for both cases. In the crayfish, a supervisory system is necessary merely to fix the I-M joint and the C-P joint during normal walking by turning off the positive feedback. Two applications of positive feedback-in the selector net and in the stance control-have been discussed here. The use of positive feedback in the context of motor control has been considered previously with respect to recurrent circuits within the vertebrate brain (Houk et al. 1993) and in leech crawling (Baader & Kristan 1995). The interpretation advanced is that the positive feedback within motor and premotor centers serves to sustain ongoing activity. This basically corresponds to the simple circuit used in the model of Cruse et al. (1993a) to sustain power stroke or return stroke activity in the walking leg (selector net, Fig. 4b). Qualitatively, the effect is the same: ongoing patterns of movement are sustained through the recurrent connections. In both cases, sensory pathways are not part of the positive feedback although sensory inputs may effect changes from one pattern to another. The positive feedback influences the dynamics of transitions among patterns and the selection of one pattern over another, but it does not influence the quantitative parameters of the motor pattern. In our use of positive feedback to control leg movement during stance, in contrast, the positive feedback loop does include elements outside the nervous system, i.e., it includes the sensorimotor loop through the world, and it modulates the pattern itself by affecting activity levels in different joints.

264

H. Cruse and J. Dean

7.3 Incorporating biological principles into artificial systems In summary, the control structure presented here is a good example showing that the manner of implementation, ,,the how", plays a critical role in the functioning of the whole system. Explicit coordinate transformations of the type involved in the algorithmic forward and inverse kinematic calculations for multi-joint manipulators are not present, although the task performed by the targeting network in associating a final joint configuration for the leg in swing with the current joint configurations of the target leg can be regarded as an approximation to a sequence of coordinate transformations (direct kinematics, translation according to the displacement of the leg insertions, inverse kinematics). Computational maps are not necessary. Similarly, central pattern generators in the sense of a central oscillator providing a basic rhythm are not needed. The role of the central oscillator present in most rhythmic behaviors is to provide an internal temporal template, an approximate or ideal version of a motor pattern which may then be modulated by sensory inputs. However, in our model all is delegated to the real world. When such a model is adequate, it simplifies the design of an adaptive system because the construction of a central oscillator and the problem of coordinating its activity with peripheral influences are avoided. As discussed above, central oscillators, which enable a measure of feedforward, predictive control, may improve the performance of systems where conduction and processing delays are significant relative to the required response times. Similarly, computational maps have their merits and may be necessary to perform specific tasks. Given the proper architecture, they may introduce entirely new capabilities. Nevertheless, the success of the present model shows that many problems can be solved without relying on such internal models. Similarly, the work illustrates two characteristic features of biological control systems: the use of approximations, which give up precision in return for increased speed, and the reliance on multiple, redundant mechanisms, which provides robustness. The view of step coordination presented here-in which step patterns arise through interactions among separate rhythm generators-follows in the line of v. Hoist 1943, Wendler 1964, 1968, Wilson 1966, Graham 1972, Pearson 1993, B~sler 1977, Cruse 1980, 1985). This hierarchy and the hierarchy in the individual step pattern generators has parallels with the subsumption architecture applied to step control in a successful hexapod robot (Brooks 1986, 1989, Maes & Brooks 1990). It is still more similar to the subsequent

Motor h~telligence in a Simple Distributed Control System

265

behavior-based architecture developed by Maes (1991) which allows for less strictly hierarchical organizations. The control system presented here illustrates a high degree of ,,motor intelligence" in its adaptive behavior even though it contains no structural plasticity. The weights in the state selector net, the swing net and a network for one coordinating influence (the targeting behavior) as well as the strengths of the coordinating influences between the legs were trained with backpropagation or optimized using random search or a genetic algorithm (e.g., Cruse et al. 1995c, Dean 1990), but after these "off-line" learning procedures are finished, the complete system contains no variable weights. Thus, it can be considered to be "hard-wired". Nevertheless, responds adaptively to disturbances and to changes in the environment and in the system itself. Other researchers have also used artificial neural networks to control hexapod walking (Beer et al. 1989, 1992). Beer and ~ a g h e r (1992) have also used genetic algorithms to successfully train intra- and interleg coordination under conditions where the tripod gait is the gait of choice, but the geometry of the legs was simplified compared to that of insects. Another approach to using artificial neural nets to control the six legs of a walking system is given by Bems et al. (1994). They use Elman-type nets to control swing and stance movements. The coordination is done by a central system allowing only a strict tripod gait (Betas 1994). In contrast, our decentralized approach allows variable and adaptive gait patterns, not simply tripod coordination. Our rationale for this is, first of all, that our original goal was to simulate the behavior and the known structure of the stick insect. Second, we feel that this decentmlizexl approach, together with the exploitation of the dynamics of the interaction with the environment, gives the system greater robusmess and adaptability which permits more stable behavior in difficult situations. Acknowledgments The authors were supported by the Deutsche Forschungsgemeinschaft (Cr 58/8) and the Bundesministerium f'tir Forschung und Technik (01 IN 104B/l). References

Baader, A.P. & Kristan, W.B., Jr. (1995). Parallel pathways coordinate crawling in the medicinal leech, Hirudo medicinalis, Journal of comparat&e Physiology A 176: 715-726.

266

H. Cruse and J. Dean

B~sler, U. (1965). Proprioreceptoren am Subcoxal- und Femur-TibiaGelenk der Stabheuschrecke und ihre Rolle bei der Wahmehmung der Schwerkraftrichtung, Kybernetik 2:168-193. B~sler, U. (1972). Zur Beeinflussung der Bewegungsweise eines Beines von Carausius morosus durch Amputation anderer Beine, Kybernetik 10: 110-114. B~sler, U. (1976). Reversal of a reflex to a single motor neurone in the stick insect Carausius morosus, Biological Cybernetics 24: 47-49. B~ssler, U. (1977). Sensory control of leg movement in the stick insect Carausius morosus, Biological Cybernetics 25:61-72. B~ssler, U. (1983). Neural basis of elementary behavior in stick insects, Springer-Verlag. B~sler, U. (1986a). On the definition of central pattern generator and its sensory control, Biological Cybernetics 54: 65-69. Bassler, U. (1986b). Afferent control of walking movements in the stick insect Cuniculbza impigra. IT. Reflex reversal and the release of the swing phase in the restrained foreleg, Journal of comparative Physiolog3' 158:351-362. B~ssler, U. (1988). Functional principles of pattem generation for walking movements of stick insect forelegs: the role of the femoral chordotonal organ afferences, Journal of experimental Biology 136: 125-147. B~sler, U. (1993). The femur-tibia control system of stick insects - a model system for the study of the neural basis of joint control, Brain Research Reviews 18: 207-226. B~sler, U. & Wegner, U. (1983). Motor output of the denervated thoracic ventral nerve cord in the stick insect Carausius morosus, Journal of experimental Biology 105:127-145. Beer, R.D., Chiel, H.J. & Sterling, L.S. (1989). Heterogeneous neural networks for adaptive behavior in dynamic environments, in D.S. Touretzky (ed.) Advances in Neural Information Processing Systems 1, Morgan Kaufmann, pp. 577-585. Beer, R.D., Chiel, H.J., Quinn, R.D., Espenschied, K.S. & Larsson, P. (1992). A distributed neural network architecture for hexapod robot locomotion, Neural Computation 4: 356-365. Beer, R.D. & Crallagher, J.C. (1992). Evolving dynamical neural networks for adaptive behavior, Adaptive Behavior 1: 91-122. Betas, K. (1994). Steuerungsansfitze auf der Basis neuronaler Netze fiir sechsbeflzige Laufmaschbzen, PhD Thesis, Universit~t Karlsruhe. Betas, K., Piekenbrock, St. & Dillmann, R. (1994). Learning control of a sixlegged walking machine, in M. Jamashidi, Ch. Nguyen, R. Lumia & J.

Motor b~telligence in a Sbnple Distributed Control System

267

Yuh (eds.), Proceedings of the 5th International Symposium on Robotics and Manufacturing, Vol. 5, ASME Press, pp. 29-34. Bizzi, E., Hogan, N., Mussa-Ivaldi, F.A. & Giszter, S. (1992). Does the nervous system use equilibrium-point control to guide single and multiple joint movements? Behavioral and Brain Sciences 15:603-613. Brooks, R.A. (1986). A robust layered control system for a mobile robot, IEEE Journal of Robotics and Automation RA-2: 14-23. Brooks, R.A. (1989). A robot that walks: emergent behavior from a carefully evolved network, Neural Computation 1: 253-262. Brooks, R.A. (1991). Intelligence without reason, in J. Mylopoulos & R. Reiter (eds.) Proceedings of the 12th international jobzt conference on artificial intelligence (IJCAI-91), Morgan Kaufmann, pp. 569-595. Brown, T.G. (1911). The intrinsic factors in the act of progression in the mammal, Proceedings of the Royal Society 84B: 308-319. Brunn, D. & Dean, J. (1994). Intersegmental and local intemeurones in the metathorax of the stick insect, Carausius morosus, Journal of Neurophysiology 72:1208-1219. Btischges, A. & Schmitz, J. (1991) Nonspiking pathways antagonize the resistance reflex in the thoraco-coxal joint of stick insects, Journal of Neurobiology 22: 224-237. Btischges, A., Schmitz, J. & B~issler, U. (1995) Rhythmic patterns in the thoracic nerve cord of the stick insect induced by pilocarpine, Journal of experimental Biology 198: 435-456. Burrows, M. (1992). Local circuits for the control of leg movements in an insect, Trends in Neurosciences 15: 226-232. Cruse, H. (1976). The control of the body position in the stick insect (Carausius morosus), when walking over uneven surfaces, Biological Cybernetics 24: 25-33. Cruse, H. (1979). The control of the anterior extreme position of the hindleg of a walking insect, Ph~'siological Entomology 4: 121-124. Cruse, H. (1980). A quantitative model of walking incorporating central and peripheral influences. II. The connections between the different legs, Biological Cybernetics 37: 137-144. Cruse, H. (1983). The influence of load and leg amputation upon coordination in walking crustaceans: A model calculation, Biological Cybernetics 49: 119-125. Cruse, H. (1985). Which parameters control the leg movement of a walking insect? II. The start of the swing phase, Journal of experimental Biology 116: 357-362.

268

H. Cruse and J. Dean

Cruse, H. (1990). What mechanisms coordinate leg movement in walking arthropods? Trends in Neurosciences 13: 15-21. Cruse, H. & Bartling, Ch. (1995). Movement of joint angles in the legs of a walking insect, Carausius morosus, Journal of lnsect Physiology 41: 761771. Cruse, H., Bartling, Ch., Cymbalyuk, G., Dean, J. & Dreifert, M. (1995a). A modular artificial neural net for controlling a six-legged walking system, Biological Cybernetics 72:421-430. Cruse, H., Bartling, Ch. & Kindermann, T. (1995b). High-pass filtered positive feedback for decentralized control of cooperation, in F. Moran, A. Moreno, J.J. Merelo & P. Chacon (eds.), Advaluzes b~ Artificial Life, Springer-Verlag, pp. 668-678. Cruse, H., Barfling, C., Brunn, D.E., Dean, J., Dreifert, M., Kindermann, T. & Schmitz, J. (1995c). Walking: a complex behavior controlled by simple systems, Adaptive Behavior 3:385-418. Cruse, H., Barfling, C., Dean, J., Kindermann, T., Schmitz, J., Schumm, M. & Wagner, H. (1996). Coordination in a six-legged walking system. Simple solutions to complex problems by exploitation of physical properties, in P. Maes, J.A. Meyer & S.W. Wilson (eds.), From animals to animats 4, M1T Press (in press). Cruse, H., Dautenhahn, K. & Schreiner, H. (1992). Coactivation of leg reflexes in the stick insect, Biological Cybernetics 67: 369-375. Cruse, H., Miiller-Wilm, U. & Dean, J. (1993a). Artificial neural nets for a 6legged walking system, in J.A. Meyer, H.L. Roitblat & S.W. Wilson (eds.), From animals to an/mats 2, MIT Press, pp. 52-60. Cruse, H., Riemenschneider, D. & Stammer, W. (1989). Control of body position of a stick insect standing on uneven surfaces, Biological ~'bemetics 61:71-77. Cruse, H., Schmitz, J., Braun, U. & Schweins, A. (1993b) Control of body height in a stick insect walking on a treadwheel, Jounzal of experimental Biology 181: 141-155. Dean, J. (1990). Coding proprioceptive information to control movement to a target: simulation with a simple neural network, Biological Cybernetics 63: 115-120. Dean, J. (199 l a). Effect of load on leg movement and step coordination of the stick insect Carausius morosus, Journal of experimental Biology 159: 449471. Dean, J. (1991b). A model of leg coordination in the stick insect, Carausius morosus. I. A geometrical consideration of contralateral and ipsilateral

Motor bztelligence in a Sbnple Distributed Control System

269

coordination mechanisms between two adjacent legs, Biological Cybernetics 64: 393-402. Dean, J. (1991c). A model of leg coordination in the stick insect, Carausius morosus. II. Description of the kinematic model and simulation of normal step pattem, Biological Cybernetics 64:403-411. Dean, J. (1992a). A model of leg coordination in the stick insect, Carausius morosus. M. Responses to perturbations of normal coordination, Biological Cybernetics 66: 335-343. Dean, J. (1992b). A model of leg coordination in the stick insect, Carausius morosus. IV. Comparison of different forms of coordinating mechanisms, Biological Cybernetics 66: 345-355. Dean, J. & Cruse, H. (1995). Motor Pattern Generation, in M. Arbib (ed.) The Handbook of Brain Theory and Neural Networks, MIT Press, pp. 600-605. Dean, J. & Wendler, G. (1982). Stick insects walking on a wheel: perturbations induced by obstruction of leg protraction, Journal of Comparative Physiology 148: 195-207. Dean, J. & Wendler, G. (1983). Stick insects walking on a wheel: Interleg coordination of leg position, Journal of experimental Biology 103: 75-94. Eltze, J. (1995). Biologisch orientierte Entwicklung einer sechsbeinigen Laufmaschine, Fortschrittberichte VDI Reihe 17, Nr. 10, VDI Verlag, Diisseldorf. Gibson, J.J. (1966). The senses considered as perceptual systems, Houghton Mifflin. Giszter, S.F., Mussa-Ivaldi, F.A. & B izzi, E. (1993). Convergent force fields organized in the frog spinal cord. Journal of Neuroscience 13: 467-491. Graham, D. (1972). A behavioural analysis of the temporal organisation of walking movements in the 1st instar and adult stick insect, Journal of Comparative Physiology 81: 23-52. Graham, D. (1985). Pattern and control of walking in insects, Advances in Insect Physiology 18:31-140. Hoist, E. von (1943). Uber relative Koordination bei Arthropoden, Pfliigers Archiv 246: 847-865. Houk, J.C., Keifer, J. & Barto, A.G. (1993). Distributed motor commands in the limb premotor network, Trends in Neurosciences 16: 27-33. Kristan, W.B., Jr., Ia~ckery, S.R., Wittenberg, G. & Brody, D. (1992). Making behavioral choices with intemeurons in a distributed system, in J. Kien, C.R. McCrohan & W. Winlow (eds.) Neurobiology of motor programme selection, Pergamon Press, pp. 170-200.

270

H. Cruseand J. Dean

Maes, P. (1991). A bottom-up mechanism for behavior selection in an artificial creature, in J.A. Meyer & S.W. Wilson (eds.), From animals to animats, MIT Press, pp. 238-246. Maes, P. & Brooks, R.A. (1990). Learning to coordinate behaviors, Proceedings of the Eighth National Conference on Artificial Intelligence (AAAI-90), pp. 796-802. Minsky, M. (1985). The Society of Mind, Simon and Schuster. Mtiller-Wilm, U., Dean, J,. Cruse, H., Weidemann, H.J., Eltze, J. & Pfeiffer, E (1992). Kinematic model of a stick insect as an example of a 6-legged walking system, Adaptive Behavior 1: 33-46. Pearson, K.G. (1972). Central programming and reflex control of walking in the cockroach, Journal of experimental Biology 56: 173-193. Pearson, K.G. (1993). Common principles of motor control in vertebrates and invertebrates, Annual Review of Neuroscience 16: 265-297. Pfeiffer, E, Weidemann, H.J. & Danowski, P. (1991). Dynamics of the walking stick insect, IEEE Control systems 11: 9-13. Pfeiffer, F., Eltze, J. & Weidemann, H.J. (1994). The TUM walking machine, in M. Jamashidi, J. Yuh, Ch. Nguyen & R. Lumia (eds.), Proceedings of the 5th hzternational Symposium on Robotics alut Manufacturing, Vol. 2, ASME Press, pp. 167-174. Schmitz, J. (1993). Load-compensation reactions in the proximal leg joints of stick insects during standing and walking, Journal of experimental Biology 183: 15-33. Schmitz, J., Battling, C., Brunn, D.E., Cruse, H., Dean, J., Kindermann, T., Schumm, M. & Wagner, H. (1995). Adaptive properties of "hard-wired" neuronal systems, Verhandlungen der Deutschen Zoologischen Gesellschaft 88.2:165-179. Schmitz, J. & HaBfeld, G. (1989). The treading-on-tarsus reflex in stick insects: phase dependence and modifications of the motor output during walking, Journal of experimental Biology 143: 373-388. Wendler, G. (1964). Laufen und Stehen der Stabheuschrecke: Sinnesborsten in den Beingelenken als Glieder yon Regelkreisen, Zeitschrifi ~ r vergleichende Physiologie 48: 198-250. Wendler, G. (1968). Ein Analogmodell der Beinbewegungen eines laufenden Insekts, in H. Marko & G. Faerber (eds.), Kybemetik 1968, Oldenbourg, pp. 68-74. Wilson, D.M. (1966). Insect walking, Annual Review of Entomology 11: 103-122.